Tool

Vendor question sheet.

Pick the kind of product you’re looking at, and print the questions to hand the vendor before the demo. Ask for the answers in writing.

What kind of product is it?

Questions for the vendor

Please answer each question in writing before the demo.

For every product

  1. What happens to our data? Is anything we put in used to train or improve the product? Where is it stored and processed, and by which subcontractors?

  2. What do you keep, and for how long? Can the product keep what we’re required to keep, delete what we’re allowed to delete and preserve what a legal hold requires?

  3. Who can see what? Does the tool respect the permissions we already have, or can anyone who can type a question see every document it can search?

  4. What can change without our knowing? When the model behind the product changes, do we hear first, and can we test the change before it reaches staff or residents? What happens if that model goes away?

  5. How do we leave? Can we export our documents, settings and history in a format we can use? What does the contract say happens when it ends?

  6. Can we run the demo on our own examples? Including hard ones, such as a scanned resolution from the 1990s or a form filled in by hand.

For meeting minutes

  1. How does it record each action taken and the vote of each member, which the Open Public Meetings Act requires the minutes to show?

  2. What does it do with a vote it couldn’t hear or a speaker it couldn’t identify: flag it, or guess?

  3. Can it leave out a closed session, and how do we check that it did?

  4. Which of the recording, the transcript and the draft does it keep, where and for how long?

  5. Can the clerk edit the draft in our own format before it goes to the board for approval?

  1. Does every answer cite the document and the page it came from?

  2. How does it handle a superseded version, such as an amended ordinance or a revised resolution?

  3. What does it say when our records don’t answer the question?

  4. Which records does it index, and how does it keep out the ones staff may not share?

  5. Where is the index stored, and who can search it?

For a resident chatbot

  1. How does it tell residents they’re talking to AI, and when?

  2. How does it hand a resident to a person, and how fast?

  3. What happens to the personal details residents type, such as addresses and phone numbers?

  4. Is every conversation kept, and could we produce one in answer to a public records request?

  5. How does it answer a question that belongs to the county or the State?

For AI in office software

  1. Which of our data can it reach: mail, files, chats, calendars?

  2. Are prompts and responses kept, and where would they turn up in a records search?

  3. Can we switch it off for some users, or for some kinds of data?

For a redaction tool

  1. How does it find home addresses Daniel’s Law protects, and the personal details the Open Public Records Act keeps out of a release?

  2. How often does it miss something, and how was that measured?

  3. Does it remove the text itself, so nothing can be copied out of the released file, or only cover it?

  4. Does a person review every redaction before release?

  5. Does it log what was redacted and why, so the custodian can state the basis for each one?