Demos are built to impress, and they usually run on the vendor’s own examples. Before I spend an hour watching one, I want written answers to five questions. None of them is technical, but each one affects whether a product can be used in government at all. They apply to any public agency and aren’t about a particular purchase.
1. What happens to our data?
Is anything we put in used to train or improve the product? Where is it stored and processed, and by which subcontractors? “We don’t train on your data” is a different promise from “we don’t keep your data.”
2. What do you keep, and for how long?
Prompts, outputs and logs can be public records.1 The agency’s retention schedule sets how long records are kept. The vendor’s default settings don’t change that. Can the product keep what we’re required to keep, delete what we’re allowed to delete and preserve what a legal hold requires?
3. Who can see what?
Does the tool respect the permissions we already have? Or can anyone who can type a question see every document it can search?
4. What can change without our knowing?
When the vendor updates the model behind the product, do we hear about it first? Can we test the change before it reaches staff or residents? And what happens if that model goes away? In June 2026, a federal export-control directive led Anthropic to suspend access to two of its frontier models for 19 days.2 A product should keep working, or hand the work back to people, when its model disappears.
5. How do we leave?
Can we export our documents, settings and history in a format we can actually use? What does the contract say happens when it ends?
And one for the demo itself
Can we run it on our own examples, including the hard ones? A product that handles a clean sample may stumble on a scanned resolution from the 1990s or a form filled in by hand.
These questions are a place to start. They aren’t a legal checklist. Purchasing rules, privacy law and records schedules set the actual requirements, and the people responsible for them have the final word. NIST’s voluntary AI Risk Management Framework3 and the policy, vendor agreement and incident response templates the GovAI Coalition shares with public agencies4 are good places to look for more questions.
Sources
-
New Jersey Division of Revenue and Enterprise Services, Guidelines for Developing Retention and Disposition Policies for Artificial Intelligence/Machine Learning Systems, February 2025, revised July 2025. Accessed September 24, 2026. ↑
-
Anthropic, statement on access to Claude Fable 5 and Claude Mythos 5, June 12, 2026, and CNBC, June 30, 2026, on the controls being lifted. Accessed September 26, 2026. ↑
-
National Institute of Standards and Technology, AI Risk Management Framework (AI RMF 1.0), January 2023, intended for voluntary use. Accessed September 24, 2026. ↑
-
Digital Government Hub, GovAI Coalition Templates and Resources. Accessed September 24, 2026. ↑