Much of municipal work comes down to what the record says. When was this approved? Which version of the ordinance applies? Did the board attach a condition? The answer has to come from a specific document, and someone has to be able to find that document again.
That’s why I care more about where an answer came from than about how well it’s written.
A worked example
The details are illustrative, not drawn from any real municipality or ordinance.
Someone asks, “How tall can a fence be in a front yard?” A tool answers smoothly: “Four feet, under the fence ordinance adopted in 2015.” It sounds right. But suppose the governing body amended that ordinance in 2021 and changed the limit to three and a half feet. The 2015 text is still in the archive, so the tool found a real document. It found the wrong version, and it didn’t say so.
Nothing in that answer was invented, and it’s still wrong. A resident who relied on it could build a fence that’s too tall.
NIST’s profile for generative AI calls the broader problem confabulation, in which systems “generate and confidently present erroneous or false content in response to prompts.”1 That confidence is what makes it risky. A clumsy wrong answer usually gets a second look, but a well-written one is easy to accept.
What a trustworthy answer includes
For a question like that one, a trustworthy answer shows:
- the document itself, linked to the original, not just a summary of it;
- the page or section the answer came from;
- the document’s date and status: adopted or draft, current or superseded;
- what was searched and what wasn’t, so that “nothing found” means something;
- a plain “I can’t answer that from these records” when the records don’t settle it.
The last item matters most. “No match found” is a search result. It doesn’t prove that something never happened. A tool that can’t tell the difference will eventually tell someone that a condition doesn’t exist when it’s really in a file nobody indexed.
What I measure first
When I evaluate a search tool on municipal records, speed is the last thing I count. First I check how often the cited source actually supports the answer, how often the tool finds the right version and how often it correctly says it doesn’t know. Then I compare it with a good ordinary search, since beating a slow one proves little. An answer that’s fast and unsupported doesn’t save anyone time. It moves the checking to someone else.
Sources
-
National Institute of Standards and Technology, Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile (NIST AI 600-1), July 2024. Accessed September 24, 2026. ↑