Better with boundaries: How governed knowledge can build trust in AI
Federal leaders need confidence in the outputs AI systems produce. LLMs and agents can produce responses that are fluent and specific but incorrect. At machine speed, those errors can scale quickly, creating operational and compliance risks faster than they can be caught.
That makes defensibility particularly important in high-stakes, regulated contexts. Leaders often address this risk with oversight measures, such as review boards, approvals, and governance processes. Those safeguards matter, but they sit outside the AI implementation rather than being engineered into the system itself.
Agency leaders can build trust in AI proactively by engineering it into the system. One approach is guardrailed AI, which adds a reasoning layer over an LLM that constrains and verifies its outputs against a rule set. Another approach is to restrict an AI tool’s access to a secure, authoritative knowledge base, so that its answers are grounded only in approved sources.
When an AI solution is bounded by a specific knowledge base, the system itself becomes a governance tool. If it draws only on an organization’s existing, approved content, it can deliver answers quickly without generating unsupported material that must then be reviewed and governed.
AI accelerates answers for proposal applicants
A recent AI proposal advisor tool we built to help the National Science Foundation (NSF) scale applicant support shows how this approach can work. NSF administers research grants to support small business innovation, and applicants to those funds must provide detailed, complex proposals to be considered. Traditionally, when applicants had questions about the process, a human program director had to answer, creating a heavy burden that could lead to slow response times and unanswered questions.
We designed and implemented an AI-enabled proposal advisor tool to help relieve that burden. The tool, which appears on NSF’s Small Businesses home page, is trained on a knowledge base that’s restricted to NSF’s proposal and award policies. Because the tool relies only on content in the knowledge base, it prioritizes accuracy and reduces hallucination risk.
In early prototype testing, the AI proposal advisor tool provided relevant guidance 86% of the time. When a question fell outside of the knowledge base’s scope, the tool gave no answer rather than making up an inaccurate response. That guardrail is a design choice built into the tool, not a manual check at the end.
The approach offers two advantages:
- Time saved: Staff who previously needed to hunt for answers to policy questions can find them faster. Their work can now focus on solicitations, review panels, and portfolio oversight instead of routine inquiries.
- Increased trust: Because the solution’s answers are constrained to approved content from the start, staff and leaders have greater assurance that responses are grounded in authoritative NSF information.
Use bounded knowledge to speed onboarding
AI that works from a bounded knowledge base can bring efficiencies to internal project management, too. At ICF, we developed an AI tool to help team members onboard to complex projects more quickly.
For new team members, learning a role can be difficult when training information is fragmented. When project managers and leads spend time answering repetitive questions, it takes their attention away from mission-critical tasks. But without access to the information they need, new team members may make mistakes that surface later as delivery risks.
The AI tool we developed has access to a restricted set of project-specific documents. New team members can ask plain-language questions and retrieve project-specific information without asking project managers or leads. In early testing, the tool sped up onboarding by as much as 40%.
AI defensibility by design
These examples demonstrate the value of designing AI systems with defined roles and bounded knowledge from the start. In federal environments, that approach can strengthen trust and defensibility. The key is not expanding what an AI system can generate but giving it boundaries that it can confidently operate within.
For high-stakes applications, knowing when not to answer can be as important as knowing what to say. The stronger AI strategy is to define those boundaries before deployment.