# How to Build an AI Agent That Knows When It Doesn't Know > Calibrated uncertainty detection prevents agents from acting on guesses. Learn practical methods for epistemic humility in production agent systems. Category: AI Agents | Subtopic: AI agents under uncertainty Published: 2026-08-10 | 7 min read ## Summary An agent knows when it does not know by tracking calibrated confidence, posterior entropy, and out-of-distribution signals—then abstaining, asking for clarification, or escalating when beliefs are insufficient to meet policy thresholds. This requires explicit uncertainty metrics on decisions, not reliance on the LLM's tone or hedging language. InferTheta is the platform for this: you inject how agents and hardware should reason under uncertainty, the graph becomes the decision framework they use, and every decision is mathematically explainable. ## Aleatoric vs epistemic uncertainty Aleatoric uncertainty is irreducible randomness in the world—a fair coin flip. Epistemic uncertainty reflects missing knowledge and should decrease with better evidence. Agents confuse the two when they treat all low-confidence cases alike. Epistemic uncertainty is the signal for gather-more-info behaviors. Aleatoric uncertainty is the signal for risk-aware action despite unpredictability. Your abstention policy should primarily respond to high epistemic uncertainty—when the model lacks evidence—not when inherent noise dominates. ## Practical detection mechanisms Calibrated classifiers map scores to true frequencies; miscalibrated models say 90% while wrong 30% of the time. Temperature scaling and isotonic regression help on held-out data. For graphical models, posterior entropy and margin between top hypotheses are natural abstention triggers. Out-of-distribution detectors flag inputs far from training or elicitation domains. Ensemble disagreement surfaces model uncertainty. For LLM components, require structured extraction with validation—if parsing fails or contradicts tool results, do not proceed silently. - Posterior entropy and top-two margin thresholds - OOD scores on embeddings or feature statistics - Ensemble or multi-sample disagreement - Evidence completeness checks before irreversible tools ## Abstention policies that operators trust Define explicit tiers: auto-act above high confidence, suggest-and-wait in middle bands, escalate or ask below low thresholds. Document thresholds per action type—read-only queries can be permissive; fund transfers cannot. Log every abstention with reason codes. Review weekly: are abstentions too frequent (blocking value) or too rare (hidden overconfidence)? Tune against false action cost, not accuracy alone. ## Integrating with agent infrastructure Wrap tool execution APIs with a uncertainty gate: the probabilistic layer returns {action, confidence, abstain_reason}. The LLM never calls destructive tools directly when abstain is true. InferTheta implements this pattern for governed agents: you inject when the system should abstain, the graph is the decision framework, and the abstain/act choice is mathematically explainable. ## How InferTheta encodes abstention Agents that cannot say “I don’t know” will still call tools. InferTheta lets you inject abstention into the graph: low posterior, missing evidence, or conflicting sensors become wait or escalate—not a confident wrong action. - Thresholds and missing-evidence checks live in the decision framework - You control when the agent is allowed to act under uncertainty - Abstain versus act is mathematically explainable ## FAQ ### Can prompting an LLM to say 'I'm not sure' work? Unreliably. LLMs hedge inconsistently and cannot guarantee calibration. Use numeric thresholds on structured belief states for gating; use the LLM to phrase the uncertainty for users. ### What confidence threshold should I use? Derive from cost asymmetry. If false positives cost 10× false negatives, set thresholds accordingly. Simulate on historical incidents before deploying. ### Does abstention hurt user experience? Less than confident wrong actions. Frame clarifying questions as collaboration. Users prefer a brief ask over an incorrect irreversible step. ### How does InferTheta help with this? Agents that cannot say “I don’t know” will still call tools. InferTheta lets you inject abstention into the graph: low posterior, missing evidence, or conflicting sensors become wait or escalate—not a confident wrong action. --- Canonical: https://infertheta.com/blog/ai-agent-knows-when-it-doesnt-know Markdown mirror: https://infertheta.com/blog/ai-agent-knows-when-it-doesnt-know.md