Limitations
What AI gets wrong, and why confidence is not accuracy
The fine print
Fluency is not understanding. Systems can invent facts, follow hostile instructions in documents, agree when you are wrong, and botch arithmetic without tools. Use these demos as literacy, not fear.
Lab honesty
Scenarios are hand-authored teaching fixtures, not live model runs. The “safe” injection reply is the product behavior we want to teach, not a guarantee of every deployed model. Math error numbers are fixed examples to show the tool fix path.
Failure modes
Confident answers, unequal truth
1 / 3Which is the real capital of Australia?
Both options can be stated with equal confidence. Which is real?
Key Insight
Pair models with retrieval, tools, human review, and evals. Ask what was retrieved, which tools ran, and whether the answer is checkable. Next: history and model choice for real-world tradeoffs.
See also: Unhobbling
Autonomy without an evaluator invites silent success(polite “done” while tests fail). Unhobbling adds loops and harnesses; it does not remove review or sandboxes. Continue in Unhobbling AI.
Check your understanding
1/4What is an AI hallucination?
Sources, scope & review statusVerified Jul 29, 2026
Claims this lesson makes
- Stable conceptFluent output is not evidence of factual accuracy; deployments need evaluation, monitoring, and risk controls.
Primary reading
Next content review: 2027-01-29. Source links establish the lesson's claims; interactive numbers remain labeled simulations unless explicitly identified as measured data.