Hallucinations and reliability boundaries

Section 01

Fluent, confident, and wrong

An LLM generates text by predicting the most likely next token. Nothing in that process checks whether the prediction is true. The result is a system that produces fluent, confident-sounding output that may be factually wrong, unsupported by evidence, internally contradictory, or irrelevant to the question.

This is not a bug. It is the direct consequence of the training objective. The model learned to produce plausible continuations — not to verify facts. Understanding the failure modes, where they originate, and what controls can mitigate them is the foundation for building reliable LLM systems.

Section 02

The polished joint with a hidden crack

Two handcrafted joints sit on an inspection bench. They look identical — polished, well-fitted, expertly finished. But one has a hairline crack running through its core, invisible under normal light, revealed only when raking light catches the fracture at the right angle.

LLM output is like those joints. A well-formed, fluent response and a factually wrong response look identical on the surface. Fluency and confidence do not distinguish them. You need a reference gauge — external evidence, a fact-check, a test — to tell them apart.

Where the analogy breaks: the crack in a joint is a single defect. LLM failures overlap and interact. A single response can contain a factual error, an unsupported claim, a contradiction with earlier context, and a citation failure — all at once. The failure categories are not mutually exclusive.

Section 03

Categories of unsupported output

Use precise product metrics instead of the vague term 'hallucination.' The following categories overlap but help locate where a failure originates:

• Factual inaccuracy: the statement is wrong (wrong date, wrong API name, wrong dosage).
• Unsupported/extrinsic claim: the statement may be true but is not grounded in the provided context or the model's training data.
• Contradiction with context (intrinsic unfaithfulness): the response contradicts information supplied in the prompt or retrieved evidence.
• Nonsensical or incoherent: the response is internally contradictory or meaningless.
• Instruction unfaithfulness: the response ignores or violates explicit instructions (wrong format, wrong language, wrong scope).
• Irrelevant: the response does not address the question.
• Citation failure: the fact is correct but attributed to the wrong source, or the cited source does not support the claim.

Section 04

Where failures originate

Failures can enter at multiple stages of the pipeline:

• Data: training data is incomplete, stale, contradictory, or noisy.
• Retrieval: evidence is omitted, mis-ranked, or poisoned.
• Prompt: instructions are ambiguous, conflicting, or missing context.
• Generation: the model's sampling produces a plausible-but-wrong continuation.
• Validation: no check catches the error before it reaches the user.

Section 05

Layered mitigations by consequence

No single control eliminates unsupported output. Effective systems layer multiple mitigations, each addressing a different failure category:

• Retrieval (Lesson 07): ground generation in verified external evidence.
• Citations with verification: require the model to cite sources, then programmatically check that the cited passage supports the claim.
• Lower temperature: reduce variance for tasks where consistency matters (but this does not improve factuality).
• Cross-checking: ask the model to verify its own claims against retrieved evidence in a separate call.
• Guardrails: detect and filter known failure patterns (PII, toxic content, specific hallucination signatures).
• Human review: for high-consequence outputs, require human approval before delivery.

Section 06

What does not fix hallucinations

Lower temperature reduces variance. It does not supply missing knowledge. A model at temperature 0 will confidently and consistently produce the same wrong answer.

RAG adds evidence to the context. It does not force the model to use it. A model can retrieve relevant evidence and then ignore it, or cite a source that does not support its claim.

Citations provide traceability. They do not guarantee faithfulness. The model may cite a real source for a claim the source does not make.

None of these controls independently establishes truth. Each addresses a specific failure category. Stack them proportional to consequence.

Section 07

Unscored lab: classify and trace failures

Take 10 outputs from an LLM on factual questions. For each output, classify any failures using the categories from this lesson.

1. Generate 10 responses to factual questions (history, science, API documentation).
2. For each response, identify whether it contains any failure.
3. Classify each failure: factual error, unsupported claim, contradiction, instruction violation, citation failure, or irrelevant.
4. For each failure, trace the origin: was it data, retrieval, prompt, generation, or validation?
5. For each failure, propose a mitigation: what control would have caught it?
6. Count: how many failures would each mitigation catch if applied alone? How many require multiple layers?