Skip to lesson content
Practical LLM Systems Guide

型 · 6 implementation references

Patterns

Token budget calculator

Problem: You need to know how many tokens a prompt component consumes.

Solution: Tokenize each component separately. Sum and compare against the context limit. Never estimate from character counts.

From Lesson 02

Explicit overflow policy

Problem: When the context window overflows, the request may be rejected or truncated.

Solution: Define what is protected, what is trimmed, what is summarized. Never rely on implicit truncation.

From Lesson 02

Retrieval quality measurement

Problem: RAG answers are wrong and you cannot tell if retrieval or generation failed.

Solution: Maintain a labeled eval set. Measure retrieval precision/recall separately from answer faithfulness.

From Lesson 07

Citation verification

Problem: The model cites sources that may not support the claims.

Solution: Check programmatically: cited passage exists, requester is authorized, passage supports the claim.

From Lesson 07

Per-action tool authorization

Problem: The model requests a tool call the user should not execute.

Solution: Every tool call passes through application-layer authorization. The model proposes; the application disposes.

From Lesson 11

Agent state machine with stop conditions

Problem: An agent loops indefinitely or takes harmful actions.

Solution: Explicit state machine with step limit, cost ceiling, success criterion, human approval for irreversible actions.

From Lesson 12