用語集 · 15 terms
A piece of text that the model processes as a unit.
Engineer translation: The atomic unit of model input/output. Billing, context limits, and latency all operate on token counts.
First appears: Lesson 00a · Related: Tokenization, Context window
A raw score for each possible next token. Not a probability.
Engineer translation: Unnormalized output of the model's final layer, before softmax.
First appears: Lesson 01 · Related: Softmax, Decoding
Converts raw scores into probabilities that sum to 1.
Engineer translation: exp(z_i) / sum(exp(z_j)). Applied to logits.
First appears: Lesson 05 · Related: Logit, Temperature
Reshapes probability distribution before selection. Lower = predictable, higher = varied.
Engineer translation: Divides logits before softmax. T approaches 0 approaches greedy; most APIs treat 0 as greedy mode. Does not improve factuality.
First appears: Lesson 05 · Related: Softmax, Top-k, Top-p
Maximum tokens the model can process in one request.
Engineer translation: Fixed by architecture and training. Everything for one inference must fit.
First appears: Lesson 00c · Related: Token, KV cache
Temporary store of computed attention state to avoid reprocessing.
Engineer translation: Each layer retains key/value tensors for prior tokens. Grows with sequence length.
First appears: Lesson 01 · Related: Context window, Attention
A vector representing meaning. Similar meanings are nearby.
Engineer translation: Learned mapping from input to high-dimensional vector space.
First appears: Lesson 00b · Related: Cosine similarity, RAG
Directional similarity between two vectors, ignoring magnitude. Range -1 to 1.
Engineer translation: dot(A,B) / (||A|| * ||B||). Not a probability of correctness.
First appears: Lesson 04 · Related: Embedding, Vector search
Fluent, confident output that is factually wrong or unsupported.
Engineer translation: Catch-all for factual error, unsupported claim, unfaithfulness. Use precise metrics.
First appears: Lesson 06 · Related: RAG, Citation
Finding external evidence and adding it to context before generating.
Engineer translation: Pipeline: ingest, embed, retrieve, authorize, generate, verify. Does not modify weights.
First appears: Lesson 07 · Related: Embedding, Hallucination
Attack where untrusted text overrides trusted instructions.
Engineer translation: OWASP LLM01:2025. Model cannot distinguish instructions from data.
First appears: Lesson 11 · Related: Tool calling, Authorization
Permanently adjusting model weights to change behavior.
Engineer translation: Continued training on curated data. Teaches patterns, not facts.
First appears: Lesson 08 · Related: Pretraining, RAG
Measuring whether output meets your quality bar.
Engineer translation: Versioned dataset + deterministic checks + model graders + human review.
First appears: Lesson 09 · Related: Testing, Model grader
A system where the model decides what to do next.
Engineer translation: Loop with explicit state machine, tool policies, budgets, stop conditions.
First appears: Lesson 12 · Related: Tool calling, Bounded autonomy
Mechanism for looking at relationships between all tokens simultaneously.
Engineer translation: query × key → weight; weight × value → output. Learned, dynamic, quadratic scaling.
First appears: Lesson 00b · Related: Transformer, Embedding
