M01-L02 — Tokens, context, and generation
Outcome: Explain why repeated requests can differ and why selected context helps.
Start: M01-L01 and the original BenchNote passage in Chapter 2.
Tokens are units used to represent model input and output. A token is not reliably one English word; code and identifiers can divide differently. A useful simplified mental model is repeated generation of a next token from the available context. This can support useful explanations and transformations without automatically retrieving a current specification. OpenAI key concepts
Training changes model parameters. Inference uses a model to respond to current input. Supplying a project passage during a conversation is information for that interaction, not the same operation as retraining the model. Save durable project facts in files so a later session can inspect them.
Attention lets relationships among input representations contribute to output. The original Transformer paper established an architecture centered on attention. This is a conceptual explanation, not a diagram of a particular assistant's hidden reasoning. More text does not guarantee that the governing sentence will be applied correctly. Attention Is All You Need
For the worked comparison, ask how the fictional BenchNote monitor handles invalid samples before providing the original note. Save the answer. Then provide the complete note and ask again, using only it for product-specific claims. The note says invalid samples produce FAULT and their numeric temperature cannot select OK or WARN. It intentionally leaves a recovery count unspecified.
Compare claims, not wording. A first answer that declines to guess is sound. A second answer that traces FAULT to the supplied note is grounded for that claim. Two correct answers do not establish universal reliability, and repeated agreement on an invented rule does not establish truth. Keep the exact request and source alongside the comparison so you know what changed.
D02. Training adjusts model parameters; inference uses the resulting model with current context. Supplying a project note is not the same as retraining.
D03. The current request, selected history, relevant references, and actual tool results enter bounded context. The generated answer still needs verification.
Practice and completion
Complete the exercise, then compare your artifact with the separately distributed instructor answer. A lesson acknowledgment records your own progress; it does not certify that a physical test occurred.
Chapter 2. Resources: Original BenchNote note and original reference responses. Figure references: SS02-01–03; D02–03.
Recording transcript
A language model can be helpful without being a database of verified project facts. This lesson gives us a practical mental model for that difference. We will compare answers with and without a short source passage, then ask what the comparison actually proves. You do not need to implement a transformer or inspect hidden reasoning to use the result at your workbench.
Begin with tokens. A token is a unit used to represent input or output. It may be a word, part of a word, punctuation, or another encoded unit. Do not assume that one token means one English word. A long component identifier, a block of code, and an ordinary sentence can divide differently. Token limits therefore cannot be understood simply by counting lines in an editor.
A useful simplified picture is that the model predicts a next token from the available context, adds it to the growing sequence, and continues. Learned patterns can support explanations, summaries, and useful code. That process does not automatically inspect the current release of your sensor library or run the test whose output it describes. A tool or supplied reference is needed for those actions.
The conceptual diagram separates training from inference. Training adjusts model parameters using data and an optimization process. Further training can shape how a model follows instructions. Inference is using the resulting model for a new request. When you paste a project note into a conversation, you are supplying information for the current interaction. You have not performed the same operation as retraining the model.
That distinction matters when you resume work. If you correct a requirement in one chat, a later session may need the corrected document again. Save the requirement in a project file with a useful name. Do not rely on the assistant remembering a correction forever because it agreed with you once. Later modules will make those persistent documents part of a repeatable workflow.
The second diagram shows context entering the interaction. The request, selected reference material, retained conversation information, and tool results can all contribute. The diagram is a teaching simplification. It is not a private trace of the model's internal process. Our practical influence is over the quality and organization of information that we supply and the checks we perform afterward.
Attention is a mechanism that allows relationships among input representations to contribute to output. That helps explain why a short note can connect a temperature unit, threshold, and validity rule in one answer. It does not mean every relevant sentence is always used correctly. A larger input containing old and contradictory rules may make the task harder rather than easier.
Now open the original BenchNote passage. This is a fictional teaching specification written for the exercise, not a manufacturer's manual. It says samples are synthetic, temperatures use Celsius, invalid samples produce FAULT, and valid samples at or above twenty-eight produce WARN. It explicitly describes a simplified policy without the capstone's hysteresis. We will keep those boundaries visible.
Before supplying the note, ask how BenchNote handles an invalid sample. Save the actual answer. The assistant may say it lacks enough information. That is a good response because the product-specific rule has not been supplied. It may instead suggest a typical behavior, such as retaining the previous reading. If it presents that suggestion as a fact about BenchNote, mark the claim as unsupported.
Now provide the complete passage and ask the same question. Add a clear instruction to use this note for product-specific claims and identify missing facts. The answer should say that invalid input produces FAULT and that its numeric temperature does not select OK or WARN. Check that statement against the actual sentence. A correct answer is useful because you can trace it, not merely because it sounds more detailed.
Repeat the with-reference request in a fresh conversation or compare the supplied original alternatives. Wording may differ. One answer may discuss the validity flag first and another may name the state first. Those differences are not necessarily technical disagreements. Your worksheet should compare claims: fault behavior, unit, valid threshold boundary, and information that remains unspecified.
Ask how many consecutive valid samples are required to recover. This introductory note does not define that count. The correct judgment is that another requirement is needed. A plausible number is still unsupported. Keep the gap visible instead of filling it with a pattern the assistant recognizes from another project. The final sensor monitor will have its own explicit recovery requirement later in the course.
What does a successful comparison establish? It shows that these answers handled this supplied information appropriately. It does not show that every future answer will be correct. A repeated error can remain consistent, and different wording can remain correct. Avoid treating agreement among responses as a substitute for a source or a test.
If an answer contradicts the note, isolate the claim and point to the governing sentence. Ask for a corrected artifact, then inspect it yourself. You can also save the proposed alternative as a design idea, clearly separate from the current requirement. Do not silently rewrite the requirement to match a convincing explanation.
There is another useful comparison you can make without another model call. Remove one essential sentence from your own view of the passage and ask what conclusion you can still support as a reader. If the validity rule is missing, you cannot derive FAULT merely from the temperature value. This exercise shows why the information packet matters independently of any particular assistant. The brief should contain the facts needed for the decision, while the result should preserve gaps that the packet leaves open.
Complete the comparison worksheet before advancing. Keep the original question, the passage, the actual or labeled sample answers, and your conclusions together. Your success check is simple: another maker can see what information changed, which claims became grounded, and which detail is still unknown. That is a more durable result than finding a prompt that happened to produce an impressive paragraph once.
Download transcript
