Next-token prediction

Learn in five minutes

• The model assigns scores to possible next tokens.
• A fluent continuation is not a retrieved fact.
• Training predicts text. It does not by itself check your files.
• GPT-3's paper describes few-shot prompting of a large language model.
• Grounding is a separate step you add.

Ask a model a question about a note it was not given. If it answers smoothly, you still do not have evidence the note says that.

This section HTML

Visual

Diagram SVGDiagram PNGDiagram HTML

Prediction versus lookup

1. Write a private sentence on paper and do not type it.
2. Ask for that sentence's third word.
3. Treat any answer as ungrounded. Then type the sentence and ask again, and still check the words yourself.

This section HTML

Quiz

This section HTML

Ecosystem

Grounding

Tying a claim to a passage you can open.

Sisters: Tokenizers, Transformers, Decoding, Model cards.

The same next-token idea sits under chat, voice, and tool calls.

Pin a model id when you leave the toy explanations.

Config

• Write the goal in one sentence.
• Name the input and the allowed output.
• Save the trace next to the result.

Troubleshooting

• The answer looks fluent and is wrong. Check whether a source passage actually contains the claim.
• The loop never stops. Set a step cap and stop when the tool result repeats.

This section HTML