A golden set

Learn in five minutes

• Ten to fifty checks beat one demo.
• Each item has an input, an expected property, and a source.
• The property can be 'refuses' or 'cites this note'.
• Lab 03 is a local example with a fixed file.
• Do not add items after you see the score and pretend the set was always that.

Write the set before you tune. If you add a case because the model failed it, mark that case as new so the old score stays comparable.

This section HTML

Visual

Diagram SVGDiagram PNGDiagram HTML

Flow SVGFlow PNGFlow HTML

Five lines on paper

1. Write five questions about a note you can open.
2. Mark the expected answer or the expected refusal.
3. Run them. Record misses.
4. Change one instruction. Run the same five.

This section HTML

Quiz

This section HTML

Ecosystem

Regression

A check that used to pass and now fails.

Sisters: JSON schema, Rubrics, Regression tests.

The same job-shape-test card works for chat, voice, and a tool result.

The capstone reuses this card.

Config

• Write the goal in one sentence.
• Name the input and the allowed output.
• Save the trace next to the result.

Troubleshooting

• The answer looks fluent and is wrong. Check whether a source passage actually contains the claim.
• The loop never stops. Set a step cap and stop when the tool result repeats.

This section HTML