Tests and a small eval

Learn in five minutes

• A test names an input and an expected output.
• An eval is a set of those checks for model behavior.
• Lab 03 scores a fixed 10-question fixture.
• The bar in that lab is written in the file, not invented at reporting time.
• A failing check is a result, not a reason to hide the run.

Change one prompt, re-run the same questions, and keep both scores. The difference is the evidence. A new anecdote is not.

This section HTML

Visual

Diagram SVGDiagram PNGDiagram HTML

Flow SVGFlow PNGFlow HTML

Score the golden file

1. Open labs/evals/golden10.json.
2. Run python3 labs/03-golden-eval.py.
3. Read the report under labs/reports if the run writes one.
4. Do not change the expected answers to make the score rise.

This section HTML

Quiz

This section HTML

Ecosystem

Golden set

Questions with agreed answers, reused so scores compare.

Sisters: Git, HTTP, JSON, unit tests.

A Python script can later call a local runner or a cloud API with the same JSON shape.

The same notebook becomes the capstone log.

Config

• Write the goal in one sentence.
• Name the input and the allowed output.
• Save the trace next to the result.

Troubleshooting

• The answer looks fluent and is wrong. Check whether a source passage actually contains the claim.
• The loop never stops. Set a step cap and stop when the tool result repeats.

This section HTML