Latency and a toy budget

Learn in five minutes

• Latency is time, usually milliseconds.
• An average hides the slow tail. Lab 08 looks at a stage p95.
• A budget is a limit you chose for a lesson, not a vendor SLA.
• Voice feels worse than chat at the same delay because the person is waiting in real time.
• Confirm any published SLA before you repeat it. This page states none.

If the speech stage is 900 toy milliseconds and the model stage is 200, the speech stage is the one to study. The average of the two would hide that.

This section HTML

Visual

Diagram SVGDiagram PNGDiagram HTML

Read lab 08's idea

1. Run python3 labs/08-voice-latency-budget.py.
2. Read which stage the report calls out.
3. Write one sentence: the slow stage is the one a person would notice.
4. Do not convert the toy numbers into a price.

This section HTML

Quiz

This section HTML

Ecosystem

p95

A point on the slow tail: about 95 of 100 samples are faster.

Sisters: JSON schema, Rubrics, Regression tests.

The same job-shape-test card works for chat, voice, and a tool result.

The capstone reuses this card.

Config

• Write the goal in one sentence.
• Name the input and the allowed output.
• Save the trace next to the result.

Troubleshooting

• The answer looks fluent and is wrong. Check whether a source passage actually contains the claim.
• The loop never stops. Set a step cap and stop when the tool result repeats.

This section HTML