Evaluation and testing

Learn in five minutes

• Keep a fixed set of lines you are allowed to use: a script you wrote, not a customer tape.
• Score the transcript against the script. A wrong name is a miss even when the sentence sounds kind.
• Score the handoff packet: channel, what was said, note id, open question.
• Score the refusal: a missing fact stays missing.
• LiveKit's pipeline page says every stage of a cascaded pipeline produces text or audio you can inspect, log, and audit. Save that trail, not only a feeling about the audio.

Three checks on one script are an evaluation. A single impressive sentence is not.

This section HTML

Visual

Diagram SVGDiagram PNGDiagram HTML

Flow SVGFlow PNGFlow HTML

A script of three lines

1. Line 1: the note says the desk is closed on Sunday. Expect that fact, or a quote.
2. Line 2: the note is silent on price. Expect a refusal.
3. Line 3: the person asks for a supervisor. Expect the four-line packet.
4. Save the three results. Do not edit the expected line after you see the output.

This section HTML

Quiz

This section HTML

Ecosystem

Scripted test

Lines you wrote, with an expected property, saved next to the result.

Sisters: Human handoff, NIST AI RMF, OWASP LLM risks, Recording statutes.

The voice channel does not remove the tool cap or the person who must approve an action that is hard to undo.

A later test can add a script you wrote. It cannot add a legal conclusion.

Config

• Write the goal in one sentence.
• Name the input and the allowed output.
• Save the trace next to the result.

Troubleshooting

• The answer looks fluent and is wrong. Check whether a source passage actually contains the claim.
• The loop never stops. Set a step cap and stop when the tool result repeats.

This section HTML