Cloud endpoints

Learn in five minutes

• A cloud model API lets you send messages without hosting the weights.
• AWS, Azure, and Google document such APIs. Confirm the current one before you code.
• The key stays off the page.
• The region is a choice that affects where logs go. Confirm the region's meaning in that provider's docs.
• The notebook and the eval stay the same if you swap a mock for an endpoint.

Lab 01 already branches: local runner or mock. A cloud call would be a third branch with a key in the environment. This room does not add the key.

This section HTML

Visual

Diagram SVGDiagram PNGDiagram HTML

Flow SVGFlow PNGFlow HTML

A third branch on paper

1. Branch A: local runner.
2. Branch B: mock, labeled mock.
3. Branch C: cloud, key from the environment, region written down.
4. The golden set runs on whichever branch you actually have.

This section HTML

Quiz

This section HTML

Ecosystem

Endpoint

The address of a hosted model API. Confirm it before you call it.

Sisters: Schedulers, Regions, Key management, Risk frameworks.

A cloud endpoint can replace a local runner behind the same notebook and the same eval.

Pin the endpoint, the region, and the model id the day you check them.

Config

• Write the goal in one sentence.
• Name the input and the allowed output.
• Save the trace next to the result.

Troubleshooting

• The answer looks fluent and is wrong. Check whether a source passage actually contains the claim.
• The loop never stops. Set a step cap and stop when the tool result repeats.

This section HTML