One GPU, then a cluster

Learn in five minutes

• A GPU is a processor often used for the matrix math in model training and serving.
• A cluster is many machines working as one pool.
• NVIDIA's H100 page is a public spec sheet. Confirm it before you quote a number from it.
• This lab quotes no peak numbers.
• Serving a model and training a model are different jobs.

One GPU can be enough to study a small open-weight model. A cluster is a scale story. You do not need the cluster to learn the notebook.

This section HTML

Visual

Diagram SVGDiagram PNGDiagram HTML

Two jobs

1. Write 'train' next to a job that changes weights.
2. Write 'serve' next to a job that answers calls.
3. Do not put a price on either.
4. If you cite a chip, cite the vendor page and the date.

This section HTML

Quiz

This section HTML

Ecosystem

Serving

Answering calls with a model that is already trained.

Sisters: Schedulers, Regions, Key management, Risk frameworks.

A cloud endpoint can replace a local runner behind the same notebook and the same eval.

Pin the endpoint, the region, and the model id the day you check them.

Config

• Write the goal in one sentence.
• Name the input and the allowed output.
• Save the trace next to the result.

Troubleshooting

• The answer looks fluent and is wrong. Check whether a source passage actually contains the claim.
• The loop never stops. Set a step cap and stop when the tool result repeats.

This section HTML