Running the Model: Hosted APIs, Local Weights and the Choice Between Them

The interface is the API shape. Everything behind it is an implementation detail you should be able to swap with one environment variable, and this course makes that literally true for the code you built in the previous four.

You will build the backend abstraction, price the hosted path per converged artifact rather than per token, run local weights with llama.cpp, Ollama, vLLM and MLX, do the sizing arithmetic against RAM and VRAM, read licence terms properly, and then run the agent regression suite against two backends and put quality, cost and latency in one table.

Five lessons, about two hours, a graded quiz on each. Free and open to access.

What you will learn

• Put a backend abstraction behind one environment variable
• Model hosted cost per converged artifact, not per token, and name the contractual questions that decide enterprise deployments
• Run local weights and choose between llama.cpp, Ollama, vLLM and MLX on purpose
• Do the sizing arithmetic: parameters, quantisation, RAM and VRAM, and what fits in 8GB
• Compare two backends on quality, cost and latency with your own suite

Course curriculum

5 lessons, 2 hours, a graded quiz on each.

1. The Interface Is the API Shape Artifact: Backend abstraction
2. Hosted: Cost per Converged Artifact, Latency, Limits and Contracts Artifact: Cost model
3. Local: llama.cpp, Ollama, vLLM and MLX, and the Sizing Arithmetic Artifact: Local model running
4. Choosing Weights: Families, Size Classes and Licence Terms Artifact: Selection rubric
5. Swap and Measure: Two Backends, One Table Artifact: Comparison table

Gate

Regression suite passing against two backends, comparison table produced.

Frequently asked questions

Is local always cheaper?

No. Local shifts cost from per-token to hardware, operations and your time. The course gives you the arithmetic to find the crossover for your own volume rather than an opinion.

What actually fits in 8GB?

The course works through the sizing arithmetic: parameter count multiplied by bytes per parameter at your quantisation level, plus context and overhead. You will be able to answer this for any model rather than memorising a list.

Why does licence terms get its own lesson section?

Because open weights is not the same as open source, and several widely used families carry conditions that matter the moment the word production is used.