LLM APIs and Open-Source Models

Learn how to call an LLM from code: messages and roles, token pricing, API key safety, retries with backoff, and how to choose between hosted APIs and open-weight models that you run yourself.

28 minBeginnerCode Examples

The Core Idea: An LLM API is a stateless request and response: you send messages, you get a completion, you pay per token. Keep keys secret, plan for failures, and choose between hosted and open-weight models based on cost, privacy and control.

  1. Beginner: What Is an LLM API?

An API (Application Programming Interface) lets your code talk to a model that runs on someone else's servers. You send a request over the internet and get the model's answer back. You do not manage GPUs, you only pay for what you use.

Almost every provider uses the same basic idea: you send a list of messages and settings, and you receive a completion.

1
Build the request: A JSON body with the model name, the messages and settings such as temperature and max tokens.
2
Send it securely: An HTTPS call with your secret API key in a header.
3
Read the response: The reply text, the reason it stopped, and how many tokens were used.
4
Handle problems: Rate limits, timeouts and errors need retries and clear messages.

  1. Beginner: Messages, Roles and Settings

FieldWhat it does
modelWhich model should answer
messagesThe conversation: a list of role and content pairs
system roleStanding instructions: tone, rules, persona
user roleWhat the person asked
assistant roleEarlier answers, so the model sees the history
temperature / top_pHow random the answer is (see the sampling lesson)
max_tokensA limit on the length of the answer
streamSend the answer piece by piece, so users see text appear immediately

Remember: The API is stateless. The model does not remember your last call. You must send the earlier messages again each time.

  1. Medium: Cost, Speed and Keys

Providers bill by tokens, usually per million, with separate prices for input and output tokens. Output tokens are normally more expensive. A quick estimate:

cost=input tokens×input price+output tokens×output price1,000,000\text{cost} = \frac{\text{input tokens} \times \text{input price} + \text{output tokens} \times \text{output price}}{1{,}000{,}000}

Protect your API key: Treat it like a password. Keep it in an environment variable on a server. Never put it in browser code, a public repository or a mobile app. Anyone who has it can spend your money.

Ways to save money
Choose the smallest model that does the job, shorten prompts, cap max tokens, cache repeated prefixes, and send simple tasks to cheaper models.

  1. Medium: Reliability, Rate Limits and Retries

Networks fail and busy servers say slow down. A production app must expect this:

ProblemTypical responseWhat your code should do
Rate limit (HTTP 429)Too many requestsWait and retry with exponential backoff: 1, 2, 4, 8 seconds
Server error (5xx)Provider problemRetry a few times, then show a friendly message
TimeoutNo answer in timeSet a timeout and retry once or twice
Invalid request (4xx)Your request is wrongDo not retry. Fix the request
Context too longPrompt exceeds the windowShorten or summarize before sending

Good practice: Add a little random delay (jitter) to each wait so many clients do not retry at exactly the same moment.

  1. Advanced: Open-Weight Models and Choosing Between Them

Besides hosted APIs there are open-weight models, such as the Llama, Mistral, Qwen and Gemma families, whose trained weights you can download and run yourself. Open weights is not always the same as open source: always read the license, because some limit commercial use or large-scale deployment.

QuestionHosted APISelf-hosted open model
Setup effortMinutesHours to days (GPU, serving software)
Cost patternPay per token, nothing when idlePay for hardware even when idle
PrivacyData goes to the providerData stays on your machines
ControlLimited to provider settingsFull: fine-tune, quantize, modify
Top qualityOften the strongest modelsStrong and improving, size dependent
Tools to run models yourself
Weights are commonly shared on the Hugging Face Hub. Popular ways to run them include llama.cpp and Ollama for a laptop or single machine, and vLLM for high-throughput servers.

  1. Visual Explanation: The API Journey and the Hosting Choice

The figure traces one request from your server to the provider and back, then compares three ways to get a model.

Part 1 · One API CallFrom your code to an answer
Your sideClient
Build JSON: model, messages, settings
↓
Add the secret API key (server side)
↓
Send over HTTPS
↓
Request leaves your server
Provider sideServer
Check key and limits
↓
Run the model on GPUs
↓
Return text and token counts
↓
Response arrives
Part 2 · Hosted or Self-Hosted?Choose by need
Hosted API
Fast start
Best when you want quality and zero setup.
Open model, your GPU
Control
Best for privacy, customization, steady heavy use.
Hybrid
Both
Cheap local model for easy tasks, hosted model for hard ones.
Click to see the retry ruleCompare ▼
If a call fails with 429 or a server error, wait 1 second and retry, then 2, then 4, doubling each time up to a limit, with a little random jitter. Never retry requests that are simply wrong (4xx). The code below shows the loop.

Part 1 shows who does what in a single call. Part 2 compares hosted, self-hosted and hybrid setups. Open the bar for the retry rule.

  1. Python Implementation: Requests, Cost and Retries

This script builds a chat-style request as JSON (no network or key needed), estimates a monthly bill using example prices, and shows retry with exponential backoff against a simulated busy server. The prices are made-up examples, not real quotes.

llm_api_basics.pyPython 3.10+ · standard library only
import jsonimport random # 1. What a chat API request really is: a JSON body (no network needed to build it)request = {    "model": "example-model",    "messages": [        {"role": "system", "content": "You are a concise assistant."},        {"role": "user", "content": "Explain what an API key is in one sentence."},    ],    "temperature": 0.2,    "max_tokens": 100,}print("Request body is", len(json.dumps(request)), "characters of JSON with keys:", list(request)) # 2. Estimating the bill: providers charge per million tokens, input and output separatelydef cost_usd(input_tokens, output_tokens, in_price, out_price):    return (input_tokens * in_price + output_tokens * out_price) / 1_000_000 calls, in_tok, out_tok = 100_000, 800, 300for name, in_p, out_p in [("small model (example prices)", 0.15, 0.60), ("large model (example prices)", 3.00, 15.00)]:    print(name, "-> USD per month:", round(calls * cost_usd(in_tok, out_tok, in_p, out_p), 2)) # 3. Retry with exponential backoff when the server is busyrandom.seed(1)def flaky_call(attempt):    if random.random() < 0.6:        raise RuntimeError("429 Too Many Requests")    return "ok" def call_with_retry(max_attempts=6):    wait = 1    for attempt in range(1, max_attempts + 1):        try:            return flaky_call(attempt), attempt        except RuntimeError:            print("  attempt", attempt, "failed, waiting", wait, "seconds")            wait *= 2                                   # 1, 2, 4, 8 ... seconds    return "gave up", max_attempts result, attempts = call_with_retry()print("Result:", result, "after", attempts, "attempt(s)")
EXPECTED OUTPUT (your numbers may differ slightly)
Request body is 218 characters of JSON with keys: ['model', 'messages', 'temperature', 'max_tokens']
small model (example prices) -> USD per month: 30.0
large model (example prices) -> USD per month: 690.0
attempt 1 failed, waiting 1 seconds
Result: ok after 2 attempt(s)

The cost comparison shows why model choice matters: with the same traffic, the larger model costs about 23 times more in this example. In the retry demo the first attempt fails, the code waits, and the second succeeds.

Interactive Knowledge Check

Question 1: Where should you keep your API key?
Question 2: What should you do after a 429 Too Many Requests error?

Key Points

✓A chat API takes messages and settings and returns a completion plus token usage.
✓The API is stateless: send the conversation history again with every call.
✓Cost = tokens × price per million, with different prices for input and output.
✓Keep API keys on the server in environment variables and never in front-end code.
✓Retry rate-limit and server errors with exponential backoff; never retry invalid requests.
✓Open-weight models give privacy and control but need hardware and a careful reading of the license.

The Big Picture

APIs in One Line

Messages + settings → HTTPS call with a secret key → completion → handle errors and cost

With an API key and a few lines of code you can build real products. The model choice then becomes an engineering trade-off between quality, price, speed and privacy.

Up Next: Your app must read the answer, not just show it. In Structured Outputs and Tool Calling you will learn how to get reliable JSON and let the model use tools safely.