The Core Idea: An LLM API is a stateless request and response: you send messages, you get a completion, you pay per token. Keep keys secret, plan for failures, and choose between hosted and open-weight models based on cost, privacy and control.
- Beginner: What Is an LLM API?
An API (Application Programming Interface) lets your code talk to a model that runs on someone else's servers. You send a request over the internet and get the model's answer back. You do not manage GPUs, you only pay for what you use.
Almost every provider uses the same basic idea: you send a list of messages and settings, and you receive a completion.
- Beginner: Messages, Roles and Settings
| Field | What it does |
|---|---|
| model | Which model should answer |
| messages | The conversation: a list of role and content pairs |
| system role | Standing instructions: tone, rules, persona |
| user role | What the person asked |
| assistant role | Earlier answers, so the model sees the history |
| temperature / top_p | How random the answer is (see the sampling lesson) |
| max_tokens | A limit on the length of the answer |
| stream | Send the answer piece by piece, so users see text appear immediately |
Remember: The API is stateless. The model does not remember your last call. You must send the earlier messages again each time.
- Medium: Cost, Speed and Keys
Providers bill by tokens, usually per million, with separate prices for input and output tokens. Output tokens are normally more expensive. A quick estimate:
Protect your API key: Treat it like a password. Keep it in an environment variable on a server. Never put it in browser code, a public repository or a mobile app. Anyone who has it can spend your money.
- Medium: Reliability, Rate Limits and Retries
Networks fail and busy servers say slow down. A production app must expect this:
| Problem | Typical response | What your code should do |
|---|---|---|
| Rate limit (HTTP 429) | Too many requests | Wait and retry with exponential backoff: 1, 2, 4, 8 seconds |
| Server error (5xx) | Provider problem | Retry a few times, then show a friendly message |
| Timeout | No answer in time | Set a timeout and retry once or twice |
| Invalid request (4xx) | Your request is wrong | Do not retry. Fix the request |
| Context too long | Prompt exceeds the window | Shorten or summarize before sending |
Good practice: Add a little random delay (jitter) to each wait so many clients do not retry at exactly the same moment.
- Advanced: Open-Weight Models and Choosing Between Them
Besides hosted APIs there are open-weight models, such as the Llama, Mistral, Qwen and Gemma families, whose trained weights you can download and run yourself. Open weights is not always the same as open source: always read the license, because some limit commercial use or large-scale deployment.
| Question | Hosted API | Self-hosted open model |
|---|---|---|
| Setup effort | Minutes | Hours to days (GPU, serving software) |
| Cost pattern | Pay per token, nothing when idle | Pay for hardware even when idle |
| Privacy | Data goes to the provider | Data stays on your machines |
| Control | Limited to provider settings | Full: fine-tune, quantize, modify |
| Top quality | Often the strongest models | Strong and improving, size dependent |
- Visual Explanation: The API Journey and the Hosting Choice
The figure traces one request from your server to the provider and back, then compares three ways to get a model.
Click to see the retry ruleCompare ▼
Part 1 shows who does what in a single call. Part 2 compares hosted, self-hosted and hybrid setups. Open the bar for the retry rule.
- Python Implementation: Requests, Cost and Retries
This script builds a chat-style request as JSON (no network or key needed), estimates a monthly bill using example prices, and shows retry with exponential backoff against a simulated busy server. The prices are made-up examples, not real quotes.
import jsonimport random # 1. What a chat API request really is: a JSON body (no network needed to build it)request = { "model": "example-model", "messages": [ {"role": "system", "content": "You are a concise assistant."}, {"role": "user", "content": "Explain what an API key is in one sentence."}, ], "temperature": 0.2, "max_tokens": 100,}print("Request body is", len(json.dumps(request)), "characters of JSON with keys:", list(request)) # 2. Estimating the bill: providers charge per million tokens, input and output separatelydef cost_usd(input_tokens, output_tokens, in_price, out_price): return (input_tokens * in_price + output_tokens * out_price) / 1_000_000 calls, in_tok, out_tok = 100_000, 800, 300for name, in_p, out_p in [("small model (example prices)", 0.15, 0.60), ("large model (example prices)", 3.00, 15.00)]: print(name, "-> USD per month:", round(calls * cost_usd(in_tok, out_tok, in_p, out_p), 2)) # 3. Retry with exponential backoff when the server is busyrandom.seed(1)def flaky_call(attempt): if random.random() < 0.6: raise RuntimeError("429 Too Many Requests") return "ok" def call_with_retry(max_attempts=6): wait = 1 for attempt in range(1, max_attempts + 1): try: return flaky_call(attempt), attempt except RuntimeError: print(" attempt", attempt, "failed, waiting", wait, "seconds") wait *= 2 # 1, 2, 4, 8 ... seconds return "gave up", max_attempts result, attempts = call_with_retry()print("Result:", result, "after", attempts, "attempt(s)")The cost comparison shows why model choice matters: with the same traffic, the larger model costs about 23 times more in this example. In the retry demo the first attempt fails, the code waits, and the second succeeds.
Interactive Knowledge Check
Key Points
The Big Picture
Messages + settings → HTTPS call with a secret key → completion → handle errors and cost
With an API key and a few lines of code you can build real products. The model choice then becomes an engineering trade-off between quality, price, speed and privacy.
Up Next: Your app must read the answer, not just show it. In Structured Outputs and Tool Calling you will learn how to get reliable JSON and let the model use tools safely.