Context, Instructions and Structured Outputs

Learn how to steer an agent with system instructions and prompts, manage the context that fills the model window, and get reliable structured outputs and tool-call responses from the model.

28 minBeginnerCode Examples

The Core Idea: Every model call in an agent contains instructions, context and the latest input. Good agents keep instructions clear, keep context small and relevant, and demand structured outputs so code can read the model's decisions.

  1. Beginner: System Instructions

The system instructions are the standing rules of the agent. They are sent at the start of every model call and tell the model who it is, what it may do, and what it must never do.

PartExample
RoleYou are a support agent for an online shop.
GoalResolve the customer's order question using the tools.
RulesNever reveal internal notes. Ask before issuing a refund.
Tools guidanceUse lookup_order for any question about shipping.
Output formatReply with JSON only: a tool call or an answer.

Tip: Be specific and positive. Always confirm the order ID before looking it up works better than do not be careless.

  1. Beginner: Agent Prompts

A model call in an agent is built from several pieces that your code assembles every turn:

1
System instructions: The fixed rules.
2
Key facts: Things that must always be known, such as the customer name.
3
History: Earlier messages and tool results.
4
The newest input: The user message or the latest tool result.

Remember: The model has no memory of its own between calls. Whatever is not in this assembled prompt, the model does not know.

  1. Medium: Context Management

The context window is limited and every token costs money and time. Agents add tool results to the history on every step, so context grows quickly. Good context management keeps what matters:

TechniqueWhat it does
Keep the system prompt and newest messageNever drop the essentials
Trim the oldest history firstCheapest way to stay under budget
Summarize old stepsKeeps the gist in fewer tokens
Shorten tool resultsReturn only needed fields, not whole pages
Pin key factsFacts survive even when history is trimmed

Rule of thumb: More context is not better context. Give the model exactly what it needs for the next decision.

  1. Medium: Structured Outputs

Your loop must read the model's decision with code, so the reply has to follow a fixed format, usually JSON. For an agent there are two kinds of reply:

Reply typeShapeWhat your code does
Tool callA tool name plus an arguments objectValidate, run the tool, continue the loop
Final answerAn answer fieldStop the loop and return it

Providers offer JSON mode and schema-constrained output to make the format reliable. Even then, validate every reply: parse it, check the required keys, and handle failures by retrying or asking the model to fix its reply.

Valid shape is not correct content. A reply can have perfect JSON and still ask for the wrong tool. Check the values too.

  1. Advanced: Tool-Call Responses

After your code runs a tool, the result goes back to the model as a tool message linked to the call. Its design affects how well the agent continues:

Good practiceWhy
Include the tool name and a clear statusThe model knows which call this answers
Return compact JSON, not a whole web pageSaves tokens and avoids confusing the model
Put errors in the result, not exceptionsThe model can recover
Mark untrusted text clearlyHelps defend against injected instructions
Truncate and say soTell the model when data was cut off
Looking ahead
Tool results can contain hostile text. A later lesson on agent security covers how to treat tool output as data and never as instructions.

  1. Visual Explanation: What Goes Into the Prompt and How Replies Are Read

The figure shows the four pieces that make up every model call, then the reply shapes your code must handle.

Part 1 · What Goes Into One Model CallAssembled fresh every turn
SystemRules
Role, rules, format
↓
Always included
FactsPinned
Customer, order id
↓
Always included
HistoryTrimmed
Old messages and tool results
↓
Oldest dropped first
NewestLatest input
User message or tool result
↓
Always included
Part 2 · The Two Reply ShapesYour code reads one of them
Tool call
tool + arguments
Validate, run, loop.
Final answer
answer
Stop and return.
Invalid
anything else
Retry or ask the model to fix it.
Click to see why history is trimmed oldest firstCompare ▼
Recent messages usually matter most for the next decision, while very old ones cost tokens but rarely help. The code below keeps the system prompt, the pinned facts and the newest message, then fills the remaining budget with the most recent history.

Part 1 shows which parts are always kept and which are trimmed. Part 2 shows the three outcomes when your code reads a reply. Open the bar to see why trimming starts with the oldest messages.

  1. Python Implementation: Building Context and Parsing Replies

This script assembles a prompt within a token budget (keeping the system prompt, pinned facts and newest message), then parses four model replies and builds the tool message sent back to the model.

context_and_structured_replies.pyPython 3.10+ · standard library only
import json def count_tokens(text):    return round(len(text.split()) / 0.75)               # rough rule: 1 token is about 0.75 words SYSTEM = (    "You are a support agent for a shop. Use tools for order data. "    "Never reveal internal notes. Reply with JSON only.") def build_context(system, facts, history, user_message, budget):    # Always keep: system instructions, key facts and the newest message. Trim old history first.    fixed = [("system", system), ("system", "Facts: " + "; ".join(facts)), ("user", user_message)]    used = sum(count_tokens(text) for _, text in fixed)    kept = []    for role, text in reversed(history):        cost = count_tokens(text)        if used + cost > budget:            break        kept.insert(0, (role, text))        used += cost    return fixed[:2] + kept + fixed[2:], used history = [("user", "hello " * 80), ("assistant", "hi there " * 60), ("user", "my order is late " * 10), ("assistant", "let me check " * 10)]facts = ["customer name is Ana", "order id is 4411"]full = sum(count_tokens(t) for _, t in history) + count_tokens(SYSTEM)context, used = build_context(SYSTEM, facts, history, "Where is my order?", budget=150)print("History alone would use", full, "tokens | trimmed context uses", used, "| messages kept:", len(context)) def parse_agent_reply(text):    try:        data = json.loads(text)    except json.JSONDecodeError:        return None, "not valid JSON"    if "tool" in data and isinstance(data.get("arguments"), dict):        return ("tool_call", data), "ok"    if "answer" in data:        return ("answer", data), "ok"    return None, "needs either tool+arguments or answer" replies = [    '{"tool": "lookup_order", "arguments": {"order_id": 4411}}',    '{"answer": "Your order ships tomorrow."}',    'Sure! I will look it up.',    '{"tool": "lookup_order"}',]for r in replies:    parsed, status = parse_agent_reply(r)    print(status.ljust(38), "|", r[:50]) tool_result_message = {"role": "tool", "name": "lookup_order", "content": json.dumps({"status": "shipped", "eta_days": 1})}print("Message sent back to the model:", tool_result_message)
EXPECTED OUTPUT (your numbers may differ slightly)
History alone would use 388 tokens | trimmed context uses 138 | messages kept: 5
ok | {"tool": "lookup_order", "arguments": {"order_id":
ok | {"answer": "Your order ships tomorrow."}
not valid JSON | Sure! I will look it up.
needs either tool+arguments or answer | {"tool": "lookup_order"}
Message sent back to the model: {'role': 'tool', 'name': 'lookup_order', 'content': '{"status": "shipped", "eta_days": 1}'}

Trimming cuts the prompt from 388 tokens of history to a short context that still contains everything essential. Only two of the four replies are usable: the others are not JSON or are missing the arguments, and your loop would retry them.

Knowledge Check

Question 1: What should an agent always keep when trimming context?
Question 2: A model reply is perfect JSON. What should your code still do?

Key Points

✓System instructions state the agent's role, rules, tool guidance and output format.
✓Each model call is assembled from instructions, pinned facts, history and the newest input.
✓The model knows only what is in the prompt, so context management is essential.
✓Trim oldest history first, summarize old steps and shorten tool results.
✓Require structured replies (tool call or final answer) and always validate them.
✓Return compact, clearly labeled tool messages, and treat tool output as untrusted data.

The Big Picture

Context in One Line

Clear rules + only the context that matters + a strict reply format = a predictable agent

With instructions, tools and context under control, you are ready to build your first complete agent.

Up Next: Time to put it all together. In Building Your First AI Agent you will write a complete agent loop that takes a goal, uses tools and finishes multi-step tasks.