Agent Reasoning and Decision Making

Learn how agents reason and decide: common reasoning patterns, how an agent chooses its next action, a simple expected-value model of decision-making, and how to improve tool selection.

28 minIntermediateCode Examples

The Core Idea: At every step an agent must choose: answer now, call a tool, ask the user, or stop. Good decisions weigh the chance of success, the value of the result and the cost of trying, and they know when to ask for help.

  1. Beginner: Reasoning Patterns

Reasoning is how the model works out what to do before it acts. Several patterns are common, each trading cost for care:

PatternIdeaBest forCost
Direct answerAnswer immediatelySimple, familiar questionsLowest
Step-by-step (chain of thought)Write out the reasoning before the answerMath, logic, multi-step questionsMore tokens
DecompositionSplit the problem into smaller partsComplex tasksMore calls
Reason + act (ReAct)Alternate thinking and tool useTasks that need outside informationMany steps
Multiple attemptsTry several times and compare or voteHard problems with checkable answersSeveral times more

Tip: Match the pattern to the problem. Using the heaviest pattern for every question wastes time and money.

  1. Beginner: Choosing Actions

At each step the agent's menu of possible actions is short. Naming them clearly makes the behavior easier to control:

ActionWhen it is right
Answer nowThe agent already has enough reliable information
Call a toolIt needs data or must do something
Ask the userThe request is ambiguous or a risky choice needs approval
Retry or change approachThe last attempt failed
StopThe goal is met, or no safe path remains

Remember: Ask the user and stop are real actions. An agent that can never ask or stop will guess instead.

  1. Medium: Decision-Making With Expected Value

A simple way to think about choosing is expected utility: how likely is the action to work, how valuable is success, and what does it cost (time, money, risk)?

utility=P(success)×value−cost\text{utility} = P(\text{success}) \times \text{value} - \text{cost}

In the code, four options for the same question are scored. The database query wins because it is both reliable and cheap, even though asking the user is the most likely to succeed.

Why this matters: Real agents do not compute numbers like these, but the model is implicitly weighing the same factors. You can shape its choices by changing tool descriptions, adding costs to your policy or setting rules such as ask when unsure.

  1. Medium: Tool Selection

Choosing the right tool is the most common decision an agent makes. Wrong choices usually come from unclear tools, not from a weak model.

ProblemFix
Two tools sound alikeMerge them or write descriptions that say when to use each
Too many tools (dozens)Show only a relevant subset, found by search or by routing
The model skips a tool it needsState in the instructions when a tool is required
The model overuses an expensive toolDescribe the cost and give a cheaper first option
A tool failsOffer a fallback tool and explain it in the error message
Routing
For large systems, a first small step picks a category of tools (for example billing, search, files) and then only those tools are shown to the model.

  1. Advanced: When Decisions Go Wrong

FailureSymptomMitigation
OverconfidenceAnswers without checkingRequire a tool or a source for factual claims
Wrong toolRepeated irrelevant callsBetter descriptions, fewer tools, retries with feedback
Analysis paralysisEndless reasoning, no actionLimit reasoning steps, set a deadline
LoopsSame action again and againDetect repeats and force a different action
Acting when it should askRisky guessesA confidence threshold: ask if nothing looks reliable

In the code: When every option has a low chance of success, the agent chooses ask_user instead of guessing. A confidence threshold like this is one of the cheapest safety features you can add.

  1. Visual Explanation: Scoring Options and Switching When Things Change

The figure shows how the best action changes when conditions change, then lists the four factors behind every decision.

Part 1 · Scoring Four OptionsSame question, three situations
Normal dayDatabase works
Memory 5.4, web 7.5, database 8.5, ask 6.5
↓
Best: run_database_query
Database downSuccess chance 0
Database drops out of the ranking
↓
Best: search_web (7.5)
Nothing reliableAll chances are low
No option is above the confidence threshold
↓
Best: ask the user
Part 2 · Four Factors in Every DecisionWhat the agent weighs
Chance
Will it work?
Reliability of the action.
Value
Is it worth it?
How useful success is.
Cost
What does it take?
Time, money, tokens.
Risk
What if wrong?
Harm if it fails.
Click to see why asking the user can be the best actionCompare ▼
If every automatic option is unreliable, a quick question to the user is cheaper than a wrong action. The code returns ask_user when the best success chance is below the threshold.

Part 1 reads left to right as three situations for the same question. Part 2 lists the factors. Open the bar to see when asking the user is the best move.

  1. Python Implementation: Choosing an Action by Expected Value

This script scores four possible actions with expected utility, picks the best, switches when the database is down, and asks the user when no option is reliable enough.

choose_action.pyPython 3.10+ · standard library only
# Choosing an action: expected value of success minus the cost of tryingdef utility(option):    return option["p_success"] * option["value"] - option["cost"] def choose(options, min_confidence=0.5):    best = max(options, key=utility)    if best["p_success"] < min_confidence:        return "ask_user (nothing looks reliable enough)", None    return best["action"], round(utility(best), 2) options = [    {"action": "answer_from_memory", "p_success": 0.55, "value": 10, "cost": 0.1},    {"action": "search_web", "p_success": 0.85, "value": 10, "cost": 1.0},    {"action": "run_database_query", "p_success": 0.90, "value": 10, "cost": 0.5},    {"action": "ask_user", "p_success": 0.95, "value": 10, "cost": 3.0},] print("Scores:", {o["action"]: round(utility(o), 2) for o in options})print("Normal day:     ", choose(options)) down = [dict(o, p_success=0.0) if o["action"] == "run_database_query" else o for o in options]print("Database down:  ", choose(down)) hopeless = [dict(o, p_success=0.3) for o in options if o["action"] != "ask_user"]print("Nothing reliable:", choose(hopeless))
EXPECTED OUTPUT (your numbers may differ slightly)
Scores: {'answer_from_memory': 5.4, 'search_web': 7.5, 'run_database_query': 8.5, 'ask_user': 6.5}
Normal day: ('run_database_query', 8.5)
Database down: ('search_web', 7.5)
Nothing reliable: ('ask_user (nothing looks reliable enough)', None)

The ranking changes with the situation: the database query is chosen normally, web search takes over when the database is down, and the agent asks the user when nothing is reliable. Choosing among options with a clear rule is the heart of decision-making.

Knowledge Check

Question 1: What does expected utility combine?
Question 2: When is asking the user the right action?

Key Points

✓Reasoning patterns trade cost for care: direct, step by step, decomposition, ReAct and multiple attempts.
✓An agent's actions are: answer, call a tool, ask the user, retry, or stop.
✓Expected utility is chance of success times value, minus cost.
✓Most tool-selection mistakes come from unclear or overlapping tools.
✓Large tool sets need routing so only relevant tools are shown.
✓A confidence threshold lets the agent ask instead of guessing.

The Big Picture

Decisions in One Line

Weigh chance, value, cost and risk → pick the best action → ask when unsure

Better decisions come from clear options, clear tools and clear rules about when to stop or ask.

Up Next: Many tasks are too big to decide step by step. In Planning and Task Decomposition you will see how agents break big goals into subtasks and plan their order.