LLM Agents: Architecture, Reliability, and Production Reality¶
Agents are the most over-hyped and under-engineered component in the current ML ecosystem. The gap between “it worked in the demo” and “it works reliably at 3 AM on a Tuesday” is not a polish problem — it is a fundamental property of probabilistic systems making sequential decisions without ground-truth feedback loops. This file treats agents as distributed systems with stochastic nodes, which is the only mental model that will keep you from shipping disasters.
1. What an Agent Actually Is¶
An agent is a control loop that uses an LLM as its reasoning engine. The LLM decides what action to take; the environment provides feedback; the loop repeats until a termination condition is met.
┌─────────────────────────────────────────────────────────┐
│ AGENT CONTROL LOOP │
│ │
│ Observation → [LLM Reasoning] → Action → Environment │
│ ↑ │ │
│ └─────────── Feedback ────────────────────┘ │
└─────────────────────────────────────────────────────────┘
Components:
Perception — What the agent can observe (context window)
Memory — Short-term (context), long-term (vector store), episodic (conversation history)
Reasoning — The LLM’s inference step
Action — Tool calls, API requests, code execution, sub-agent delegation
Planning — Decomposing goals into executable subtasks
2. ReAct Framework¶
ReAct (Reasoning + Acting) is the foundational pattern for LLM agents. Published in arXiv 2210.03629 (Yao et al., 2022).
The pattern interleaves Thought (reasoning trace), Action (tool call), and Observation (tool result):
Thought: I need to find the current stock price of NVDA.
Action: search("NVDA current stock price")
Observation: NVIDIA stock price is $875.40 as of market close.
Thought: Now I need to calculate the P/E ratio given EPS of $11.93.
Action: calculator("875.40 / 11.93")
Observation: 73.38
Thought: I have all the data to answer the question.
Final Answer: NVDA P/E ratio is approximately 73.4x.
Why it works: The explicit reasoning trace conditions the model to stay on-task and provides a natural debug surface. Without it, the model pattern-matches to an answer; with it, the model decomposes the problem.
Why it fails: Each reasoning step is a new generation. Error compounds multiplicatively. A 95% per-step accuracy on a 10-step task yields 0.95^10 = 59.9% task-level accuracy. That math is brutal.
3. Function Calling (Tool Use)¶
Modern LLMs support structured tool calling natively, which is more reliable than parsing free-form text for tool invocations.
import json
from openai import OpenAI
client = OpenAI()
tools = [
{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get current weather for a city",
"parameters": {
"type": "object",
"properties": {
"city": {
"type": "string",
"description": "City name"
},
"unit": {
"type": "string",
"enum": ["celsius", "fahrenheit"]
}
},
"required": ["city"]
}
}
}
]
def get_weather(city: str, unit: str = "celsius") -> dict:
# Real implementation would call a weather API
return {"city": city, "temperature": 22, "unit": unit, "condition": "sunny"}
def run_agent(user_message: str):
messages = [{"role": "user", "content": user_message}]
while True:
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=messages,
tools=tools,
tool_choice="auto"
)
message = response.choices[0].message
messages.append(message)
# No tool call = final answer
if not message.tool_calls:
return message.content
# Execute tool calls
for tool_call in message.tool_calls:
function_name = tool_call.function.name
function_args = json.loads(tool_call.function.arguments)
if function_name == "get_weather":
result = get_weather(**function_args)
else:
result = {"error": f"Unknown function: {function_name}"}
messages.append({
"tool_call_id": tool_call.id,
"role": "tool",
"name": function_name,
"content": json.dumps(result)
})
result = run_agent("What's the weather like in Chennai?")
print(result)
4. Planning Patterns¶
4.1 Plan-and-Execute¶
Generate the full plan first, then execute each step. More reliable for complex tasks because the plan provides global coherence constraints.
from langchain_openai import ChatOpenAI
from langchain.agents import AgentExecutor
from langchain_experimental.plan_and_execute import (
PlanAndExecute, load_agent_executor, load_chat_planner
)
# Planner generates high-level steps
# Executor handles each step with tools
model = ChatOpenAI(temperature=0)
planner = load_chat_planner(model)
executor = load_agent_executor(model, tools, verbose=True)
agent = PlanAndExecute(planner=planner, executor=executor, verbose=True)
Trade-off: Better for structured tasks (research, analysis). Brittle when plan needs replanning after new information.
4.2 Tree of Thought (ToT)¶
Explore multiple reasoning paths in parallel, select best via self-evaluation. arXiv 2305.10601 (Yao et al., 2023). Significantly better on tasks requiring search and exploration (e.g., Game of 24: ReAct 4% → ToT 74% success). Expensive: requires 3-5x LLM calls.
4.3 Reflexion¶
Self-critique after failure, retry with updated strategy. arXiv 2303.11366 (Shinn et al., 2023). AlfWorld tasks: CoT 33% → Reflexion 97% (but benchmarks are optimistic). Good for code generation and debugging loops.
5. Memory Architecture¶
┌────────────────────────────────────────────┐
│ AGENT MEMORY LAYERS │
│ │
│ Working Memory │ Context window │
│ (in-context) │ 4K - 128K tokens │
│ │ Lost on session end │
├────────────────────┼───────────────────────┤
│ Episodic Memory │ Conversation history │
│ │ Summarized or stored │
│ │ in vector DB │
├────────────────────┼───────────────────────┤
│ Semantic Memory │ Facts, knowledge │
│ │ RAG-retrieved │
├────────────────────┼───────────────────────┤
│ Procedural Memory │ Few-shot examples │
│ │ Tool descriptions │
└────────────────────┴───────────────────────┘
Practical memory management:
from langchain_community.memory import ConversationSummaryBufferMemory
from langchain_openai import ChatOpenAI
llm = ChatOpenAI()
# Keeps recent messages verbatim, summarizes older ones
memory = ConversationSummaryBufferMemory(
llm=llm,
max_token_limit=2000, # Keep 2K tokens of recent history
return_messages=True
)
6. Multi-Agent Systems¶
When a single agent with a large context becomes unreliable, decompose into specialized agents.
# LangGraph multi-agent pattern (2025 production standard)
from langgraph.graph import StateGraph, END
from langgraph.prebuilt import ToolNode
from typing import TypedDict, Annotated
import operator
class AgentState(TypedDict):
messages: Annotated[list, operator.add]
next_agent: str
def research_agent(state: AgentState):
"""Specialized agent for web research"""
# ... research logic with search tools
return {"messages": [research_result], "next_agent": "writer"}
def writer_agent(state: AgentState):
"""Specialized agent for content generation"""
# ... writing logic
return {"messages": [written_content], "next_agent": "reviewer"}
def reviewer_agent(state: AgentState):
"""Quality checker - routes back or to END"""
# ... review logic
quality_ok = check_quality(state)
return {
"messages": [review_result],
"next_agent": END if quality_ok else "writer"
}
# Build graph
workflow = StateGraph(AgentState)
workflow.add_node("researcher", research_agent)
workflow.add_node("writer", writer_agent)
workflow.add_node("reviewer", reviewer_agent)
workflow.set_entry_point("researcher")
workflow.add_conditional_edges(
"researcher",
lambda s: s["next_agent"],
{"writer": "writer"}
)
workflow.add_conditional_edges(
"writer",
lambda s: s["next_agent"],
{"reviewer": "reviewer"}
)
workflow.add_conditional_edges(
"reviewer",
lambda s: s["next_agent"],
{"writer": "writer", END: END}
)
app = workflow.compile()
7. LangChain vs. LlamaIndex vs. Raw API — Honest 2025 Assessment¶
This is not a religious question. Each tool is genuinely better at specific things.
Dimension |
LangGraph |
LlamaIndex |
Raw API |
|---|---|---|---|
Agent orchestration |
✅ Best (graph-based, stateful) |
⚠️ Limited |
⚠️ Manual wiring |
RAG / retrieval |
⚠️ Adequate |
✅ Best-in-class |
❌ Build yourself |
Overhead |
~10ms |
~6ms |
~0ms |
Abstraction cost |
Medium |
Low-medium |
None (you write all logic) |
Debugging |
LangSmith (paid) |
LlamaDebugHandler |
Print statements (honest) |
Community/integrations |
119K stars, 500+ |
44K stars, 300+ |
N/A |
When to use |
Complex multi-step agents |
RAG pipelines |
Simple, latency-critical |
Stability (2025) |
LangGraph 1.0 stable |
Stable |
N/A |
Production patterns |
Common in enterprise |
Common in enterprise |
Common in inference-heavy |
2026 pattern emerging: LlamaIndex for retrieval layer + LangGraph for agent orchestration. Not mutually exclusive — they compose.
The honest flag: LangChain 0.x had severe breaking changes. LangGraph 1.0 (October 2025) stabilized the API. If you’re reading code from 2023-2024, assume it’s outdated.
8. Production Reliability: The Hard Numbers¶
This is what most tutorials omit entirely.
From MAP survey (November 2025, 86 deployed agent systems):
Failure Mode |
Prevalence |
Detection |
|---|---|---|
Silent failures |
70% discovered via user reports |
Monitoring gaps |
Rate limiting / API errors |
60% of LLM errors |
Easy to detect |
Tool initialization failures |
Primary bottleneck |
Startup-time |
Prompt injection |
Growing attack surface |
Hard |
Infinite loops / runaway costs |
<5% but catastrophic |
Circuit breakers |
Non-determinism in critical paths |
Universal |
Test suites |
ReliabilityBench (arXiv 2601.06112): Perturbations reduce agent success from 96.9% (ε=0) to 88.1% (ε=0.2). Rate limiting was the most damaging single fault type.
The cost spiral: One production case at GetOnStack: an undetected infinite loop escalated from $127/week to $47,000/week in 4 weeks. Circuit breakers are not optional.
Production Patterns That Actually Work¶
import asyncio
from functools import wraps
import time
# 1. Circuit Breaker
class CircuitBreaker:
def __init__(self, failure_threshold=5, timeout=60):
self.failure_count = 0
self.failure_threshold = failure_threshold
self.timeout = timeout
self.last_failure_time = None
self.state = "CLOSED" # CLOSED, OPEN, HALF_OPEN
def call(self, func, *args, **kwargs):
if self.state == "OPEN":
if time.time() - self.last_failure_time > self.timeout:
self.state = "HALF_OPEN"
else:
raise Exception("Circuit breaker OPEN — failing fast")
try:
result = func(*args, **kwargs)
if self.state == "HALF_OPEN":
self.state = "CLOSED"
self.failure_count = 0
return result
except Exception as e:
self.failure_count += 1
self.last_failure_time = time.time()
if self.failure_count >= self.failure_threshold:
self.state = "OPEN"
raise e
# 2. Token + Step Budget Enforcer
class BudgetedAgent:
def __init__(self, max_steps: int = 15, max_tokens: int = 50_000):
self.max_steps = max_steps
self.max_tokens = max_tokens
self.steps_taken = 0
self.tokens_used = 0
def step(self, action_fn, *args, **kwargs):
if self.steps_taken >= self.max_steps:
raise StopIteration(f"Step budget exhausted: {self.max_steps} steps")
if self.tokens_used >= self.max_tokens:
raise StopIteration(f"Token budget exhausted: {self.max_tokens} tokens")
self.steps_taken += 1
result = action_fn(*args, **kwargs)
# Track tokens from response
if hasattr(result, 'usage'):
self.tokens_used += result.usage.total_tokens
return result
# 3. Structured Output Enforcement
# Use Pydantic models to force structured outputs — prevents free-form hallucination
from pydantic import BaseModel
from openai import OpenAI
class ActionDecision(BaseModel):
thought: str
action: str # One of: "search", "calculate", "answer"
action_input: str
confidence: float # 0-1, fail fast if < 0.6
client = OpenAI()
response = client.beta.chat.completions.parse(
model="gpt-4o-mini",
messages=[{"role": "user", "content": "What's 15% of 847?"}],
response_format=ActionDecision
)
decision = response.choices[0].message.parsed
if decision.confidence < 0.6:
# Escalate or request clarification
pass
9. Full Working Agent: Research + Synthesis¶
"""
Production-grade research agent with:
- Tool use (search + calculator)
- Budget enforcement
- Structured outputs
- Circuit breaker
"""
import json
import os
from openai import OpenAI
from pydantic import BaseModel
from typing import Optional
import requests
client = OpenAI()
# --- Tool Definitions ---
tools = [
{
"type": "function",
"function": {
"name": "web_search",
"description": "Search the web for current information. Use for facts, recent events, data.",
"parameters": {
"type": "object",
"properties": {
"query": {"type": "string", "description": "Search query"},
"num_results": {"type": "integer", "default": 3}
},
"required": ["query"]
}
}
},
{
"type": "function",
"function": {
"name": "calculate",
"description": "Evaluate a mathematical expression. Input must be a valid Python expression.",
"parameters": {
"type": "object",
"properties": {
"expression": {"type": "string", "description": "Python math expression, e.g. '15 * 847 / 100'"}
},
"required": ["expression"]
}
}
}
]
def web_search(query: str, num_results: int = 3) -> str:
"""Stub — replace with SerpAPI, Tavily, or Brave Search API"""
return f"[Search results for: {query}] — integrate real search API here"
def calculate(expression: str) -> str:
try:
# Restricted eval — only math operations
allowed = {k: v for k, v in vars(__builtins__).items()
if k in ['abs', 'round', 'min', 'max', 'sum', 'pow']}
result = eval(expression, {"__builtins__": allowed})
return str(result)
except Exception as e:
return f"Calculation error: {e}"
TOOL_MAP = {
"web_search": web_search,
"calculate": calculate
}
# --- Agent Runner ---
def run_research_agent(question: str, max_steps: int = 10) -> str:
messages = [
{
"role": "system",
"content": (
"You are a research assistant. Think step by step. "
"Use web_search for facts and calculate for math. "
"Be concise. Stop when you have a confident answer."
)
},
{"role": "user", "content": question}
]
steps = 0
total_tokens = 0
while steps < max_steps:
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=messages,
tools=tools,
tool_choice="auto",
temperature=0.1 # Low temperature for reasoning tasks
)
total_tokens += response.usage.total_tokens
message = response.choices[0].message
messages.append(message)
# Budget check
if total_tokens > 20_000:
return f"[Budget exceeded at {total_tokens} tokens] Partial answer: {message.content}"
# Final answer — no tool calls
if not message.tool_calls:
print(f"[Agent] Completed in {steps} steps, {total_tokens} tokens")
return message.content
# Execute tools
for tool_call in message.tool_calls:
fn_name = tool_call.function.name
fn_args = json.loads(tool_call.function.arguments)
if fn_name in TOOL_MAP:
result = TOOL_MAP[fn_name](**fn_args)
else:
result = f"Unknown tool: {fn_name}"
messages.append({
"role": "tool",
"tool_call_id": tool_call.id,
"name": fn_name,
"content": result
})
steps += 1
return f"[Step budget exhausted] Last response: {messages[-1].get('content', 'None')}"
# Run it
if __name__ == "__main__":
answer = run_research_agent(
"What is the market cap of NVIDIA as a multiple of AMD's market cap?"
)
print(answer)
10. What Most People Get Wrong¶
“Agents are not reliable reasoners — they are probabilistic pattern matchers pretending to reason.”
This is not a criticism; it is the specification. A system built with this understanding will have:
Hard budgets (steps, tokens, time, money)
Deterministic tool execution with explicit error handling
Structured outputs at every LLM call boundary
Monitoring for silent failures, not just exceptions
Human-in-the-loop escalation paths for uncertain states
A system built without this understanding will reach production and then destroy itself in slow motion.
The other mistake: Treating agent reliability as a prompt engineering problem. Infrastructure guardrails (circuit breakers, structured output parsers, budget enforcers) are more reliable than prompt guardrails. Prompts are text; infrastructure is code.
Papers¶
Paper |
arXiv ID |
Relevance |
|---|---|---|
ReAct: Synergizing Reasoning and Acting |
2210.03629 |
Foundation of modern agents |
Tree of Thoughts |
2305.10601 |
Multi-path reasoning |
Reflexion |
2303.11366 |
Self-critique and retry |
AutoGPT / early agents survey |
2308.11432 |
Production failure modes |
ReliabilityBench |
2601.06112 |
Hard reliability numbers |
Toolformer |
2302.04761 |
Tool-use pretraining |
Return to README.md · Previous: 03_rag_systems.md · Next: 05_multimodal_and_frontier_models.md