MLOps

Production-Grade Agents: LangGraph Fault Tolerance Primitives

LangGraph ships three declarative fault tolerance primitives — RetryPolicy, TimeoutPolicy, and error_handler — that turn fragile demo agents into resilient production services.

Production-Grade Agents: LangGraph Fault Tolerance Primitives

Production-Grade Agents: LangGraph's Fault Tolerance Primitives

Production is where agents meet reality. External APIs return 5xx errors. HTTP requests hang. Subprocesses freeze. An agent that can't handle these failures blocks, produces inconsistent state, or silently crashes — and the on-call engineer spends their night triaging half-corrupted graph runs. On June 4, 2026, LangChain published a comprehensive write-up on LangGraph's fault tolerance mechanisms, documenting three primitives that make agent graphs genuinely resilient: RetryPolicy, TimeoutPolicy, and error_handler.

These aren't optional extras. They're the boundary between a demo agent and a production service.

The context: why agent fault tolerance is different

A standard web service crashes, restarts, and most transient failures resolve with a simple retry. An agent graph is more complex: each node may call an LLM, an external tool, a sub-agent, a vector database. The call chain is long, latencies are high, and a mid-graph hang can leave the graph state partially written and hard to recover.

Without built-in fault tolerance, teams wrap each call in ad hoc try/except blocks — brittle, untested, diverging across nodes. LangGraph addresses this with declarative primitives, configurable per node, that apply uniformly across the graph.

RetryPolicy: automatic retries with configurable backoff

RetryPolicy is the foundational primitive. Configured directly when adding a node, it automatically handles retries on transient failures:

from langgraph.retry import RetryPolicy

StateGraph(State).add_node(
    "call_llm",
    call_llm,
    retry_policy=RetryPolicy(
        max_attempts=4,
        initial_interval=0.5,
        backoff_factor=2.0,
        max_interval=128.0,
        jitter=True,
    ),
)

Key parameters:

  • max_attempts: maximum number of attempts (default: 3)
  • initial_interval: initial delay in seconds before the first retry (default: 0.5s)
  • backoff_factor: multiplier applied to each successive delay (default: 2.0)
  • max_interval: cap on the delay between attempts (default: 128s)
  • jitter: adds randomization to avoid thundering herds
  • retry_on: list of exception types or a callable predicate

What LangGraph retries (and what it refuses to retry)

The default eligible errors are opinionated and documented. LangGraph retries ConnectionError, 5xx responses from httpx and requests, and a few generic transient categories. It does not retry ValueError, TypeError, RuntimeError, and their subclasses — because these almost always indicate programming bugs, not network failures. Retrying them would mask the real issue and waste tokens.

This design avoids the classic pitfall of overly permissive retries: an API returning 422 (invalid parameter) doesn't deserve to be attempted three times at escalating cost.

TimeoutPolicy: stopping a node that hangs

A node calling a remote LLM can stall indefinitely if the connection stays open but data stops flowing. Without an explicit timeout, this hang immobilizes the entire graph. TimeoutPolicy enforces two types of limits:

from langgraph.timeout import TimeoutPolicy

StateGraph(State).add_node(
    "call_llm",
    call_llm,
    timeout=TimeoutPolicy(run_timeout=30, idle_timeout=5),
)
  • run_timeout: absolute wall-clock cap in seconds on a single attempt, regardless of node activity. If the attempt exceeds this limit, LangGraph raises NodeTimeoutError, discards partial writes from the failed attempt, and lets the RetryPolicy decide what happens next.
  • idle_timeout: maximum time without an observable progress signal — a channel write, a streamed chunk, a sub-agent event, a LangChain callback. In default auto mode, this resets on any sign of activity, avoiding killing a node that's legitimately generating a slow response.

Why timeouts are treated as transient failures

A nuanced but important design choice: LangGraph treats a timeout as a transient failure, not a permanent one. The assumption is that the request may have succeeded server-side, and a retry is legitimate. Combining TimeoutPolicy(run_timeout=30) with RetryPolicy(max_attempts=3) therefore gives up to three attempts of 30 seconds each — a ceiling of 90 seconds total before the graph gives up.

error_handler: the graceful exit after retries are exhausted

When all retries are exhausted, the error_handler takes over. It's a plain function that receives the current graph state and a NodeError object with the failure details:

def handle_model_failure(state: State, error: NodeError):
    # error.node: name of the failing node
    # error.error: the underlying exception
    return {"status": "degraded", "last_error": str(error.error)}

StateGraph(State).add_node(
    "call_llm",
    call_llm,
    retry_policy=RetryPolicy(max_attempts=4, backoff_factor=2.0),
    timeout=TimeoutPolicy(run_timeout=30, idle_timeout=5),
    error_handler=handle_model_failure,
)

Three properties that matter in practice:

  1. It only fires after retries are exhausted — not on each failed attempt.
  2. The transition is atomic — the handler runs in the same execution cycle as the final failure.
  3. Error handlers can't be chained — no infinite recursion if the handler itself raises.

The error_handler is where graceful degradation lives: fall back to a cheaper model, return a partial result, log the incident to your observability stack, or route the graph to a human-in-the-loop node.

Composing the three primitives: production patterns

The real power comes from combining all three primitives on the same node, as shown in the official documentation:

StateGraph(State)
    .add_node(
        "call_llm",
        call_llm,
        retry_policy=RetryPolicy(max_attempts=4, backoff_factor=2.0),
        timeout=TimeoutPolicy(run_timeout=30, idle_timeout=5),
        error_handler=handle_model_failure,
    )

In practice, three patterns come up repeatedly in production.

Graceful degradation: the error_handler falls back to a cheaper or less capable model — switching from Opus to Haiku, for instance, if the primary call has exhausted its retries. The graph continues, the user gets a less rich response, but the service stays available.

Manual circuit-breaker: the error_handler writes a flag into graph state ({"llm_circuit_open": True}) that downstream nodes read to skip the failing call. LangGraph doesn't ship a native circuit-breaker, but error_handler plus conditional edges implement one cleanly.

Human-in-the-loop escalation: for high-impact agents (irreversible actions, financial transactions), the error_handler can route to a human-review node that pauses the graph and notifies the on-call team. LangGraph natively supports graph interruptions for exactly this pattern.

Configuration mistakes to avoid

Adding fault tolerance without thinking through the implications creates new problems.

Aggressive retries on non-idempotent calls. If a node triggers a non-idempotent side effect — a database write, a sent message, a financial debit — retrying it may duplicate the action. Either make the external call idempotent (idempotency token on the API side), or place the action in a node without a RetryPolicy and handle failures explicitly.

run_timeout too short for slow LLMs. A run_timeout=5 on a model generating 2,000 tokens guarantees a systematic NodeTimeoutError. Calibrating timeouts requires measuring P99 latencies of the target API in production, not picking a round number.

Ignoring retry cost on token budgets. With max_attempts=4 and backoff_factor=2.0, the worst case is: 0.5s → 1s → 2s → 4s of wait, plus four complete LLM calls. On a node with a large context window, that's a significant spend. Tracking cost per node and per attempt in your observability stack lets you detect nodes that continuously burn their retry budget.

What it means for AI teams

These three primitives aren't implementation details — they define the architecture of a resilient agent.

Observability comes from structured failures. When LangGraph catches a NodeTimeoutError and passes it to the error_handler with error.node and error.error, the incident log is structured at the source. No need to wrap every call in custom try/except blocks — the information is already there, ready for LangSmith or any observability backend.

Retries are a cost, not insurance. Setting max_attempts=5 everywhere "for safety" multiplies worst-case latency and token spend by five on every failure path. The right approach is to calibrate max_attempts and backoff_factor per node, matching the real SLA of each external call, and use retry_on to limit retries to errors that genuinely warrant them.

Fault tolerance is an architectural decision, not just code. Deciding which nodes get an error_handler, which ones fall back to a cheaper model, and which ones should alert a human — that's a product and engineering decision as much as a technical one. This is exactly the productionization work SeedVision conducts when rolling agents out: mapping failure points, defining retry and degradation policies, and wiring observability at the right layer.

In short

  • LangGraph provides three declarative fault tolerance primitives — RetryPolicy, TimeoutPolicy, and error_handler — configurable per node.
  • RetryPolicy handles exponential backoff with jitter; it excludes programming errors (ValueError, TypeError) by default, retrying only genuine transient failures.
  • TimeoutPolicy distinguishes absolute timeout (run_timeout) from idle timeout (idle_timeout), and treats both as transient failures eligible for retry.
  • error_handler fires after retries are exhausted, receives full failure context, and enables graceful degradation strategies.
  • Properly configured, these primitives turn a fragile agent into an observable, resilient service — without ad hoc wrappers on every external call.

Industrialising AI agents? SeedVision offers 3-5 day AI audits and 15-30 day production rollouts. See the packages or book a 30-min call.

Cover photo: Photo by Jake Walker on Unsplash.