MLOps

Agentic Search: Mistral Triples RAG Accuracy

Mistral takes RAG accuracy from 26.7% to 86% with Agentic Search: five tools, zero fine-tuning. The numbers, and what they mean for your pipeline.

Agentic Search: Mistral Triples RAG Accuracy

Agentic Search: Mistral Triples RAG Accuracy

Give a standard RAG pipeline 368 real SEC filings and 150 precise questions, and it gets roughly one answer in four right. Turn the same architecture into an agent that can search, open, and navigate through those documents the way a human analyst would, and it gets more than eight in ten right. Mistral AI published these results on August 20, 2026 under the name Agentic Search, and the numbers are worth sitting with: this isn't a marginal tuning gain, it's a different way of retrieving information altogether.

The background

Classic RAG (Retrieval-Augmented Generation) works in a single pass: the system chunks documents, pulls a handful of them by vector similarity, and asks the model to answer from that one batch. It's fast and cheap, but answer quality is capped by the quality of that initial chunking. If the right information is spread across three different pages, or the relevant chunk simply wasn't retrieved on the first try, the model answers off the mark — and nothing in the architecture lets it notice and try again.

Agentic Search changes that mechanic by giving the model tools instead of a fixed batch of chunks. It can decide on its own to refine a search, verify a source before concluding, or navigate to the right section of a long document — the same instinct a human analyst has when they don't stop reading at the first paragraph.

That gap between "retrieved" and "actually relevant" is the everyday failure mode teams run into once a RAG system leaves the demo and meets real documents: long PDFs, cross-references between filings, numbers buried three tables deep. A single retrieval pass has no mechanism to notice it grabbed the wrong section and try again — it just answers with whatever it has.

This isn't the first attempt to push past single-pass RAG's ceiling. MLOps teams have already tried hybrid search (combining vector similarity with BM25 keyword search), reranking after retrieval, or simply pulling more chunks per query. These approaches improve the quality of that first batch of results, but they're still fundamentally single-pass: the model answers from whatever was retrieved once, with no way to go back and look further if it judges the information insufficient. Agentic Search moves the lever somewhere else entirely — instead of optimizing a single retrieval step, it gives the model the ability to decide for itself when another search is warranted.

Five tools, not a fixed pipeline

Mistral structures Agentic Search around five primitives that read less like a traditional RAG pipeline and more like filesystem commands: search locates relevant documents across the corpus via an existing index, open opens a specific document, navigate moves the read position to a page, section, or region within it, read retrieves content at that location, and grep finds patterns inside documents already open.

The model chains these operations in a loop, as many times as it needs, until it has enough to answer with a verified source behind it. One detail matters here: the system requires no fine-tuning and no model-specific training. It leans on the tool-calling capability models already have, and works against an existing search index — no need to rebuild your vector store to adopt it. For teams that want to try it directly, Mistral ships a Search Starter App that spins up a local corpus index with a default configuration, alongside the open Search Toolkit for custom integration.

Why retrieval quality stops being capped by chunking strategy

This is the key line in Mistral's writeup: with Agentic Search, "retrieval quality scales with model capability instead of being capped by your chunking strategy." In classic RAG, a stronger model doesn't compensate for bad chunking — it still answers from the same limited fragments. In Agentic Search, a more capable model navigates better, formulates better intermediate search queries, and verifies more of its sources: end-to-end performance improves with the model, not just with the index.

The numbers: two benchmarks, two kinds of hard documents

Mistral tested the approach on two datasets picked for genuine difficulty, not to flatter the demo.

FinanceBench covers 368 real SEC regulatory filings (roughly 53,900 pages), with 150 precise-answer questions. On Mistral Medium 3.5, single-pass RAG hits 26.7% accuracy. Adding just iterative search capability lifts the score by 47.3 points; adding full navigation (opening, moving within a document, verifying) adds another 8.7 points, landing around 82.7%. GLM-5.2 shows a comparable effect: +52.6 points from search alone, +6.7 more points with full navigation. Mistral rounds the whole experiment to a "3x" improvement in correctness, from 26.7% to 86%. There's a second effect worth flagging, one easy to miss in this kind of comparison: latency drops too. p90 latency falls from 255 seconds to 154 seconds, mean latency from 108 to 71 seconds, and token consumption drops 23.9% on Mistral Medium 3.5.

OfficeQA Pro is harder still: 696 US Treasury Bulletins (roughly 89,000 pages), scanned, table-dense, with precise numeric answers to extract. This is where classic RAG collapses most visibly — GLM-5.2 starts at just 6.3% accuracy in a single pass. The full Agentic Search loop pushes that to 51.9%, a 45.6-point gain, 8.3 points of which come from navigation alone. On Mistral Medium 3.5, search alone adds 27.1 points, navigation another 7.5, landing around 35.6% overall.

A detail that matters for anyone evaluating an agent harness, not just a model

Mistral flags a telling gap: the same GLM-5.2 model scores 41.4% on OfficeQA Pro with the Claude Code harness, versus 51.9% with Mistral's own search harness — a 10.5-point swing on the same model. The takeaway isn't that Mistral is gaming the measurement; it's that the orchestration wrapped around a model — how tools are exposed, in what order, with what instructions — carries nearly as much weight as the model itself on dense document tasks. That's a useful reminder for any team benchmarking LLMs without accounting for the harness surrounding them.

What this means for AI engineering teams

For a team running a RAG pipeline in production — document support, internal legal search, financial report analysis, FAQ over a large knowledge base — this study has three concrete implications.

First, a quality ceiling that felt architectural ("RAG just handles cross-document questions poorly") might actually be a retrieval-architecture ceiling, not a model ceiling. Before throwing a bigger model or a fine-tuning pass at the problem, the question worth asking is: does the current pipeline give the model a way to verify and refine its answer, or only a fixed batch of chunks it has to make do with? The gains measured here — 45 to 55 points from iterative search capability alone, before navigation is even added — suggest retrieval architecture deserves the first audit, not the model.

Second, the latency gain is counterintuitive and worth verifying on your own workload rather than assumed impossible. It's tempting to assume an iterative search loop costs more time than a single round trip — true at the level of one search call, false at the level of the full task: an agent that finds the right information in three targeted searches can be faster than a RAG pipeline that produces a wrong answer, forces the user to rephrase, and restarts the whole process.

Third, the gap measured between two harnesses on the same model is a reminder that a model-only benchmark says nothing about a document system's real-world performance. A team evaluating model vendors for a RAG use case needs to test the full orchestration — tools, system prompts, loop logic — not just query the bare model against a generic question set.

One last thing worth planning for before adopting this in production: an iterative search loop means more model calls, so per-call cost can rise even as total token consumption falls (which is what happened here — a 23.9% reduction on FinanceBench). The metric to track isn't cost per isolated call, but cost per correctly resolved task — a wrong answer delivered in a single call is often more expensive in the end, once you count the user's rephrase and second attempt.

In brief

  • Mistral released Agentic Search: five tools (search, open, navigate, read, grep) letting a model iteratively navigate documents instead of answering from a fixed batch of chunks.
  • On FinanceBench (368 SEC filings), accuracy rises from 26.7% to roughly 86%, alongside a latency drop (p90: 255s → 154s) and a 23.9% token reduction.
  • On OfficeQA Pro (696 scanned, table-dense documents), GLM-5.2's accuracy rises from 6.3% to 51.9%.
  • No fine-tuning required: the method relies on existing tool-calling capability and an already-built search index.
  • The same model can swing more than 10 accuracy points depending on the orchestration harness used — architecture matters as much as the model.
  • Available via the Mistral Search Toolkit (self-hostable), built into Libraries in Studio and Vibe, with a Search Starter App to get going.

Industrialising AI agents? SeedVision offers 3-5 day AI audits and 15-30 day production rollouts. See the packages or book a 30-min call.

Cover photo: Photo by Agence Olloweb on Unsplash.