Mistral's Agentic Search Triples Accuracy on Financial-Document Retrieval by Ditching Single-Pass RAG

Mistral says its new Agentic Search system triples correctness on financial-document retrieval — from 26.7% to 86% on the FinanceBench benchmark — by letting models iteratively navigate, search, open, read, and grep across documents instead of running a single retrieval pass and hoping the top-k chunks contain the answer. On OfficeQA Pro, Mistral reports a 45.6-point jump, from 6.3% to 51.9%, alongside up to 39.6% lower inference latency and roughly a third less token consumption than standard RAG. The system is live now through the Mistral Search Toolkit, already wired into Studio and Vibe, and requires no model-specific fine-tuning; it works on both cloud and on-prem deployments.

The pitch is narrow and specific: long documents, questions that need synthesis across multiple sources, and answers that need to cite back to something real — exactly where single-pass RAG tends to fail, because a fixed top-k retrieval either misses the passage that answers the question or drowns it in irrelevant context.

The retrieval layer underneath any of this still sets the ceiling. NVIDIA’s Nemotron 3 Embed, released in July, currently ranks first on the RTEB leaderboard at 78.5%, with a 32k-token context window built for exactly the kind of multi-turn agent memory Agentic Search depends on. And Hugging Face’s rundown of multi-vector “late interaction” embedding models shows the real tradeoff underneath better retrieval quality: keeping one vector per token instead of one per document lifts NanoBEIR NDCG from 0.6764 to 0.6868, but at roughly 42 times the index storage cost — a cost most teams will only accept once the downstream accuracy gain, like Mistral’s FinanceBench numbers, is large enough to justify it.

Iterative retrieval and richer embeddings are separate bets, but they’re converging on the same target: closing the gap between “the model retrieved something” and “the model retrieved the right thing.” For teams evaluating an agentic system’s grounding layer, that gap — not the model’s raw benchmark score — is increasingly the number that matters.