AI hallucination rate: benchmarks across models in 2026

Ask five people what the AI hallucination rate is and you will get five different numbers, from under 2% to over 75%. All of them can be correct. The figure depends almost entirely on the task, the model and how the test was scored. On the Vectara hallucination leaderboard, last updated in October 2026, the best model summarises documents with a 1.8% hallucination rate, while several widely used models sit above 20% on exactly the same test.

This post pulls together published figures from independent leaderboards, vendor system cards, peer-reviewed and preprint research and a large international broadcaster study. We set them side by side, explain why they disagree, and give a realistic range you can plan around if you are putting a model in front of customers or staff.

Quick answer

There is no single AI hallucination rate. On grounded summarisation, where the model has the source text, the best current models hallucinate in roughly 2% to 5% of summaries and many popular models in 5% to 25%. On open questions answered from memory, error rates of 30% to 80% have been published. Grounding, abstention and checking narrow that gap.

What counts as a hallucination

The term is used loosely, which is one reason the numbers vary so much. The US National Institute of Standards and Technology uses the word confabulation and, in its Generative AI Profile (NIST AI 600-1) published in July 2024, describes it as a phenomenon in which generative AI systems generate and confidently present erroneous or false content in response to prompts. The key word is confidently. A model that says it does not know is not hallucinating. A model that invents a plausible answer is.

In practice, benchmarks measure two quite different failures:

A third category sits between them: attribution errors, where an answer may be broadly right but cites the wrong source or no source at all. As the broadcaster study below shows, this is often the biggest single problem for assistants that search the web.

AI hallucination rate benchmarks compared

The figures below come from at least four independent types of source. Each line gives the source, the figure, the year and the scope of the test, so you can see why they are not directly comparable.

Read in a line, that looks like chaos. Read by task, it is fairly consistent. Grounded tasks sit in single digits or low double digits. Questions answered from memory with no option to abstain sit far higher. Specialist domains with long, complex answers land in between.

Why hallucination rates differ so much by task

The biggest driver is whether the model has the facts in front of it. In grounded summarisation the answer is in the document, and the test only asks whether the model stayed faithful. In SimpleQA the model must recall obscure facts with no source, which is far harder. That alone explains most of the gap between a 3% and a 75% figure. The FaithJudge research from Vectara and academic collaborators notes that the original leaderboard has now tested over 160 models on summarisation, and extends the same approach to retrieval question answering and data to text tasks because results differ across them.

The second driver is how answers are scored. OpenAI's own research argues that models hallucinate because standard training and evaluation procedures reward guessing over acknowledging uncertainty. If a benchmark only counts accuracy, a model that guesses on every question can score better than one that says it does not know, even though it produces far more wrong answers. That is exactly what the SimpleQA comparison shows: o4-mini was slightly more accurate than gpt-5-thinking-mini (24% against 22%) but almost three times as likely to be wrong. Source: OpenAI.

The third driver is model behaviour, and here newer is not always better. OpenAI's system card reports that o3 tends to make more claims overall, which produces more accurate claims and more hallucinated ones, and that more research is needed to understand the cause. Source: OpenAI system card. The Vectara table shows the same pattern: several reasoning models score worse on faithful summarisation than smaller, plainer models, because they add inferences the source does not support.

The fourth driver is who checks the answers. Automated judges are cheaper but miss things. The FaithJudge paper reports that zero-shot language model detectors reach only modest effectiveness, with balanced accuracy below 78% on average, which is why it adds human annotated examples to guide the judge. Source: FaithJudge paper. Human expert reviews, like the legal and broadcaster studies, tend to find more problems because they also catch subtle errors and poor sourcing.

What the specialist studies add

Domain studies matter because they test the long, messy answers businesses actually rely on. In the Stanford legal study, researchers found that the leading AI legal research tools from LexisNexis and Thomson Reuters each hallucinate between 17% and 33% of the time, while noting that hallucinations are reduced relative to general purpose chatbots such as GPT-4. Source: Stanford legal AI study. Reporting on the revised study, LawNext gave the split as 33% for Westlaw AI-Assisted Research against 17% for Lexis+ AI, and noted that Lexis+ AI correctly answered 65% of queries while Westlaw was accurate 42% of the time.

That is a useful lesson in itself. Both tools use retrieval over trusted legal databases, which is the standard fix for hallucination, and both still produced wrong or unsupported answers in a meaningful share of cases. Retrieval lowers the rate. It does not remove it.

The broadcaster study adds an international view. It covered 14 languages across 18 countries and found the problem consistent across languages and territories. Sourcing was the biggest weakness, with 31% of responses showing serious sourcing problems such as missing, misleading or incorrect attributions. Gemini had significant issues in 76% of responses, more than double the other assistants, largely because of poor sourcing. Source: EBU.

A realistic range for business use

Reconciling these sources gives a planning range rather than a single number. Treat these as orders of magnitude, not guarantees, because your documents and questions will differ from any benchmark.

Two points follow. First, the model you pick matters, but the design of the system around it matters more. Moving a task from memory to grounded retrieval can cut errors by an order of magnitude. Our guide to choosing the best AI model for business covers the model side, and our explainer on building an AI knowledge base covers the grounding side. Second, even a 3% rate means three bad outputs in every hundred, so anything customer facing or regulated needs a check step.

How to reduce hallucinations in practice

The research points to a short list of controls that consistently lower error rates in production systems:

Hallucination is one of several ways agents go wrong once they are live. Our breakdown of AI agent failure modes covers the others, including tool errors and runaway loops.

How Klevere approaches hallucination risk

Across more than 500 AI agents deployed for clients in 12 industries, we treat hallucination as an engineering problem to be measured, not a vague risk to be accepted. Every build starts with an evaluation set taken from the client's real questions and documents, and we report the error rate on that set before anything goes live.

We default to grounded designs: agents retrieve from approved sources, cite what they used, and are told to decline when the sources do not cover the question. Where a figure or statement would carry real consequences, such as a price, a legal position or a compliance claim, the agent routes it to a person instead of answering alone. We also re-run the evaluation set whenever a model provider ships a new version, because the leaderboards show that a newer model can hallucinate more on a given task, not less.

How we compiled this

We used only published sources that state their method: an open leaderboard with a documented evaluation model, vendor system cards and research that report exact figures, a legal study by Stanford researchers published as a preprint, and a large multi country broadcaster evaluation. We quote figures exactly as each source reports them and give the test and date alongside each one. None of these figures are Klevere's own data. Leaderboard positions change often, so check the live table before relying on any single model's rank.

Frequently asked questions

What is the hallucination rate of ChatGPT?

It depends on the model behind ChatGPT and the task. On grounded summarisation, OpenAI models on the Vectara leaderboard range from about 3% to over 20%. On hard factual questions from memory, OpenAI's own system card reported 51% for o3 on SimpleQA. Newer models that decline uncertain questions report far lower error rates on the same test.

Which AI model hallucinates the least?

On the Vectara summarisation leaderboard updated in October 2026, the lowest rate was 1.8%, followed by several models between 3% and 5%, including small models from OpenAI, Google, Microsoft and Meta. The best model for your use case depends on your documents, so test the shortlist on your own data before choosing.

Does RAG stop AI hallucinations?

Retrieval augmented generation reduces hallucinations a great deal because the model answers from supplied sources rather than memory. It does not stop them. The Stanford study found retrieval based legal tools still hallucinated between 17% and 33% of the time. Combine retrieval with citations, abstention and automated checks to get rates down further.

Why do newer reasoning models sometimes hallucinate more?

Reasoning models tend to make more claims and add their own inferences. OpenAI reported that o3 made more claims overall, producing both more correct and more hallucinated statements. On faithful summarisation, that habit of adding unsupported detail counts as hallucination, which is why some reasoning models rank below smaller, plainer models.

Is a 3% hallucination rate good enough for business?

For internal drafting with human review, it is usually workable. For customer facing answers, financial figures or regulated advice, three errors in every hundred outputs is too many to send unchecked. The usual answer is to keep the low rate model and add a verification step or human approval for high risk outputs.

How do you measure hallucination rate on your own data?

Collect a few hundred real questions or documents with known correct answers, run them through your system, and have reviewers or a calibrated judge model mark any unsupported or wrong claim. The share of outputs with at least one error is your rate. Repeat after every model or prompt change.

If you want to know what hallucination rate your own workflows would see, and which tasks are safe to automate, start with a free AI audit. We will look at where AI fits in your business and how to build it so the errors are caught before your customers see them.