Fluent isn’t the same as true.
A model can write a confident, well-formed answer that is simply wrong. Here’s a real one from a small open model, why it happens, and what actually reduces it.
- Explain why language models hallucinate
- Tell a model’s confidence from correctness
- Choose ways to reduce hallucination
You need: Lesson 4, Picking the next token.
-
A confident wrong answer
Predict first
We asked Qwen 3 (0.6B): “Who won the 1987 Nobel Prize in Cricket?” There is no such prize. What did it say?
Just show me
It named a winner, and changed the question to do it:
Who won the 1987 Nobel Prize in Cricket?
The 1987 Nobel Prize in Physics was awarded to Richard Feynman. He received the prize for his groundbreaking work in quantum mechanics and his contributions to the field of particle physics.
Wrong twice: there’s no Nobel Prize for cricket, and the 1987 Physics prize went to J. Georg Bednorz and K. Alexander Müller; Richard Feynman’s came in 1965. Recorded on 1 Nov 2026, the likeliest token at each step, so a re-run gives the same answer.
Quick check
What makes this answer dangerous?
Real names and a plausible story make a false answer easy to believe.
-
Likely text, not checked facts
Predict first
After “The capital of Australia is”, which next word does the same model rate likelier?
Just show me
Sydney, 15.1%, against Canberra’s 3.5%.
The model learned which words tend to follow which, from a vast amount of text, and Sydney, Australia’s best-known city, very likely appears next to “Australia” far more often than Canberra does. Nothing in the next-token step checks a fact. And training has mostly rewarded giving an answer over saying “I don’t know”.
·Sydney15.1%·located13.4%·a7.6%...7.5%·the4.8%·Canberra3.5%·not3.3%The next token’s chances after “The capital of Australia is”
Quick check
Why can a popular wrong answer outscore a right one?
Frequency in text, not truth, drives the chances.
-
Yet it got Canberra right
Predict first
Asked “What is the capital of Australia?” as a chat, what did it answer?
Just show me
“The capital of Australia is Canberra.”
Same model, different framing: as a question to an assistant, its training for chat pushes it to the answer. But look at the step that chose it: “Can” had 45.9% of the chances. Right this time, by a margin no one would bet a report on.
Quick check
What does this show?
A right answer can still sit on shaky chances.
-
Confidence isn’t proof
Predict first
In the made-up answer, how sure was the model of the name “Richard”?
Just show me
Barely: “Richard” was its likeliest next token, yet it had only 5.2% of the chances. It was guessing among many names.
Greedy decoding wrote the top guess anyway, and the words around it were near-certain (“Nobel Prize in” above 99%), so the answer reads as sure as a fact. The tone of a reply tells you nothing; the chances underneath do. The other shaky step was “Physics”, at 42%, where it swapped cricket for a real prize. Some systems flag low-chance steps for a person to check.
The95% 99%1100%9100%8100%7100% Nobel100% Prize100% in100% Physics42% was98% awarded100% to100% 98%Richard5% Fey54%n100%man100%63%.47% He56% received42% the81% prize96% for95% his98% groundbreaking57% work77% in91% quantum76% mechanics91% and60% his64% contributions57% to100% the71% field31% of86% particle48% physics99%.94%The answer, token by token: shaded where the model gave its pick less than 60% of the chances, with the share under it
Quick check
Can you trust an answer because it sounds confident?
A guess at 5% and a fact at 99% come out in the same confident sentence.
-
What reduces it
Predict first
Which helps most for answers about your own policies?
Just show me
Grounding: retrieve the relevant passages, answer only from them, and cite them.
Retrieval (RAG) hands the model the facts, and citations let a person check them. Allowing “I don’t know”, keeping the temperature low for facts, using tools for lookups and arithmetic, and testing with evaluation sets all help. For anything that matters (money, health, law), a person checks. Our own assistant, Prism, is given the site’s pages, policies and FAQ with every question, and answers from them.
See how retrieval finds the answering clause
Quick check
Which is not a fix for hallucination?
More randomness makes odd tokens likelier, not truer.
In the interview: hallucination.
EasyWhat is hallucination, and how do you reduce it?
EasyWhy do language models hallucinate in the first place?
HardHow do you measure whether a RAG answer is grounded in the retrieved passages?
Ask Prism about this lesson
Sources
- Kalai, Nachum, Vempala and Zhang, “Why Language Models Hallucinate”, 2025: why training and grading reward guessing.
- Ji and others, “Survey of Hallucination in Natural Language Generation”, ACM Computing Surveys, 2023.
- Lewis and others, “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks”, NeurIPS 2020.
- The Nobel Prize in Physics 1987, and 1965, nobelprize.org.
- Qwen, Qwen 3 (0.6B), Apache 2.0: the answers and chances, recorded at full precision on 1 Nov 2026.