Why AI Makes Things Up

Same mechanism, different outcomes. Fluency is not accuracy.

The Analogy

Why AI Makes Things Up

A confident storyteller who, when they don't remember the details, fills in the gaps with something that sounds right. They're not lying. They genuinely don't know the difference between what they remember and what they invented.

Use the arrows below, the dots above, or your keyboard arrow keys to move through the stages.

Stage 1 -- Same Mechanism, Different Outcome

Two Prompts, One Process

When It Works
What is photosynthesis?
When It Doesn't
Tell me about Dr. Sarah Chen's research at MIT
Same process. Same mechanism. Different outcome.

Both generated the same way. Photosynthesis matches reality. The researcher is fiction.

Stage 2 -- Fluency vs. Accuracy

Can You Spot the Fiction?

AI-generated response about the history of the Rosetta Stone
Click each sentence to reveal whether it is accurate or fabricated

Fluency and accuracy are independent. Beautifully written and completely false can coexist.

Stage 3 -- Why It Can't Be Fixed

Creativity and Hallucination Share the Same Source

More Creative More Accurate

Hallucination isn't a bug for the next version. It's the same mechanism that makes AI creative. You can tune the balance but can't eliminate it.

Stage 4 -- Risk Factors

Where Hallucination Hides

Knowing where hallucination is likely is your best defense.

Takeaway

Every response is a prediction, not a retrieval. The more specific the claim, the higher the risk. If it matters, check it.

Now you know why AI can produce confident fiction. But there's another limit -- why does AI seem to forget what you told it earlier in the conversation? →