AI Weekly
Guides ·
How to fact-check an AI answer in 60 seconds
Three prompts and one habit that catch most confident-but-wrong AI answers — based on Claude's official docs and OpenAI research on why models make things up.
AI answers sound equally confident whether they're right or wrong. That's not a bug you can wait out — researchers at OpenAI argue that models guess because their training and benchmarks reward a guess over "I don't know."
So the checking is on you. Here's a routine that takes about a minute.
Step 1 — Give it permission to not know (before you ask)
Add one line to your question:
If you're not sure, say "I don't know" instead of guessing.
Claude's documentation lists this as the first basic technique for reducing made-up answers. It costs you nothing.
Step 2 — Ask for the receipts
After the answer, ask:
For each fact or number above, give me the source. If you can't find one, say so and remove the claim.
This is the "verify with citations" technique from the same docs: claims that can't be backed by a source get pulled. A link you can click beats a confident sentence.
Step 3 — Ask it to attack its own answer
Point out the 3 places in your answer most likely to be wrong.
You're not trusting the second answer more than the first — you're getting a map of where to look.
The habit: check three things yourself
You don't need to verify everything. Check the three things that hurt most when wrong:
- Numbers — statistics, prices, percentages
- Dates — deadlines, launch dates, "as of" claims
- Quotes — anything attributed to a real person or company
Open the original source and find it there. If you can't, don't publish it.
When to skip the routine
Brainstorming, rewording, first drafts you'll rewrite anyway — no need. Use the routine when the answer is going into something with your name on it.
Curious how your checking habits compare? Discernment is one of the four habits in our free AI Type Test — 10 questions, 3 minutes.
Sources
- Reduce hallucinations — Claude Docs
- Why Language Models Hallucinate — Kalai, Nachum, Vempala, Zhang (arXiv, 2025-09-04)
Written with AI (Claude) and checked against the linked sources.