How Reliable is AI? Understanding Accuracy & Limitations
Learn when to trust AI, when to verify, and how to spot when AI is making things up.
Can You Guess?
If ChatGPT or Gemini gives you an answer, how accurate is it?
What AI is great at
Let’s start with the positives - AI excels at many things:
Strong areas
| Task | Reliability | Why |
|---|---|---|
| General knowledge | High | Trained on vast information |
| Grammar and editing | Very High | Excellent at language patterns |
| Brainstorming ideas | High | Creative combinations work well |
| Explaining concepts | High | Good at simplifying complex topics |
| Writing drafts | High | Strong at structure and flow |
| Summarizing text | Very High | Excellent comprehension |
What AI struggles with
Weak areas
| Task | Reliability | Why |
|---|---|---|
| Current events | Low (unless internet-connected) | Training data has a cutoff date |
| Specific facts/dates | Medium | Can confuse details |
| Math calculations | Medium | Better now, but can make errors |
| Personal experience | None | AI has no personal experiences |
| Medical advice | Low | Dangerous to rely on without verification |
| Legal advice | Low | Laws vary and change |
Can You Guess?
You ask ChatGPT "What's the current president of France?" and it gives a wrong answer. Why?
Hallucinations: When AI makes things up
“Hallucination” is the technical term for when AI confidently states false information.
Real examples of AI hallucinations
| What Someone Asked | What AI Said | The Problem |
|---|---|---|
| ”Books by [author]“ | Listed 3 books that don’t exist | Made up plausible-sounding titles |
| ”Case law on [topic]“ | Cited legal cases | The cases were completely fabricated |
| ”Statistics on [subject]“ | Gave specific percentages | Numbers were invented |
Why does AI hallucinate?
Can You Guess?
Why would an AI make up information instead of saying "I don't know"?
How to verify AI information
The verification checklist
When AI gives you important information, verify it:
☐ For facts: Google it separately to confirm
☐ For statistics: Check the original source
☐ For quotes: Verify they’re real and accurate
☐ For dates: Double-check timelines
☐ For technical info: Consult official documentation
☐ For health/legal: Always consult a professional
Fact-Check an AI
When to trust AI vs when to verify
High trust scenarios
✅ Grammar and spelling corrections
✅ Rephrasing or rewriting content
✅ Brainstorming ideas (not facts)
✅ Explaining how something generally works
✅ Creative writing and drafts
✅ Translation of simple phrases
Always verify
⚠️ Medical or health advice
⚠️ Legal information
⚠️ Financial advice
⚠️ Specific facts, dates, or statistics
⚠️ Recent events (unless using internet-connected AI)
⚠️ Academic citations
⚠️ Technical specifications
Can You Guess?
You're writing a blog post about healthy eating. Should you trust AI's suggestions?
Biases in AI
AI learns from human-written content, which means it can reflect human biases.
Common biases
| Type | Example |
|---|---|
| Cultural | May default to Western/American perspectives |
| Recency | Knowledge cuts off at training date |
| Language | Strongest in English, weaker in others |
| Representation | Reflects biases in training data |
Red flags that AI might be wrong
Watch for these warning signs:
| Red Flag | What It Means |
|---|---|
| Overly specific numbers | ”97.3% of people…” might be made up |
| Suspicious citations | Book titles or papers that sound odd |
| Contradicts common knowledge | Major departure from what you know |
| Very recent “facts” | Beyond the AI’s training date |
| Medical/legal certainty | These should always have caveats |
Asking AI to verify itself
Can You Guess?
Can you ask AI "Are you sure?" to make it check its answer?
Better verification prompts
Instead of just accepting an answer, try:
- “What are the sources for this information?”
- “When was this information last updated?”
- “What are the most common misconceptions about [topic]?”
- “Are there any caveats or exceptions to what you just said?”
Using multiple AIs to cross-check
One useful technique:
- Ask ChatGPT a factual question
- Ask Gemini the same question
- Google it yourself
- If all three match, it’s probably accurate
- If they differ, dig deeper
The “it depends” problem
AI often simplifies complex topics that actually depend on context.
Example
Question: “Should I take vitamin D supplements?”
AI might say: “Yes, vitamin D is important for bone health…”
Reality: It depends on your location, diet, sun exposure, health conditions, and blood levels. You need personalized medical advice!
Can You Guess?
Why does AI struggle with "it depends" situations?
Spot the Hallucination
Ask AI some specific questions that might trigger hallucinations:
See if you can spot when it’s making things up vs admitting uncertainty.
The golden rules
For using AI reliably
- Use it for drafts, not finals - Always review and edit
- Verify important facts - Especially for publishing or decisions
- Cross-reference medical/legal - Always consult professionals
- Be skeptical of specifics - Exact numbers, dates, citations
- Update your knowledge - Check if AI’s info is current
- Use multiple sources - Don’t rely on AI alone
Key Takeaways
- ✓ AI is excellent for drafts, explanations, and brainstorming but not infallible
- ✓ 'Hallucinations' are when AI confidently makes things up
- ✓ Always verify facts, statistics, citations, medical, and legal information
- ✓ Use internet-connected AI (like Gemini) for current information
- ✓ AI reflects biases from its training data
- ✓ Ask follow-up questions and cross-check important information
Next up: Understanding ChatGPT Canvas and advanced AI features