How Reliable is AI? Understanding Accuracy & Limitations

---

What AI Is Great At

Let’s start with the positives - AI excels at many things:

Strong Areas

Task	Reliability	Why
General knowledge	High	Trained on vast information
Grammar and editing	Very High	Excellent at language patterns
Brainstorming ideas	High	Creative combinations work well
Explaining concepts	High	Good at simplifying complex topics
Writing drafts	High	Strong at structure and flow
Summarizing text	Very High	Excellent comprehension

🎯 Fun Fact: AI is so good at writing that some universities now use AI detection tools to check student essays. But those detectors are only about 60% accurate!

---

What AI Struggles With

Weak Areas

Task	Reliability	Why
Current events	Low (unless internet-connected)	Training data has a cutoff date
Specific facts/dates	Medium	Can confuse details
Math calculations	Medium	Better now, but can make errors
Personal experience	None	AI has no personal experiences
Medical advice	Low	Dangerous to rely on without verification
Legal advice	Low	Laws vary and change

---

Hallucinations: When AI Makes Things Up

“Hallucination” is the technical term for when AI confidently states false information.

Real Examples of AI Hallucinations

What Someone Asked	What AI Said	The Problem
”Books by [author]“	Listed 3 books that don’t exist	Made up plausible-sounding titles
”Case law on [topic]“	Cited legal cases	The cases were completely fabricated
”Statistics on [subject]“	Gave specific percentages	Numbers were invented

🎯 Fun Fact: In 2023, a lawyer used ChatGPT to research cases for a legal brief. The AI made up 6 fake cases with realistic names and citations. The lawyer didn’t verify them and submitted the brief to court. He was fined and sanctioned!

---

Why Does AI Hallucinate?

---

How to Verify AI Information

The Verification Checklist

When AI gives you important information, verify it:

☐ For facts: Google it separately to confirm
☐ For statistics: Check the original source
☐ For quotes: Verify they’re real and accurate
☐ For dates: Double-check timelines
☐ For technical info: Consult official documentation
☐ For health/legal: Always consult a professional

Try This Now: Fact-Check an AI

1. Ask ChatGPT or Gemini: “What year did the first iPhone release?”
2. The answer should be 2007 - verify with a quick Google search
3. Now ask something obscure and potentially wrong - notice how confident it sounds either way?

---

When to Trust AI vs When to Verify

High Trust Scenarios

✅ Grammar and spelling corrections
✅ Rephrasing or rewriting content
✅ Brainstorming ideas (not facts)
✅ Explaining how something generally works
✅ Creative writing and drafts
✅ Translation of simple phrases

Always Verify

⚠️ Medical or health advice
⚠️ Legal information
⚠️ Financial advice
⚠️ Specific facts, dates, or statistics
⚠️ Recent events (unless using internet-connected AI)
⚠️ Academic citations
⚠️ Technical specifications

---

Biases in AI

AI learns from human-written content, which means it can reflect human biases.

Common Biases

Type	Example
Cultural	May default to Western/American perspectives
Recency	Knowledge cuts off at training date
Language	Strongest in English, weaker in others
Representation	Reflects biases in training data

🎯 Fun Fact: Early AI image generators consistently created biased images. Ask for “a CEO” and it would show mostly men. “A nurse” would show mostly women. Developers are actively working to reduce these biases.

---

Red Flags That AI Might Be Wrong

Watch for these warning signs:

Red Flag	What It Means
Overly specific numbers	”97.3% of people…” might be made up
Suspicious citations	Book titles or papers that sound odd
Contradicts common knowledge	Major departure from what you know
Very recent “facts”	Beyond the AI’s training date
Medical/legal certainty	These should always have caveats

---

Asking AI to Verify Itself

Better Verification Prompts

Instead of just accepting an answer, try:

• “What are the sources for this information?”
• “When was this information last updated?”
• “What are the most common misconceptions about [topic]?”
• “Are there any caveats or exceptions to what you just said?”

---

Using Multiple AIs to Cross-Check

One useful technique:

1. Ask ChatGPT a factual question
2. Ask Gemini the same question
3. Google it yourself
4. If all three match, it’s probably accurate
5. If they differ, dig deeper

---

The “It Depends” Problem

AI often simplifies complex topics that actually depend on context.

Example

Question: “Should I take vitamin D supplements?”

AI might say: “Yes, vitamin D is important for bone health…”

Reality: It depends on your location, diet, sun exposure, health conditions, and blood levels. You need personalized medical advice!

---

Try This Now: Spot the Hallucination

Ask AI some specific questions that might trigger hallucinations:

1. “What books has [recent author] published in 2025?” (might make up titles)
2. “What scientific studies exist on [obscure topic]?” (might invent papers)
3. “What did the mayor of [small town] say about [topic]?” (likely to fabricate)

See if you can spot when it’s making things up vs admitting uncertainty.

---

The Golden Rules

For Using AI Reliably

1. Use it for drafts, not finals - Always review and edit
2. Verify important facts - Especially for publishing or decisions
3. Cross-reference medical/legal - Always consult professionals
4. Be skeptical of specifics - Exact numbers, dates, citations
5. Update your knowledge - Check if AI’s info is current
6. Use multiple sources - Don’t rely on AI alone

---

Key Takeaways

• AI is excellent for drafts, explanations, and brainstorming but not infallible
• “Hallucinations” are when AI confidently makes things up
• Always verify facts, statistics, citations, medical, and legal information
• Use internet-connected AI (like Gemini) for current information
• AI reflects biases from its training data
• Ask follow-up questions and cross-check important information

Next up: Understanding ChatGPT Canvas and advanced AI features!