Class 6 · Age 10–11 · AI for Students · Lesson 10 of 12
When AI Gets It Wrong
AI can sound completely confident and still be completely wrong. Learn why AI makes mistakes, how to spot them, and how to fact-check anything AI tells you.
Read all 8 sections below while we prepare the video.
📖 Meet Aryan
The Project That Went Wrong
Aryan is 11 years old and lives in Delhi. His Social Science teacher assigned a project on the Maurya Empire — specifically about Emperor Chandragupta Maurya and the founding of India's first great empire.
Aryan was excited. He had seen his older cousin use AI for assignments and it seemed incredibly fast. He opened ChatGPT, asked it to explain the Maurya Empire, and wrote down everything it said. The AI was confident, detailed, and impressive — it gave him dates, names, and specific facts. Aryan submitted his project feeling great about it.
The teacher returned it with corrections marked throughout. Several dates were wrong. One minister the AI mentioned — "Amitraghat" — did not appear to exist in any historical record. The AI had invented a plausible-sounding name. The founding date it gave was off by decades.
"Aryan," said his teacher, "the AI gave you a very confident answer. But confident and correct are not the same thing."
That evening, Aryan's older sister Divya, who studies journalism, sat with him. "This is the most important lesson about AI," she said. "AI never tells you when it's uncertain. It gives a wrong answer in the same calm, fluent voice it gives a right answer. Your job is to check."
⚠️ The problem wasn't that Aryan used AI — it was that he trusted AI without checking. This lesson teaches you to use AI as a starting point and always verify before you submit, share, or act on what AI tells you.
Section 1 of 8
🤔 When AI Sounds Confident But Is Wrong
One of the strangest things about AI mistakes is that they don't look like mistakes. When AI gets something wrong, it doesn't say "I'm not sure about this" or "you should check this". It writes in exactly the same fluent, confident, detailed way as when it's correct.
This is dangerous. If you see a person looking uncertain, you know to double-check. If you see a person looking completely confident, you might trust them. AI always looks completely confident — even when it is completely wrong.
Question asked to AI: "Who founded the Maurya Empire and when?"
AI answer (contains error)
"The Maurya Empire was founded by Chandragupta Maurya in 312 BCE, with the support of his advisor Chanakya and a minister named Amitraghat who oversaw the treasury of the new empire."
Historical fact
"The Maurya Empire was founded by Chandragupta Maurya around 321 BCE with the guidance of his mentor Chanakya (Kautilya). There is no historical record of a minister named 'Amitraghat' — this appears to be a fabricated name."
Aryan's AI answer was mostly right — Chandragupta Maurya did found the Maurya Empire and Chanakya was his mentor. But the date was wrong, and the AI invented an official who did not exist. If Aryan had cross-checked with his textbook, he would have caught both errors in two minutes.
The core rule: Treat everything AI tells you as a first draft that needs checking — not a final answer you can trust immediately. This is especially important for facts you will share with others: school work, conversations, social media posts.
Section 2 of 8
📋 Types of AI Mistakes
AI doesn't make just one type of mistake. Here are the six most common categories — with Indian examples so you can recognise them in real life:
❌
Factual Errors
Stating wrong facts confidently. Example: giving the wrong year for India's independence, wrong capital for a state, or invented historical details.
🔢
Maths Mistakes
AI language models can make arithmetic errors. Example: AI calculates a train ticket discount wrong, or gets a percentage calculation wrong.
📅
Stale Information
AI was trained until a cutoff date. It cannot know current events, new laws, recent cricket results, or latest exam patterns.
⚖️
Biased Output
AI can reflect stereotypes or imbalances from its training data. Example: AI descriptions of professions might skew towards certain genders or regions.
📰
Hallucinated Sources
AI invents books, papers, articles, or quotes that don't exist. Example: AI cites "a 2019 NCERT report" that was never published.
🔄
Context Confusion
AI misunderstands your question and answers a different, related question confidently. Example: You ask about Hyderabad, Telangana — AI answers about Hyderabad in Pakistan (Sindh).
Important: All six types of mistakes can look identical on the surface — fluent, detailed, confident writing. The only way to tell them apart from correct information is to check against a reliable source.
Section 3 of 8
🧠 Why Does AI Hallucinate?
The word "hallucination" is the technical term AI researchers use for when AI generates content that sounds plausible but is factually wrong or invented. It comes from the idea of seeing something that isn't really there.
Why does this happen? Here is a simple explanation:
Why AI hallucinations happen — step by step
1
AI is trained on billions of text documents — books, websites, Wikipedia, articles. It learns patterns of how words, ideas, and facts connect.
2
When you ask a question, AI generates a response token by token — predicting what word comes next based on those patterns. It is not searching a database for verified facts.
3
If the training data had gaps, errors, or conflicting information — AI fills the gap with the most statistically plausible-sounding content. This can be wrong.
4
AI has no way to internally verify whether its output is true. It has no separate fact-checking system built in. It produces fluent text and is not always certain when it is wrong.
5
The result: AI generates confident-sounding, well-written content that can contain invented names, wrong dates, or fabricated statistics — with no warning to you.
The key insight: AI is not lying — it doesn't know it's wrong. It is doing exactly what it was designed to do: generate fluent, contextually appropriate text. "Being accurate" and "being fluent" are not the same thing, and AI optimises for fluency. Your job is to check for accuracy.
Section 4 of 8
🔢 AI and Numbers — Be Extra Careful
Many students assume that because computers are good at maths, AI must be perfect at calculations. This is a common and dangerous misconception. AI language models are text prediction systems — they were not built primarily as calculators.
Type of maths
AI reliability
What to do
Simple arithmetic (2+2, 15×4)
Usually correct but can still fail
Double-check with a calculator for any work you submit
Multi-step word problems
Unreliable — AI often makes setup errors
Solve yourself, use AI only to check your approach
Statistics and percentages
Low reliability — often fabricated
Always find the original source
Logical reasoning problems
Inconsistent — depends on the question type
Verify with a textbook or teacher
Complex algebra or geometry
Can work but makes errors in multi-step problems
Use a dedicated maths tool like Wolfram Alpha or check manually
Example: Aryan asks AI a maths word problem
Question given to AI
"A train travels 360 km in 4 hours for the first part of a journey, then 210 km in 3 hours for the second part. What is its average speed for the whole journey?"
AI incorrect answer (sometimes seen)
"The average speed is 85 km/h." (Simply averaging 90 and 70 — incorrect method)
Correct calculation
Total distance = 360 + 210 = 570 km. Total time = 4 + 3 = 7 hours. Average speed = 570 ÷ 7 = 81.4 km/h. The correct method uses total distance ÷ total time, not the average of the two speeds.
Safe approach: For all maths in school work, solve the problem yourself first. You may use AI to explain a concept you don't understand — but always verify the final answer with your own calculation or a trusted maths tool.
Section 5 of 8
⚖️ AI Bias — When AI Is Unfair
AI is trained on human-generated data — and human data contains biases, stereotypes, and imbalances. As a result, AI can produce outputs that are unfair, stereotyped, or represent some groups better than others.
This is not always obvious, which makes it important to think critically about AI output — especially when AI describes people, professions, communities, or cultures.
Type of bias
Example in Indian context
Why it matters
Gender bias
AI consistently generates male names when asked for "a scientist" or "an engineer," and female names for "a nurse" or "a teacher"
Reinforces harmful stereotypes about who belongs in what profession
Urban bias
AI examples about "Indian life" often feature cities, ignoring that most Indians live in rural areas
Misrepresents the majority of Indian experience
Language bias
AI performs much better in English than in Telugu, Hindi, or other Indian languages — more training data exists in English
AI may give worse answers for Indian-language queries
Western-centric bias
AI may give better historical coverage of European events than of South Asian events — more English-language content exists about the West
Can lead to a distorted view of world history and knowledge
Recency bias
Events that happened frequently online are better represented in AI training data than older or less-documented events
AI may have gaps in knowledge about important topics that were less documented
What to do: When AI gives you information about people, communities, professions, or cultures — ask yourself: "Who might be missing from this picture?" If the answer concerns you, search for sources that represent a broader range of perspectives. AI is not the only source of knowledge.
Section 6 of 8
📅 AI Knowledge Cutoffs — Why AI Doesn't Know "Now"
Every AI language model has a knowledge cutoff — a date beyond which it has no information. It is trained on data collected up to that date, and then it stops learning automatically. The world keeps changing; AI's knowledge does not update in real time.
Never ask AI for current information without checking the date: Current news, recent match results, new government policies, latest exam dates, new scientific discoveries, current prices, recent court judgments — these could all be outdated or completely unknown to AI.
Question type
AI reliability
Better source
"Who is the current Chief Minister of Telangana?"
⚠️ May be outdated
Official state government website or a current news site
"What are the CBSE Class 6 maths topics this year?"
⚠️ May be outdated
CBSE official website or your current textbook
"Who won the 2025 IPL?"
⚠️ May not know
Any sports news website
"Explain the water cycle"
✅ Reliable
AI is fine for stable scientific concepts
"What does democracy mean?"
✅ Reliable
AI is fine for definitions of established concepts
"What caused the 2008 financial crisis?"
🔶 Check key facts
AI is mostly reliable for well-documented past events but verify statistics and dates
Tip: You can ask AI directly: "What is your knowledge cutoff date?" A well-designed AI will tell you. If a topic involves anything that may have changed in the last year or two, always verify with a current source.
Section 7 of 8
🔍 How to Fact-Check AI Output — 5 Steps
Divya (Aryan's sister, the journalism student) taught him the same fact-checking process journalists use. It works just as well for school students checking AI output.
1
Identify the checkable claims
Read the AI output and underline every specific fact — dates, names, numbers, statistics, quotes, locations. These are the things you can verify. Opinions and general descriptions are harder to check, but specific facts can be confirmed or denied.
2
Check your primary source first
For school work, your NCERT or state board textbook is the primary source. If AI says something different from your textbook, your textbook is almost certainly right for exam purposes. For general knowledge, official websites, encyclopedias, or reputable news sites are primary sources.
3
Find at least two independent sources
If your textbook doesn't cover the topic, find two independent reliable sources that confirm the same fact. "Independent" means they did their own research — not two websites copying from each other. Reliable sources include: government websites (.gov.in), established newspapers (The Hindu, Times of India), encyclopedias (Britannica), and official educational bodies (NCERT, ISRO, etc.).
4
Be suspicious of very specific numbers
If AI gives you a very specific statistic — "67.3% of Indian farmers…" or "₹2.4 lakh crore was spent on…" — be extra careful. AI frequently fabricates precise numbers. Ask: "Does AI tell me where this number comes from?" If not, find the original source before using it.
5
Use Indian fact-checking resources
For news-related claims: Boom Live (boomlive.in), Alt News (altnews.in), and The Quint WebQoof are dedicated Indian fact-checking platforms. For historical and academic facts: NCERT textbooks, India.gov.in, and the National Archives of India (nationalarchives.nic.in) are reliable official sources.
Aryan's revision: After learning this process, Aryan redid his project. He used AI to get a quick overview of the Maurya Empire — then checked every single date and name against his NCERT Class 6 Social Science textbook. Two minutes of checking caught all the errors. His revised project was excellent.
Section 8 of 8
🛡️ Building Critical AI Habits
The goal of this lesson isn't to make you afraid of AI — it's to make you a smarter user. The best AI users are not the ones who trust AI most; they are the ones who know when to trust it, when to check it, and when to look elsewhere.
Red flags — be extra careful when you see these:
🔢
Specific numbers without sources
"73.6% of students…" — where did this come from? Verify before using.
📚
Named books, reports, or quotes
AI frequently invents academic citations. Search for the source directly.
📅
Recent or current events
Anything that happened recently may be outside AI's knowledge cutoff or wrong.
👤
Specific person's statements
"Dr. X said…" — AI may fabricate quotes. Find the original speech or article.
🗺️
Precise geographic or local facts
Local details about specific cities, districts, or villages are often least reliable.
⚖️
Legal or health information
Laws and medical facts change. Always verify with official or professional sources.
Good habits to build now:
📖
Textbook first
For school subjects, check your textbook before trusting AI. Your textbook is aligned to your exam.
🔍
Two-source rule
Never use a fact in your work if you only have AI as a source. Find at least one other reliable source.
❓
Ask "how do I know this is true?"
Develop the habit of asking this question for every important fact — whether from AI, a website, or even a friend.
🗓️
Check the date
For anything current, ask when the information was last updated. AI may not know things from the past year or two.
💬
Tell others when AI helped
If you use AI and then verify, tell your teacher. Showing your verification process is a sign of maturity.
🎯
Use AI for concepts, check facts
AI is reliable for explaining concepts and definitions. Be extra careful with specific names, dates, and numbers.
The bigger picture: The habit of checking facts — whether from AI, social media, TV news, or friends — is one of the most important thinking skills of this century. People who develop this habit early will be better students, better professionals, and better citizens. Start now, with AI. Apply it everywhere.
🧠 Lesson 10 Quiz — 10 Questions
1. What is an AI "hallucination"?
2. Rohan asks AI: "Who is the current Chief Minister of Andhra Pradesh?" AI gives a confident answer. What should Rohan do?
3. AI gives you the answer to a maths word problem for your homework. What should you do?
4. Why does AI have a "knowledge cutoff"?
5. What is the BEST way to fact-check an important fact that AI gave you?
6. AI tells you: "According to a 2022 study by the Indian Education Research Institute, 71.4% of Class 6 students struggle with fractions." You want to use this in your school essay. What should you do?
7. What does "AI bias" mean when it comes to AI outputs?
8. For which of these questions is AI MOST likely to give a reliable answer?
9. Priya asks AI about Chandragupta Maurya and AI confidently states that his advisor was "Amitraghat." Priya cannot find this name in any textbook or reliable website. What should she conclude?
10. What is the BEST overall approach when using AI for school research?
0/10
Your Score
📝 Worksheet — AI Fact-Check Challenge
Tip: in the print dialog, choose "Save as PDF" to download.
Choose any topic from your current school curriculum. Ask AI 5 specific factual questions about it. Then check each answer against your textbook or a reliable website. Record your findings below.
Topic I chose: _______________________________ Subject: _______________________
#
Question I asked AI
What AI said
What I verified in my source
Correct / Wrong / Couldn't verify
1
2
3
4
5
My score: AI was correct ___ / 5 times.
My reflection: What surprised me about AI's accuracy? What will I do differently when I use AI for school work next time?
Do this challenge every month with a different subject. Track whether AI accuracy varies by subject — you may find that AI is more reliable for some topics than others.
👨👩👧 Note for Parents and Teachers
This lesson teaches Class 6 students one of the most critical AI literacy skills: understanding that AI can be wrong — and learning how to check. The lesson covers AI hallucinations, types of AI errors, the knowledge cutoff problem, AI bias, and a practical 5-step fact-checking process.
What your child learned today:
What AI hallucinations are — why AI invents plausible-sounding but false information
The six types of AI mistakes: factual errors, maths mistakes, stale information, bias, hallucinated sources, and context confusion
Why AI language models can fail at maths — and why calculators are still needed
How AI bias reflects imbalances in training data — gender, language, geographic, and cultural bias
Why AI has a knowledge cutoff and which question types are safe vs risky
The 5-step fact-checking process: identify claims → check primary source → find two independent sources → be suspicious of specific numbers → use Indian fact-checking resources
Red flags in AI output and critical AI habits to build
How to support at home:
Ask your child to show you the AI Fact-Check Challenge worksheet and explain what they found
When your child says "AI said…" — ask them: "Did you check?" This single question builds the most important habit
Model fact-checking yourself: when you read something surprising in WhatsApp or social media, say out loud "let me check this before I forward it"
For teachers: The AI Fact-Check Challenge is an excellent formative assessment — students demonstrate both subject knowledge and critical thinking by verifying AI output against their curriculum. Consider making it a regular monthly activity. It builds research skills alongside AI literacy.
