Hallucination
Definition
When an AI model generates plausible-sounding but factually incorrect or fabricated information, leading to potential misinformation and confusion.
Use Cases
- Mata v. Avianca (U.S. federal court case involving Avianca): Legal research and drafting support — An attorney used an AI chatbot to help draft a legal filing; the filing included citations to court cases that were later found to be non-existent. (The fabricated citations were identified by the court, leading to sanctions and widespread attention as a cautionary example of AI hallucinations in high-stakes workflows.)
- Microsoft (Bing Chat / Copilot early rollout): Consumer search and question answering — Microsoft integrated large language models into Bing Chat to answer questions conversationally; some early responses included confident inaccuracies, which the company iterated on with product changes and safety mitigations. (Demonstrated both the value and risk of LLM-based search; accelerated industry focus on grounding, citations, and safety testing to reduce hallucinations.)
Frequently Asked Questions
- What's the difference between hallucination and misinformation?
- Hallucination is when an AI model generates incorrect details because of how it predicts text (it may “make up” facts). Misinformation is false information more broadly, regardless of source or intent. A hallucination can create misinformation, but misinformation can also come from humans, bad data, or deliberate deception.
- When should I use (or worry about) hallucination in my AI application?
- You should plan for hallucinations anytime you use a generative AI model to produce factual statements, citations, numbers, medical/legal guidance, or instructions that could cause harm if wrong. For low-stakes tasks (brainstorming, rewriting, summarizing your own provided text), hallucinations are less risky but still possible. In high-stakes workflows, use grounding (RAG), require citations to trusted sources, add human review, and run evaluations before production.
- How much does hallucination cost?
- Hallucination isn’t a billable feature, but it creates costs: (1) rework and human review time, (2) customer support and reputational damage from wrong answers, (3) compliance/legal exposure in regulated domains, and (4) extra engineering to reduce it (retrieval systems, evaluations, monitoring). Cloud costs often rise when you add mitigations like retrieval (vector databases/search queries), longer prompts with context, more model calls for verification, and continuous evaluation pipelines.
Category: ai-ml
Difficulty: intermediate
Related Terms
See Also