Prevent AI Hallucinations: Why AI Makes Things Up

An AI hallucination is an answer that sounds sure of itself and is wrong. AI makes things up because a language model writes likely words, not checked facts, and it will still write something when it has nothing to go on. You cannot reach zero. You can make it rare by answering from your own approved content, setting clear limits, allowing "I don't know", handing over to a person, and testing before launch.

This article is for owners and managers who run, or plan to run, an AI assistant on their website, in WhatsApp or in support email. It stays in plain words and focuses on what you can control.

Key takeaways

• A model predicts plausible text. Fluent does not mean true.
• The most effective step is to let the assistant answer only from your own approved content, and to say so when the answer is not there.
• Telling a model "do not hallucinate" is not a control. Limits, sources and tests are.
• Wrong answers can cost real money: a tribunal made an airline pay for what its chatbot said.
• Keep a human behind anything that costs too much if it goes wrong.

What is an AI hallucination?

An AI hallucination is a confident answer that is false or has no support in any source. The model is not lying and does not know it is wrong. It is producing text that fits the question.

Anthropic describes it in its documentation as text that is "factually incorrect or inconsistent with the given context" (Anthropic, Reduce hallucinations). The second half matters for business. A chatbot that contradicts your own price list is hallucinating even if the number it gave is a plausible one.

A typical case: a customer asks a clinic's chatbot, "Do you open on Saturday afternoon?" The clinic only publishes weekday hours. The model has nothing to read, so it answers "Yes, we are open until 6 pm on Saturdays." It sounds right, and the customer turns up to a locked door.

Why does AI make things up?

AI makes things up because a language model is built to continue text with what is most likely, and nothing in that process checks the result against reality. Three causes matter most for a business.

It predicts words, it does not look things up. The model learned patterns from a huge amount of text. It does not hold a database of your business. Ask about your returns policy and it produces what a returns policy usually says.

There are gaps in what it was given. If your opening hours, prices or terms are not in front of it when it answers, it fills the gap with something plausible. Rare, specific or recent facts are the weakest spots.

It is rewarded for answering. A 2025 research paper argues that models hallucinate partly because training and testing reward confident guessing over admitting uncertainty, much like a test that gives no points for leaving a question blank (Kalai, Nachum, Vempala and Zhang, Why Language Models Hallucinate, arXiv, September 2025). The authors propose changing how benchmarks score answers. That is for model makers, not for you. What it tells you is useful: by default, a model leans towards answering, so you have to build in the permission to say "I don't know".

What are examples of AI hallucinations in a real business?

Hallucinations in business chatbots are usually small, specific and believable. These are the kinds that cost money.

The best-known case is Air Canada. In 2024 a Canadian tribunal ruled that the airline was responsible for what its website chatbot told a customer about bereavement fares, rejected the idea that the chatbot was a separate entity, and ordered it to pay the customer $812.02 (FAnews report, 28 February 2024). The sum was small. The principle was not: what your assistant says is treated as something you said.

How do you reduce hallucinations in a business chatbot?

You reduce them by controlling what the assistant may answer from and what it does when it cannot. Five measures do most of the work.

1. Ground answers in your own content

Grounding means the assistant reads your approved material at the moment it answers and builds its reply from that. The common technique is called retrieval-augmented generation, or RAG: the system finds the relevant passages in your documents and hands them to the model with the question. The method comes from a 2020 research paper, which reported that adding retrieval produced more factual language than a model working from its own memory alone (Lewis et al., Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks, arXiv).

For a small business this can be as simple as one clean document: prices, hours, policies, delivery, FAQs. Quality matters more than size. If two of your pages contradict each other, the assistant will too.

2. Tell it to use only that content

Anthropic's guidance lists "external knowledge restriction": instruct the model to use only the information in the documents provided, not its general knowledge. It also suggests asking the model to quote the supporting text, and to withdraw a claim if it cannot find a quote for it (Anthropic, Reduce hallucinations). This is written into the assistant's instructions. Examples of wording are in system prompt examples.

3. Give it permission to say "I don't know"

The same Anthropic page lists this first among its basic strategies: explicitly allow the model to admit uncertainty. A good fallback is short and useful: "I don't have that information. I can pass you to the team, or you can write to us at this address."

4. Narrow the scope

An assistant that only handles booking and opening hours has far fewer ways to go wrong than one that "can answer anything". List the topics it covers. For everything else, it should decline politely and offer a person. Keeping the model's randomness setting low also helps with consistency, but it does not make wrong answers right.

5. Add a handover to a person

Some questions should never be answered by the assistant alone: refunds, complaints, contracts, anything about health, safety or law. Set the rule in advance, and make the handover work: the conversation reaches a real person, and the customer knows when to expect a reply. If you are deciding which tasks to give to AI at all, when to automate a process covers the question of what to keep manual.

What does not stop AI hallucinations?

Some popular fixes do little on their own.

• Writing "do not hallucinate" in the instructions. The model has no way to know when it is doing it. A concrete rule such as "answer only from the provided text" works better than a general warning.
• A longer prompt with more rules. Past a point, extra instructions conflict with each other. Short, specific rules plus good source material beat long ones.
• Trusting a confident tone. Confidence says nothing about accuracy.
• Testing once. Answers vary between runs. Anthropic lists comparing multiple runs of the same prompt as a check: inconsistent outputs can signal a hallucination.

How do you check whether your assistant is hallucinating?

You check by asking questions with known answers, and questions with no answer, then comparing every reply with your written source. A small test set catches most of the damage before customers do.

1. Collect 50 to 100 real customer questions.
2. Write the correct answer for each from your own documents.
3. Add traps: things not in your material, wrong assumptions ("Is the 20% discount still on?"), and requests outside its scope.
4. Run each important question more than once.
5. Mark every answer right, partly right or wrong, and fix the source or the rules.
6. Re-run the full set after every change.

The full method, with a grading table and pass rules, is in chatbot testing: how to test a chatbot before launch. After launch, read a sample of real conversations every week until problems stop turning up.

When should a person stay in the loop?

A person should stay in the loop whenever a wrong answer costs more than the time saved. Anthropic's own note is blunt: these techniques significantly reduce hallucinations but do not eliminate them, so critical information should always be validated, especially for high-stakes decisions.

A simple rule: let the assistant answer questions that your documents answer plainly. Let it collect details and pass the rest on. Let a person approve prices, refunds, promises and anything legal.

Pre-launch checklist

• One approved source document for prices, hours, policies and FAQs, with no contradictions
• Instructions say to answer only from that document
• A written "I don't know" reply with a way to reach a person
• A list of topics the assistant covers, and a polite refusal for the rest
• Handover rules for refunds, complaints, contracts, health, safety and law
• A test set with at least 50 real questions, including traps
• Every price, policy and handover answer passes on repeated runs
• A named person who reads real conversations each week
• A way to update the source document when something changes

What to do next

Start with the source document. It is the cheapest fix and it improves every answer. Then write the limits, then test.

If you want help with the setup, Lobos Design is a two-person studio in Las Palmas de Gran Canaria that builds AI assistants for small businesses, working remotely with clients across Europe, the Gulf and the US. You can read how we approach it on the AI Integration service page.