← Back to blog

Why AI Makes Things Up — and Six Ways to Catch It

Why AI Makes Things Up — and Six Ways to Catch It

Ask an AI about Hong Kong's Employment Ordinance — say, the rules on annual leave. You get a beautifully structured answer: section number, quoted text, even the year it took effect. Then you look up the section. It does not exist.

Ask again. The model apologizes, and confidently gives you a different section number. Also nonexistent.

It is not lying. Lying requires knowing the truth and hiding it — the model has no idea it is making things up. This is hallucination, and it is not a malfunction. It is the built-in personality of the technology. The first lesson in using AI well is not prompt engineering; it is knowing when the model can be trusted and when it will confidently invent.

It is not looking things up. It is finishing your sentence.

A large language model does one thing: it reads your question and predicts, word by word, the most plausible next word. Its single goal, from start to finish, is to make the sentence flow — coherent, natural, human-sounding.

Note the word: plausible. Not true.

Think of a lecturer who has read an enormous library but never brings a single book on stage. Whatever you ask, they answer from memory and instinct. Most of the time they have read enough to be fast and right. But where memory runs thin, they do not stop — they fill the gap with whatever sounds right, in exactly the same fluent, confident tone as when they are correct.

Hallucination is precisely this: the model fills gaps in truth with plausibility, and it cannot tell the two apart.

One more detail makes it worse. Training has long rewarded producing an answer over admitting ignorance. Recent models have become noticeably better at saying "I'm not sure" — but the tendency is managed, not cured. The harder you push for an answer, the more likely you get an invented one.

The six highest-risk question types

Hallucination is not random; it has well-marked danger zones. Treat any answer to these six question types as unverified by default:

High-risk typeExamples
Specific numbers and datesPrices, statistics, event dates
Citations and quotesSection numbers of laws, page numbers, famous quotes
Links and sourcesA URL can be fabricated end to end
Local and niche informationA shop's opening hours, government procedure details
Recent eventsThe model has a training cutoff; everything after it is unknown
Forced answers"You must give me an answer"

The pattern in one line: the more specific, the more obscure, the more recent — the higher the risk. Conversely, explaining concepts, drafting copy, and reorganizing material you supply are the safe zone.

Six defenses

1. Demand sources — then actually check them

List a source for every claim. If you are not certain, mark it "unverified".

A caveat: asking for sources does not make the sources real — models fabricate citations too. The real value of this move is that it shrinks your checking workload: whatever the model itself marks "unverified" is usually exactly where the hallucination lives.

2. Make "I don't know" an acceptable answer

If you are not sure, say "I don't know". Do not guess. "I don't know" is a valid answer.

One sentence, immediate effect. You are telling the improvising lecturer: today, leaving a blank is allowed. No need to perform.

3. Switch modes for high-risk questions

Most AI products now offer web search. For recent events, prices, or anything time-sensitive, turn search on first — let the model look it up instead of reciting from memory. Without search enabled, it knows nothing about today.

4. Feed the source text. Do not test its memory.

If you want an analysis of a contract or an ordinance, paste the actual text and then ask. A model reading material you provide is far more reliable than a model answering from training memory — the first is an open-book exam, the second is improvised theatre. Contracts, reports, terms and conditions: paste first, ask second.

5. Cross-examine with a second model

Put the same important question to two models from different companies. If the answers agree, they are probably right. If they diverge, you have found exactly what needs manual checking. Two lecturers who have never met rarely make the same mistake in the same place.

6. Ask the model to attack its own answer

After you get an answer, follow up:

Which sentence in your answer is most likely wrong, and why?

When a model reviews its own output, it often flags the lowest-confidence part — which is precisely where your checking should start.

Three common misconceptions

1. "It searches the web." With search off, it answers purely from training memory and knows nothing recent.

2. "It sounds confident, so it must be right." A hallucination sounds exactly like a correct answer. The straight face is the defining feature of the phenomenon, not a signal of reliability.

3. "If I push back, the truth comes out." Challenge a model and it frequently changes its answer — but that may be deference, not correction. The arbiter is always the original source, never the model's attitude.

Knowing its personality is knowing the boundaries

Hallucination is not a bug awaiting a patch; it is the nature of a technology built to predict the next word. Expecting it to disappear is unrealistic. Once you know the personality, you know what to delegate with confidence and which answers go through verification first. People who use AI well are not so much technical as well-calibrated.

Next time it hands you a section number with a straight face, smile — and open the original.

Further reading: if you already run longer tasks through AI tools, [Why Claude Code Gets Expensive — and the Three Commands That Fix It] covers a related discipline — managing context — which connects directly to Defense 4 above.

Information in this article is current as of 20 July 2026 and is provided for general reference only; it does not constitute advice of any kind. Third-party product features, pricing and policies are subject to their official announcements. Essevin service details are as shown on essevin.com and in the console.