Why LLMs Hallucinate — and the Case for Computation-First AI

“A language model does not know things. It predicts what an answer would sound like. Most of the time that is the same as being right—and the danger lives entirely in the gap between most of the time and always.”

---

Hallucination Is Not a Bug. It Is the Mechanism.

We call it “hallucination” as if it were a malfunction—an otherwise-truthful system occasionally glitching. That framing is comforting and wrong. When a large language model invents a citation, a court case, a planetary position, or a statistic, it is not failing to do its job. It is doing exactly what it was built to do: produce the most probable continuation of your text.

Truth was never the objective. Plausibility was.

Hallucination (in AI) technology

When a language model generates fluent, confident content that is not grounded in any real source or computation. It is not lying—lying requires knowing the truth. The model simply produces the statistically likely shape of an answer, whether or not that shape corresponds to reality.

I have written before about what AI actually is—a probabilistic prediction engine, not a mind. Once you internalise that, hallucination stops being mysterious. A system optimised to sound right will sound right even when it is wrong, because sounding right is the only thing it was ever measuring.

Why Scale Will Not Save Us

The industry’s reflexive answer is more: more parameters, more data, more training. And scale genuinely helps—larger models hallucinate less often, because they have absorbed more of the world’s patterns.

But look closely at that sentence. Less often. Not never. Scale reduces the frequency of confident errors without touching the mechanism that produces them. You end up with a system that is wrong more rarely and, precisely for that reason, trusted more completely—which is arguably more dangerous, not less. The failures get rarer and better-disguised at the same time.

A model that is wrong one time in a thousand, in flawless prose, is more dangerous than one that is wrong one time in ten. We stop checking exactly when we can least afford to.

You cannot fine-tune your way out of a category error. If a thing estimates the shape of answers, it will keep estimating, no matter how good the estimate becomes. The problem is not the model’s size. It is being asked to do a job it structurally cannot do.

The Tell: LLMs Fail Wherever Truth Is Computable

Here is the pattern I want you to notice, because it points directly at the solution. Language models are least reliable exactly where reality is most precisely checkable:

• Arithmetic and math — they approximate answers that have exact values.
• Dates and timelines — they guess when a calculation would give certainty.
• Citations and quotes — they fabricate references that either exist or do not.
• Legal and medical specifics — they invent dosages, statutes, case names.
• Astronomical and astrological positions — they place planets where they were never located.

Every one of these is a domain where the answer is not a matter of opinion. The Moon was at an exact sidereal longitude at your birth. A statute says what it says. A drug has an actual dose. These are computations and lookups, not vibes—and asking a probability engine to perform them is asking the wrong instrument to play the wrong part.

Computation-First: A Different Architecture, Not a Bigger Model

If the failures cluster where truth is computable, the fix is not a smarter guesser. It is to stop guessing at computable things. I call the alternative computation-first AI, and the principle is almost embarrassingly simple:

Let a deterministic system establish the facts. Let the language model interpret them. Never the other way around.

The deterministic layer does the work that must be exactly right, every time—the arithmetic, the lookups, the calculations against real data. Its output is not creative; it is correct. The language model then does what it is genuinely good at: reading those established facts and rendering them into clear, useful, human language. The model never invents a number. It reads one.

This is not a hypothetical. It is the architecture I built Eternal Evals on, in a domain almost purpose-built to expose hallucination: Vedic astrology, where a chart is a precise astronomical computation that AI astrologers routinely fabricate. A deterministic engine computes the chart from real ephemeris data; the AI is only ever allowed to interpret what the engine produced. I tell that whole story in “The Machine That Refused to Guess.” Astrology was simply a clean laboratory. The principle is general.

Grounding, Tools, and the Return of “How Do You Know?”

The good news is that the whole field is, haltingly, discovering this. The techniques have unglamorous names—retrieval-augmented generation, tool use, function calling, the Model Context Protocol—but they are all the same move: give the model a source of truth outside itself, and make it reach for that source instead of its own probabilities.

When an AI calls a calculator, queries a database, or invokes a computation engine and then interprets the result, it is doing exactly what computation-first prescribes. The model stands on a foundation it is not permitted to fabricate. This is why I exposed the Eternal Evals engine as an API and an MCP connector for Claude, Cursor and ChatGPT—so that instead of an AI guessing at a chart, the AI reaches for a tool that computes it, and interprets only what comes back.

The future of trustworthy AI is not a model that knows everything. It is a model humble enough to know what it must look up, and disciplined enough to actually look it up.

There is something old in this. Indian epistemology obsessed for millennia over pramana—the valid means by which a claim can be known to be true. Perception, inference, testimony: each was scrutinised for how it could fail. “How do you know that this is true?” is not a new question. Machines have merely made it urgent again, at scale, in domains where the wrong answer has consequences.

The Real Frontier

We are pouring extraordinary resources into making language models bigger. Far less attention goes to the humbler and, I think, more important work: building the architecture around the model so that it never has to guess at what could have been computed. The frontier is not a model that hallucinates less. It is a system designed so that hallucination has nowhere to occur—because every checkable claim is anchored to something that actually checked it.

The distinction matters far beyond any single product. As we hand AI more real decisions—medical, legal, financial, personal—the gap between sounds certain and is right stops being an academic curiosity and becomes the whole game. Closing that gap is not a matter of scale. It is a matter of design, and of the discipline to keep computation and interpretation apart.

An AI that guesses will always, eventually, guess wrong—beautifully, and at the worst possible moment. The alternative was available the whole time. We just had to be willing to build the boring, honest layer underneath the fluent one.

Read On

This piece is the argument; the rest of the series is the working-out. I get concrete about the architecture itself in “Compute, Then Interpret”, lay out the practical toolbox of grounding, RAG and tools, argue that measurement beats demos in “Evals, Not Vibes”, tell the story of the domain I used to prove all this, and trace the far older tradition that sat with these questions long before we did. The origin story is The Machine That Refused to Guess.

---