Where LLMs fail

Failures are predictable, not random

In the last lesson you saw that an LLM only predicts the next most-plausible token. That single mechanism explains its strengths and its three classic failure modes. Once you can name each one and explain why it happens, you can engineer around it. None of this means the models are bad — it means you treat them like any component with known limits.

Hallucination

A hallucination is a confident statement that is simply not true: a fake citation, an invented API method, a plausible-but-wrong fact. It happens because the model optimises for plausibility, not truth. If the most fluent continuation of your prompt is a made-up function name, the model will happily produce it — with the same calm tone it uses for correct answers.

The danger is the confidence. There is no built-in “I don’t know” unless the prompt and training encourage it. Mitigations:

• Ground the model in real source text (retrieval-augmented generation), and ask it to answer only from that text and cite it.
• Give it permission to say “I don’t know” explicitly in the system prompt.
• Verify anything load-bearing — links, numbers, names — with a second check.

Knowledge cutoff and recency

A model is trained on data up to a cutoff date. After that date it knows nothing — not last week’s release, not today’s prices, not the incident you are debugging right now. Ask about a recent event and it will either admit ignorance or, worse, hallucinate a plausible-sounding answer.

This is not a bug you can prompt away; the information is genuinely absent. The fix is to bring the data to the model: retrieve relevant documents and put them in the context window, or give the model a web search / fetch tool so it can pull current information itself. As a cloud engineer you already think this way — you wouldn’t expect a cached value to reflect a change that happened after it was cached.

Weak arithmetic and multi-step logic

LLMs are surprisingly shaky at exact arithmetic and long chains of reasoning. Multiply two large numbers and a raw model often produces something close but wrong. The reason is the same prediction loop: it pattern-matches digit tokens rather than executing an exact algorithm, and small errors compound across many steps.

The mitigations are pragmatic:

• Offload the calculation to a tool — a calculator or a code-execution sandbox — and have the model call it. Deterministic code gives an exact answer every time.
• Encourage step-by-step reasoning for logic problems so the model breaks a hard task into smaller, checkable pieces.
• Verify the result when correctness matters.

Bad:  "What is 48273 * 1956?"  -> model guesses a close-but-wrong number
Good: model calls a calculator tool -> exact result, every time

The unifying mindset

Notice the pattern across all three: the fix is rarely “prompt harder.” It is to add structure around the model — grounding for hallucination, fresh data for recency, tools for math and logic — and to treat every output as untrusted input that earns trust only after validation. This is exactly the security-and-reliability instinct you already have. Modules 2 and beyond build directly on these mitigations.

Your task

Write down a real question from your own work for each failure mode — one that would trigger a hallucination, one blocked by knowledge cutoff, and one requiring exact arithmetic — and note the single mitigation you would reach for in each case.