How to check AI output before you trust it

Verification is a habit, not an event

The most dangerous property of a language model is not that it makes mistakes. Everyone and everything makes mistakes. It is that its mistakes arrive in the same confident, well-formed prose as its correct answers. A wrong date and a right date look identical on the page. This means the usual human cue for doubt, the sense that someone is unsure, is missing, and you have to replace it with a deliberate habit.

The habit is proportionate verification. Not every output needs checking to the same depth. A brainstorm of ten campaign names needs no verification at all; you will pick or discard them yourself. A rewritten paragraph needs a read-through for meaning drift. A summary of a document needs spot-checks against the source. A figure that will go into a board paper needs to be traced back to its origin or recomputed. The question to ask is not, could this be wrong, because it always could, but, what happens if it is wrong, and who finds out.

Source-checking and triangulation

When an output makes a claim of fact, the first question is: where did that come from? There are only three possibilities. It came from material you supplied, in which case you can check it against that material. It came from the model's training, in which case it is a recollection, and recollections of specifics such as figures, names, dates and quotations are exactly where models are least reliable. Or it came from nowhere, generated because it fitted the shape of a plausible answer.

The practical move is to ask the model to show its sources within the material you gave it. Quote the sentence in the document that supports each claim. If it cannot, the claim did not come from the document. The test is fast and surprisingly powerful, because it converts an unverifiable assertion into a checkable pointer. When a model produces a quotation, check that the quotation actually appears in the source. Fabricated quotations that sound exactly right are a known failure mode.

For claims that go beyond your material, use triangulation: seek at least one independent route to the same fact. Independent matters. Asking the same model the same question again is not independent; asking it to argue the opposite is somewhat independent; checking a primary source, doing the arithmetic yourself, or asking a person who would know are properly independent. The more consequential the claim, the more independent the second route should be.

Spotting confident nonsense

You cannot verify everything, so it helps to know where to look. Four textual features are correlated with unreliability.

Suspiciously specific detail in a context where the model had no source is the first. An exact percentage, a named study, a precise date or a direct quotation appearing in an answer to a general question should raise your attention immediately, because those are the details a model generates to complete a pattern rather than because it knows them.

Smooth, symmetrical structure is the second. Real evidence is lumpy. When an answer offers exactly three advantages and exactly three disadvantages, each the same length, it is often shaping the content to a template instead of reporting what is true. The same goes for lists that seem to fill a number instead of exhausting a topic.

Uniform confidence is the third. A knowledgeable person answering a mixed set of questions will be sure of some things and unsure of others, and will say so. An answer that is equally assured about everything, including the parts that would be hard for anyone to know, is showing you register, not knowledge. You can test this directly: ask which parts of the answer the model is least sure of, and why. The answer to that question is often more useful than the original.

Drift from your material is the fourth: facts, names or terms in the answer that are not in the document you supplied.

A red-flags checklist and how to use it

Practitioners keep a checklist like the one below to hand. A genuine list is clearer than prose for this, and it is short enough to run in under a minute on any output that matters.

• Does the output contain any figure, date, name or quotation? Trace each to a source you can see, or mark it unverified.
• Did I supply material? Does the output stay within it, and where it goes beyond, is that flagged?
• Is the structure suspiciously tidy for the messiness of the topic?
• Is the confidence uniform? Ask the model what it is least sure of.
• Does it agree with me a little too readily? Ask for the strongest counter-case.
• Would I put my name on this if a new colleague handed it to me? If not, what specifically would I check first?

The checklist is not a substitute for expertise. It is a way of pointing your expertise at the places most likely to hide a problem. Someone who knows the domain will catch an error in seconds once their attention is on the right sentence; the checklist gets their attention there.

Tomorrow, apply the checklist to the next three outputs you would otherwise have used as-is. Most practitioners find at least one item worth correcting, and, more usefully, they find a pattern in what needed correcting. That pattern becomes a standing instruction in their briefs, and the next month's outputs are better before they are even checked.