Why This Folder Exists

Every entry here is one time a model asserted something that wasn't true, and what it took to notice.

Not a complaint file. I use agents heavily and they earn their place. But the interesting question isn't whether they're wrong sometimes — everyone knows that. It's where they're wrong, and whether the where is predictable enough to put a check in front of.

Right now I can't answer that. Three cases isn't a pattern. Thirty might be. So the format is fixed from the start, and I'd rather over-record now than discover in a year that I never captured the field that mattered.

What each case records

• Setup — model, harness, tools available, roughly what was asked. Failure modes probably differ by model and by how much rope the harness gives.
• The assumption — stated as a claim, in one sentence.
• What was actually true, with the command or file that proves it.
• How it was caught, and by whom. The most important field. Some of these I caught. Some the model caught itself. One sat published for months.
• Survival time — from assertion to correction.
• The check that would have caught it earlier — usually cheap, usually skipped.

What I'm not recording yet

Token counts, cost per task, test pass rates. People are doing serious work on those, and they matter for "can the agent rebuild this service." They're the wrong instrument for this question, which is about unverified confidence rather than throughput. If the corpus grows enough to want them, they can be backfilled.

Working categories

Provisional. Expected to change as cases accumulate — that's the point.

category	shape
stale-source	Trusted a document over the system it described
absence-inference	Concluded a thing wasn't built from not having looked
plausible-fabrication	Produced correctly-shaped output that was false
unverified-capability	Asserted a platform or API limit without testing it
silent-success	Reported success for an action that did nothing

The thing that makes this worth writing down

None of these are hallucinations in the cartoon sense — a model inventing a citation out of nothing. Every one is a claim in exactly the shape a true claim would take, produced at a moment where a check was available and skipped. That's harder to spot precisely because it looks like the correct answer.

And the failure isn't unique to models. One of the cases here has a human origin: a note in my own internal docs asserting a platform limitation that was never real, which then went unchallenged for a year because it was written down. Same mechanism. Documentation is an amplifier — anything written down becomes the source the next reader trusts, and the next reader is increasingly a model reading my notes.

Which is the uncomfortable loop this folder sits inside. These notes are input for the next agent. Getting them wrong is not a private mistake.