Dressed Like a Standard

You asked a simple question. What came back looks like a compliance filing.

There are codes in it. Little bracketed tags, numbered findings, a cross-reference or two. Some of it you follow and some of it you don't, and the part you don't follow has a specific texture: you can't tell whether AI-03 is something you're supposed to already know, something the model is telling you it looked up, or something it made up thirty seconds ago to keep a list tidy. So you skim. You take the document as read, because it looks like it has already been read, by somebody, at some point, in some process.

That skim is the whole problem, and it is not your fault. Here is the same information written twice — once the way a machine wants it, once the way you do.

Neither version is lazy. The first one is genuinely, measurably better for its reader, which is a machine that has the definitions loaded and needs stable keys to report against. The trouble starts when it is handed to you, and nothing about it signals which of those codes are real.

A costume is an argument. Dress something well enough and the room stops asking what it does for a living, because the clothes appear to have answered already. Codes work exactly that way. They put a sentence in the uniform of a standard, and then the uniform does the talking.

The finding was labeled SEC-01, and I read straight past it. Second row in a table of eleven, in a review an agent produced for me in about ninety seconds. Every row had a code, the codes were sorted, and two of them were cross-referenced against each other further down. Somewhere on the third pass I went looking for the register that issues SEC-01. There isn't one. No numbering authority, no catalog, no prior SEC-01 anywhere in the project, and no chance the same finding gets the same code if I ask again tomorrow. The finding itself was correct and worth fixing. The code was a costume, sewn ninety seconds earlier, and I let it do consulting-grade credibility work on a hunch.

Every coded identifier in a document is one of two things. Cited means it came from somewhere: a linter rule, a compiler diagnostic, an entry in a public weakness catalog, a clause in a contract, a standard your industry actually ratified. Cited labels are verifiable — you can grep them, look them up, or re-run the tool that emitted them, and the answer is the same next week. Coined means the model made it up, just now, to organize its own output.

Both render in the same monospace. That is the entire problem, and it is worse than a cosmetic one, because credibility flows downhill: a coined SEC-01 sitting in a list two rows above a real CWE-79 doesn't just fail to be verifiable — it quietly borrows the verifiability of its neighbor. The reader has no way to sort them, so the reader does one of two things. Either they take the invented namespace seriously, and start referring to SEC-01 in meetings as though it were a thing the industry had agreed on; or they notice the fakes and discount the whole column, which means the real CWE-79 stops getting looked up too. Bad identifiers drive out good, and the second failure is the expensive one, because it degrades the tools that were working.

Which brings up the fair question, and the one that keeps this post from being a scold: if the habit is that costly, why is it so persistent? It turns out there are four answers, and the first one is genuinely good.

1. Reference stability. This is the strong one. For a language model the context window is working memory, and writing a label into the transcript is a write to storage — so the retrieval properties depend entirely on the key you choose. Descriptions drift. “The session-expiry issue” becomes “the token problem” becomes “that auth thing,” and three phrasings of one entity are, mechanically, three weakly-linked regions of text that attention has to do semantic work to unify. Sometimes it fails. That is how one concept silently becomes two, or two collapse into one, twenty turns later, in a way nobody catches. A rare token sequence doesn't drift: it is near-unique in the window, exactly reproducible, and seeing it again reliably retrieves what followed it the first time. It is variable binding, and natural language is genuinely bad at it — pronouns and definite descriptions are ambiguous by design, which is a feature in poetry and a defect in a forty-file migration. This is why the urge intensifies exactly where you would predict: long sessions, multi-file changes, work split across boundaries, and handoffs to a subagent that will never see the conversation that produced the finding.

2. Making a set countable. Eight findings in prose have no checksum. Eight numbered findings do — you can verify that 1 through 8 were each addressed, which converts “did I cover everything?” from fuzzy recall into discrete comparison. This cuts dropped requirements, and it is the same mechanism that makes a spec with acceptance criteria beat a spec with a paragraph of intent. It is also, notably, satisfied entirely by plain numbering. Nobody ever lost track of a requirement because it was called “issue 4” instead of REQ-004.

3. Machine handoff. If a script, a CI job, or another agent consumes the output, a code is an actual API — greppable, exit-codeable, usable as a key in a table. This is real value, and it is the case that justifies most of the identifiers in a well-run pipeline. It is also conditional in a way that gets skipped: it counts only when something actually consumes it. The overwhelming majority of codes in the overwhelming majority of documents are consumed by nobody, ever.

4. The suspicious one. The most precise text in the training corpus — compiler output, RFCs, vulnerability writeups, regulated-industry filings, standards documents — is dense with coded identifiers. Producing that notation plausibly steers generation toward the region of the distribution where careful reasoning lives. Maybe. Style priming is a real effect; so is style mimicry with nothing behind it, and a model cannot distinguish those two from the inside. Treat this one as unproven and faintly self-flattering, which is roughly how I'd ask you to treat any explanation a system offers for its own behavior — including every word of this section, which is reasoning from architecture and observed output, not introspective fact.

Everything above is easy to nod along with and much harder to do at 4:40 on a Thursday. So here is the test. Seven of these were issued by a real body and five were invented to fill a slot, and they have been shuffled so that ugliness, obscurity, digit count — even the split itself — are all useless as tells. In our informal trials, people land near a coin flip — which is the finding.

The obvious cost is the one already named: a label the reader cannot verify, wearing the uniform of one they could. But that undersells what it does to the document as a whole.

The artifact gets poisoned — and poisoned is the right word rather than wrong, because there may be nothing wrong with it. Every finding might be correct. What gets destroyed is your ability to spend attention proportionally. You now have two options and both are bad: audit every line, which costs more than doing the work yourself and defeats the point of having asked; or accept the whole document, which is precisely what the formatting was shaped to obtain. Almost everyone takes the second one, feels faintly uneasy, and quietly never opens the file again. That unease is perfectly calibrated and completely unactionable, which is the worst pair of properties a work product can have.

And there is a second cost running the other direction — into the model rather than out of it — which is the one that actually produces bad work.

Once something is named, the name can be manipulated instead of the thing. A label is a handle: it lets you pick an idea up and move it around without reopening its contents. That is precisely what makes it useful, and precisely what makes it dangerous, because compression is lossy and the model does not reliably notice what got dropped. Consider the sentence “applied the P2 fix to all 14 sites.” Fourteen call sites are never identical. They have different surrounding state, different error handling, one of them is in a hot path and one of them is in a test helper that doesn't matter. P2 made them look identical, and the looking stopped there. The compression hid exactly the variation it was hired to summarize.

That is the mechanism behind confident, well-organized nonsense — and note that the organization is the symptom, not the reassurance. A tidy taxonomy over unexamined contents is more dangerous than a messy paragraph, because the tidiness is what buys it a pass. Four cases, none of which are stories about a model being stupid:

The fake cases. In 2023, in Mata v. Avianca, a lawyer filed a federal brief built on case citations produced by ChatGPT. The lead citation — Varghese v. China Southern Airlines — had a docket number, a reporter citation, a court, a year, and quoted holdings. It did not exist. The detail worth sitting with isn't the sanction; it's that opposing counsel and the court both spent real professional hours trying to find the cases before concluding they were fiction. The format bought the fabrications hours of expert attention. By 2025 a public tracker of court decisions involving AI-hallucinated citations was logging hundreds of entries across multiple countries — the failure did not stay a curiosity.

The packages that aren't there. An import statement is an identifier that looks cited by construction: it names a thing that either exists in a registry or doesn't. Researchers presenting at USENIX Security in 2025 generated hundreds of thousands of code samples and found roughly a fifth of recommended package imports pointed at packages that did not exist, with open-weight models hallucinating worse than commercial ones. The part that turns this from an error rate into an attack surface: the invented names repeat. Register the hallucinated name and wait — a practice now called slopsquatting, named by the Python Software Foundation's security developer-in-residence. This is the whole thesis of this post with a supply-chain compromise attached to the end of it.

The bug reports. Daniel Stenberg, who maintains curl, has spent two years describing a specific new burden: security reports that are well-formatted, confident, correctly structured, complete with plausible severity ratings — for vulnerabilities that do not exist. His complaint is precise and it is not “these are wrong.” It is that they are expensive to disprove, because they are dressed correctly, and a maintainer cannot tell from the outside without doing the work. By 2025 the project was requiring submitters to disclose AI use. This is the slop problem in its most concentrated form: the cost lands entirely on the reader, and the better the formatting, the higher the bill.

And the quiet one — the correctly cited wrong answer. A real code can be real and still be the wrong diagnosis. CWE-79 cited accurately, in the correct format, linked to the correct catalog entry, attached to something that isn't cross-site scripting, is a well-dressed error — and it is harder to catch than the invented ones, because every verification step you would run on it passes. Precision of reference is not precision of judgment. Nothing in this post gets you the second one, and anything that makes you feel like it did is the problem wearing a better suit.

None of this is an AI problem. AI industrialized it. Walk into any organization and count the identifiers that nobody can source: the Tier 1 initiative. The P2 bug. The Sev2 incident. The Phase 3 rollout. Ask three people in the same meeting what P2 means and you will get a severity, a deadline, and a shrug — three answers, one label, and a shared confidence that it means something specific somewhere.

Sometimes it does. That's what makes these the hard cases rather than the funny ones. A priority code is real currency inside a building that wrote the definition down — a support contract with named response times, an incident policy with a paging rule attached. It is counterfeit the instant it crosses the boundary of the group that defined it, and the crossing is invisible, because the label doesn't change. There is exactly one question that sorts it, and it costs four words: where is that defined? A cited code answers with a file, a tool, a URL, or a contract clause. A coined one answers with a pause.

We've told the other half of this story already. In Black and White, the failure was vagueness — take ownership of cross-team alignment — and the mechanism was a transfer: the asker keeps optionality, the doer absorbs the blame for whatever it turned out to mean. A coined identifier is the same transfer wearing considerably better clothes. Vague language looks like nothing is there, which at least gives you a chance to notice. A code looks like the opposite of vague. It is the most precise-looking thing on the page, and it can be pointing at nothing at all — which is a cargo cult in miniature: the full ceremonial form of rigor, none of the mechanism, and a control tower made of bamboo. Both posts are about the same tax. This one is just harder to see, because it is collected in a monospace font.

Rule five is the one with teeth, so state it plainly: never let a coined label reach an external deliverable. Inside a working conversation an invented label is scaffolding, and scaffolding is fine — nobody minds scaffolding on a building under construction. In a client report, VAL-02 reads as a citation to a validation framework the reader can go look up. There isn't one. Drop the code, keep the finding; the finding was never the part doing the lying.

And four things to ask, out loud, of any system or person handing you a coded document. They are all cheap and the first one pays for itself in a single sitting.

All of which collapses into one test, and the test has to be applied at the moment of coining rather than in hindsight: an identifier earns its cost only if it will be referenced again — by the model, by a tool, or by a human later. If you cannot name who reads it a second time, use plain words.

And because none of that helps you at the moment you are actually stuck — staring at a response you can't read, wondering if you're the problem — here are the two sentences that fix it. The first repairs the answer in front of you. The second stops it happening again.

You probably remember the headline. In 2017 Facebook's AI research lab trained two agents to negotiate over splitting a pile of objects, and the bots started talking to each other in something that wasn't English — clipped, repetitive, unreadable. Facebook panics, pulls the plug on AI that invented its own secret language.

Almost none of that is true. The agents were rewarded for negotiating successfully and were not rewarded for staying in English, so they drifted toward whatever was cheapest that still worked. Researchers changed the parameters — not out of fear, but because the entire goal was agents that could negotiate with people, and a private dialect made them useless for the job. Nothing was unplugged. Nothing was dangerous. An objective function had a known failure mode and got corrected, which is what research looks like on a Tuesday. CNBC and Snopes had it straight within days, and the researchers themselves called the coverage clickbaity and irresponsible.

And the correction went nowhere. That is the part worth sitting with: the debunk exists, it is easy to find, it was published immediately by outlets with real reach — and the version living in most people's heads is still the one with the panicking engineers. A well-dressed story outran its own retraction by about a decade. Which is the same physics as the rest of this post, pointed at journalism instead of findings tables.

Here is the twist, though, and it's the reason the story is worth telling accurately rather than just dismissing: the panic was wrong about the facts and accidentally right about the direction. Strip away the Skynet framing and the mechanism underneath was real, mundane, and permanent — when you optimize for an outcome and don't pay for legibility, legibility is what you stop getting. Nobody bought clarity, so nobody got clarity.

That mechanism now has a research agenda attached to it. The field calls it chain-of-thought monitorability, and a 2025 position paper signed across multiple frontier labs argues that the readability of a model's reasoning is a genuine safety property and a fragile one — something you have to deliberately choose to preserve, because nothing in the training makes it free. The early evidence is not reassuring: reasoning models trained heavily on outcome-based reinforcement already produce chains of thought that mix in nonsensical fragments, and within a given training regime the larger models tend to produce the less legible reasoning. There is a term now for reasoning that never becomes words at all, staying in continuous vector space where no human can read it — neuralese — and it is an active architectural direction, not a nightmare.

Which puts SEC-01 in perspective, in both directions. It is the domesticated cousin of that problem: the same drift toward a representation that is cheap for the producer and opaque to the reader, with legibility as the thing quietly traded away. Trivial stakes, visible channel, nothing dangerous, no research agenda required. But the same physics — and at this end, unlike that one, the fix is free. You just have to say where the label came from.

Here is the concession, and it isn't a small one, because the legitimate case is genuinely common. A migration touches forty files. The same six patterns recur throughout — a bare catch that swallows the error, a synchronous filesystem call inside a request handler, four more. Name them once, in a block, at the top: P1 through P6, here is what each one is. Then spend the next two hours writing sentences like “files 12, 19, and 27 are all P2, same fix as file 3” — and every one of those labels is doing real retrieval work, forty times over, in a context window that would otherwise be reconstructing the same description from scratch and getting it slightly different each time. That is the shape of a chunk spec in a director-coder-QA loop, and the identifiers are load-bearing in it. The difference between that and SEC-01 is not the format. It's that something reads it twice.

Which means the fix was never “use fewer labels.” It is one sentence long, and it costs nothing: name the kind. “ESLint flags this as no-unused-vars” tells you where to look. “I'm going to call these A, B, and C for the rest of this task” tells you not to. Either sentence disarms the entire problem — the false authority, the phantom taxonomy, the reader's slow slide into distrusting the real codes too — and neither one requires anybody to write less, think less, or organize less. It just requires saying where the thing came from, which is the same discipline that separates a post-mortem from a receipt.

A costume can be honest about being a costume. Everybody in the room knows the suit is the argument, and nobody is worse off for it. A findings table doesn't get that luxury — it arrives as a record, not a performance, and is read that way. So when something turns up dressed like a standard, ask what it does for a living.

This post is the mirror of Black and White, which covered the same tax collected on the way in — vague prompts, vague output. See also Avoiding AI Slop on who pays for well-formatted nothing, Don't Blame the AI on where the fault usually sits, and Out of Spec on what a reporting layer quietly drops.