The AI Journal

Two problems that look unrelated dissolve into one when you describe them abstractly enough. A literature scholar asks who wrote an anonymous eighteenth-century pamphlet. A threat-intelligence analyst asks which state sponsored last month’s intrusion into a ministry’s network. One works from fifty-six essays bound in a volume; the other from a packed binary and a set of command-and-control domains. Nothing about the materials overlaps. The inference does. Both are asked to assign an unseen agent to an observed trace, from a finite pool of candidates, on the assumption that the trace was produced honestly — that whoever made it was not trying to make it look like somebody else.

That assumption is doing more work than either field has generally admitted, and the history of stylometry shows what happens once it is tested.

Frederick Mosteller and David Wallace took on the authorship of the disputed Federalist essays in the early 1960s, assigning twelve papers that both Alexander Hamilton and James Madison had claimed. Their method was not literary judgement but counting: the relative frequency of function words such as “upon,” “while” and “whilst,” words too habitual to be chosen for effect, compared across essays both men were known to have written alone. The disputed essays matched Madison’s habits, overwhelmingly, under the Bayesian analysis Mosteller and Wallace ran against them (Mosteller & Wallace, 1963). The appeal of the method was exactly that the signal was assumed unconscious. Nobody writes “whilst” to fool a future statistician.

Patrick Juola’s later survey of the stylometric field makes the premise explicit by extending it: features such as function-word frequencies and character-level patterns work as identifiers because they sit below the level an author edits, and the methods built on them report strong accuracy under closed-set conditions, where the true author is known to be one of the candidates and nobody in the set has tried to disguise anything (Juola, 2006). That caveat, “nobody has tried,” is the whole of the assumption I mean. It is reasonable for an eighteenth-century pamphleteer, who had no idea a computer would ever read his function words. It stopped being reasonable the moment stylometry was pointed at people who knew they were being read.

Michael Brennan, Sadia Afroz and Rachel Greenstadt tested exactly that case. They had subjects deliberately disguise their own writing and, separately, imitate someone else’s, then ran the disguised and imitated texts against the same classifiers that work so well on undisguised prose. Obfuscation dropped the classifiers’ accuracy to roughly chance. Imitation did something worse: it succeeded, in a meaningful share of attempts, in making one person’s writing identify as another’s, which is not merely evading detection but actively directing it to the wrong name (Brennan, Afroz & Greenstadt, 2012). The signature that authorship attribution relies on is not unfakeable. It only looks unfakeable when nobody in the sample has a reason to fake it.

Cyber-attack attribution runs the identical inference over a different kind of trace: not word frequencies but code reuse, infrastructure overlap, compile timestamps, working-hours patterns and the debris an intrusion leaves behind, matched against the known habits of named threat actors such as the groups publicly tracked as Fancy Bear or Lazarus. Thomas Rid and Ben Buchanan’s “Q model” formalises the comparison across technical, operational and strategic layers of evidence precisely because the technical layer alone, the one closest to a stylometric fingerprint, is the layer most easily disguised, and the two insist that attribution is better understood as an interpretive judgement assembled from converging signals than as a single forensic verdict (Rid & Buchanan, 2015). That insistence exists because the other half of the field had already demonstrated why it was necessary.

Brian Bartholomew and Juan Andrés Guerrero-Saade catalogued the practice directly: intrusions seeded with the wrong language strings, false code-comment artefacts, borrowed malware components, and fabricated timestamps, planted by attackers who understood exactly how analysts would read the debris and built the debris to be misread (Bartholomew & Guerrero-Saade, 2016). This is Brennan, Afroz and Greenstadt’s imitation attack, restaged with nation-states as the imitators and ministries of defence as the audience being told whodunit. A planted Korean string in a European power-grid intrusion is not a different kind of error from a forged “whilst” in a disputed pamphlet. It is the same move: produce the trace an investigator is trained to trust, and let the investigator’s own method supply the false conclusion.

Neither field discovered this on its own terms first. Stylometry had to be embarrassed by its own adversarial-robustness researchers before closed-set accuracy claims came with a caveat about disguise. Cyber-attribution had to be embarrassed by its own catalogue of false flags before Rid and Buchanan could get a hearing for the claim that attribution is a judgement rather than a readout. Both corrections arrived late, from inside the respective fields, and both corrections say the same thing in different vocabulary: a signature is evidence of who produced it only once you have ruled out why they might have wanted you to think it was someone else.

The honest objection to treating these as one problem is that the parallel flatters stylometry more than it earns. Cyber attribution, at its better-resourced end, triangulates across independent evidentiary layers that a literary or forensic authorship case rarely has on offer: human intelligence, signals intercepts, infrastructure seized in unrelated investigations, years of accumulated behavioural baseline on a small set of named actors. A disputed pamphlet or a messaging-app ransom note has none of that; it has the text and perhaps a handful of comparison samples, and nothing else to triangulate against. On this view the fields only resemble each other at the level of abstraction I started from, and resemblance at that level is cheap — most inference problems look alike from far enough away.

The objection narrows the claim rather than voiding it. What the richer evidentiary base buys cyber attribution is not immunity from the forger’s move; the false-flag catalogue proves attackers can still exploit the technical layer specifically because analysts have learned to lean on it. What it buys is a check elsewhere in the structure, a second and third layer that a lone stylometric signature does not have. That is a difference in how far each field’s defences against mimicry currently reach, not a difference in what they are defending against. A forensic authorship report built on one classifier and no comparison to a disguised or imitated control is making exactly the mistake Rid and Buchanan’s model was designed to stop cyber analysts from making, and it is making it with less excuse, because the adversarial-stylometry literature that would have warned it off has existed, published and cited, for over a decade.

Any field whose evidence is a trace left by an agent who knew they were being watched inherits the same prior question, and the question comes before the statistics, not after them: not “whose signature does this match,” but “did the person who made this have a reason to want it to match someone else’s.” Everything downstream of that question, the word counts, the compile timestamps, the whole apparatus of confident attribution, is only as reliable as the answer nobody thought to check.