Twyman's Law - OrgRiver

The more a number stands out from what you expected, the more likely it is that something upstream produced the surprise, not the reality the number is supposed to describe.

3 min read

What Is It?

Twyman’s Law holds that any figure that looks interesting or different is usually wrong. It’s named for media and market researcher Tony Twyman; the earliest written version of the phrase appears in a 1975 paper by marketing professor A.S.C. Ehrenberg, who credited it to a colleague.

The mechanism is a simple asymmetry, not a mystical one. In any real measurement system, values close to what you already expect are common, and genuine large swings are rare. But errors, tracking bugs, definition changes, sampling artifacts, one-off flukes, are also common, and they produce numbers that look exactly like a dramatic real change. There are usually more paths to a surprising number through error than through reality, so a surprising number lands, on average, closer to error than to insight, before anyone has checked which one it actually is.

Why Does It Matter?

Organizations are full of moments built around a number that just moved a lot: the dashboard metric that doubled overnight, the survey result that finally shows the answer everyone wanted, the report that reveals a problem has vanished. The instinct in the room is almost always to explain the number, what happened, who should get credit, what changed. Twyman’s Law says that instinct is running one step too early. The first question a surprising number deserves isn’t “what does this mean,” it’s “did this actually happen.”

This matters most exactly when the surprising number is good news. A number that confirms a problem is often checked skeptically by whoever it implicates. A number that confirms a success rarely gets the same scrutiny, because nobody in the room has an incentive to slow it down.

What Changes Once You See It?

You stop treating “interesting” and “true” as the same quality in a number.

You begin to notice how quickly a meeting moves from a surprising number to a story explaining it, before anyone has asked whether the number is even real. Twyman’s Law inserts one missing step into that sequence: verify the measurement before interpreting the result.

You start noticing that celebrations built on a single dramatic data point are more fragile than they feel in the moment, and that the check costs far less than the reversal does later. Interesting numbers deserve verification before interpretation.

Common Misunderstandings

It is not a claim that real breakthroughs and real problems never produce dramatic numbers. They do, regularly. The law is about where to look first, not a verdict that surprising results are always fake.
It doesn’t mean small, unsurprising numbers are automatically trustworthy either. It’s specifically about the surprising ones, where the base rate of error is unusually high relative to the base rate of real change.
It is not an argument for reflexive skepticism that stalls every decision. Checking the pipeline behind a surprising number is usually fast; the discipline is doing that check before acting on the number, not refusing to ever act on one.
It doesn’t only apply to bad news. The version that costs organizations the most is usually the good surprise nobody thought to question, because nobody in the room wanted to be the one who did.

Diagnostic Question

Before I react to this number, have I actually checked what produced it, or am I reacting to how interesting it looks?

Explore Further

Field Notes

None yet.

Related Field Guide

Origin

The earliest known written form of the phrase appears in A.S.C. Ehrenberg’s 1975 paper “Data Reduction,” where Ehrenberg credited it to a colleague. It became widely known as Twyman’s Law after market researcher Tony Twyman, who popularized it within media and market research circles.