Survivorship Bias

Studying only the successes that are still around to be studied produces a systematically distorted picture, because whatever caused the failures to disappear also removed them from the sample.

3 min read

What Is It?

The clearest illustration comes from statistician Abraham Wald’s work on aircraft vulnerability for the military in World War II. Engineers wanted to add armor to returning bombers, and proposed reinforcing the areas where returning planes showed the most bullet holes. Wald pointed out that this damage data was conditional on those planes having survived: the data said nothing about planes hit elsewhere, because planes hit in those other areas were disproportionately likely not to return at all. Areas showing little damage on survivors could therefore be especially important, not because they were untouched, but because they were fatal often enough that the aircraft hit there rarely made it back to be measured. The general pattern extends far beyond aircraft: whenever a selection process filters out failures before they can be observed, the resulting sample is conditional on having passed that filter, and mistaking traits common among survivors for the actual causes of survival is the error that follows from studying that sample alone.

Why Does It Matter?

Organizations study successful companies, successful products, successful hires, and successful strategies constantly, looking for the practices that produced the outcome. But the companies, products, and strategies that failed using the exact same practices are usually gone, acquired, shut down, quietly discontinued, and not part of the sample being studied. A management book built entirely on interviews with surviving companies can describe a practice as a hallmark of success even when the practice had zero real explanatory value, if it was just as common among the companies that failed and simply aren’t in the book because they didn’t survive long enough to be interesting to write about.

The same distortion shows up in ordinary internal decisions. Reviewing the track record of an approach only among the projects that are still running, only among the employees who are still with the company, only among the customers who are still paying, systematically hides whatever the approach did to the ones that didn’t make it. The missing cases matter specifically because they aren’t missing randomly, their disappearance can be caused by the very thing you’re trying to evaluate, which means the visible sample can make an approach look far more successful than it actually was across everyone who tried it.

What Changes Once You See It?

You start asking, before drawing a conclusion from a set of successes, what happened to the failures and whether they’re actually available to be studied at all.

You start treating “look at what all the successful examples have in common” with real skepticism as a research method, since shared traits among survivors can just as easily be irrelevant, or present in the failures too, as they can be the actual cause of success.

You also start designing data collection so failures remain visible, exit data, churn cohorts, abandoned-project records, lost-sale reviews, rather than trying to reconstruct the missing population after it’s already disappeared.

Common Misunderstandings

It isn’t a claim that studying successful examples is worthless. It’s a claim that shared traits among successes alone, without any comparison to failures that used similar approaches, aren’t enough to establish what caused the success.
It isn’t the same as selection bias in general, though it’s a specific and common form of it. Survivorship bias specifically involves a process that removes failures from view, not just any non-representative sample.
It doesn’t mean every trait shared by survivors is meaningless. Sometimes the shared trait really did cause the survival, the point is that you can’t tell the difference without also looking at the failures, not that survivor traits are automatically irrelevant.
It isn’t limited to dramatic failures like bankruptcy or shutdown. It applies just as much to quieter forms of disappearing from the sample, an employee who left, a customer who churned, a project that got quietly deprioritized and stopped being tracked.

Diagnostic Question

Are we looking at everyone who tried this, or only at the ones who are still around to be looked at?

Explore Further

Field Notes

None yet.

Related Field Guide

Origin

Abraham Wald, working for the Statistical Research Group during World War II, applied the reasoning to armor placement on bombers; the term “survivorship bias” came into broader use later as the pattern was recognized across statistics, finance, and business analysis.