Streetlight Effect

Organizations tend to look where evidence is easiest to collect, which means what gets measured can be determined more by visibility than by importance.

4 min read

What Is It?

The streetlight effect takes its name from an old joke: a man searches for his lost keys under a streetlight, not because he lost them there, but because that’s where the light is. As a description of observational bias, it names something researchers, investigators, and organizations do constantly and mostly without noticing, directing attention toward whatever is easiest to observe or measure, rather than toward wherever the actual answer is most likely to be. The bias isn’t usually a conscious choice to look in the wrong place. It’s what happens by default when some kinds of evidence are cheap to collect and others are expensive, invisible, or simply harder to define, and the cheap evidence gets treated as though it were representative just because it’s the evidence that’s actually available.

Why Does It Matter?

An organization’s dashboards, reports, and reviews are built from whatever’s easiest to instrument, ticket counts, response times, page views, revenue booked, not necessarily from whatever best represents the outcome leadership actually cares about. Over time, what’s easy to instrument occupies more and more of the organization’s attention, not necessarily because anyone believes it matters more, but because it keeps appearing in dashboards, meetings, and reviews while the harder-to-observe reality it was supposed to stand in for, actual customer trust, actual product quality, actual organizational health, remains absent from the conversation, since nobody built a light for it. This is a different problem from Goodhart’s Law and Surrogation, both of which describe what happens once a proxy becomes a target. The streetlight effect happens earlier, at the point of deciding what to look at in the first place, before any target has even been set. An organization can avoid every failure mode of targeting a metric and still be looking in entirely the wrong place, simply because that’s where its instruments happen to be pointed. What happens after that, the proxy getting mentally substituted for the thing it stands in for, or the proxy hardening into a target people manage to, belongs to Surrogation and Goodhart’s Law respectively.

What Changes Once You See It?

You start asking, before trusting a dashboard or a report, whether what it’s showing you is what matters or just what happened to be easy to collect. You start treating the absence of a metric as informative rather than neutral, since something that matters but isn’t measured doesn’t show up as a zero, it shows up as nothing at all, invisible rather than merely low. You also get more willing to invest in expensive, awkward measurement, direct customer interviews, spot audits, qualitative review, specifically for the things that matter most and are hardest to instrument cheaply, rather than only measuring what the existing tooling already makes convenient. You get less suspicious of evidence simply because it’s qualitative. A messy interview, a field observation, or an expert’s judgment can contain more information about the thing you actually care about than a precise metric measuring something adjacent to it.

Common Misunderstandings

It isn’t a claim that easy-to-measure metrics are worthless. Many important things are also genuinely easy to measure, the bias is specifically about substituting ease of measurement for importance when the two come apart.
It isn’t the same as Goodhart’s Law or Surrogation, which describe a proxy degrading once it becomes a target. The streetlight effect is about which things get looked at in the first place, independent of whether anyone has started managing to a number yet.
It doesn’t mean an organization should try to measure everything. That’s usually not feasible, the point is being honest about which important things currently have no light on them at all, rather than assuming the well-lit metrics are automatically the representative ones.
It isn’t limited to formal metrics and dashboards. It shows up in what gets discussed in meetings, what gets included in a status report, and what evidence gets brought to a decision, any situation where some kinds of information are simply more available than others.

Diagnostic Question

Are we looking here because this is where the answer actually is, or because this is where we happen to have a light?

Explore Further

Field Notes

None yet.

Related Field Guide

Origin

The streetlight parable is much older than the term, and has circulated in several forms. Philosopher Abraham Kaplan described the methodological problem as the “principle of the drunkard’s search” in The Conduct of Inquiry (1964): researchers risk looking where their methods make observation easiest rather than where the answer is most likely to be. The label “streetlight effect” later became a common name for the same observational bias, popularized in part by David H. Freedman’s writing on the limits of scientific and journalistic evidence-gathering in the 2000s and 2010s.