Goodhart's Law
There’s a story about Soviet nail factories that may or may not be true. Central planners in Moscow set quotas by weight, so factories made fewer, heavier nails. Planners switched to quantity, so factories made tiny useless nails. The metric optimized itself into meaninglessness.
I find myself thinking about this whenever I see dashboards.
The formal version comes from Charles Goodhart, 1975: “Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes.” Marilyn Strathern said it better in 1997: “When a measure becomes a target, it ceases to be a good measure.”
Here’s why this happens. Before you target something, the measure correlates with what you care about. Schools with high test scores probably teach well. Hospitals with low mortality rates probably provide good care. The correlation is real.
But once you make the measure a target, you create incentives to optimize the measure directly. Schools start teaching to the test. Hospitals start refusing high-risk patients. The measure still looks good. The underlying thing gets worse.
The examples are everywhere once you start looking.
Code coverage was supposed to indicate code quality. So developers write tests that touch every line without testing behavior. Coverage rises. Quality doesn’t.
Hospital mortality rates were supposed to indicate care quality. So hospitals avoid risky patients who might die. Rates fall. Care quality falls with them.
Arrest numbers were supposed to indicate crime prevention. So police arrest for minor offenses to hit quotas. Arrests rise. Safety doesn’t.
I think the problem is that measurement creates a game. Once people know what you’re measuring, they play to the metric rather than the purpose behind it. And they’re often better at gaming metrics than you are at designing ungameable ones.
What’s the solution? Probably not to stop measuring. But maybe to use metrics as signals rather than goals. To watch multiple metrics that are hard to game simultaneously. To change metrics before they calcify.
I’m not sure there’s a clean answer. Measurement helps you see. Targeted measurement makes you blind.
Go Deeper
Books
- The Tyranny of Metrics by Jerry Z. Muller — Comprehensive treatment of metric fixation across institutions.
- Measuring and Managing Performance in Organizations by Robert Austin — How measurement shapes behavior in complex systems.
Essays
- Goodhart’s original 1975 paper is technical (about monetary policy). Strathern’s 1997 formulation is more accessible.
- Campbell’s Law (1979) covers similar ground from social science: “The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures.”
Related: cobra effect, legibility, systems