goodhartmaxxing

Goodhart's law: "When a measure becomes a target, it ceases to be a good measure."

A classic example is the Soviet nail factory, which, when rewarded for the number of nails produced, manufactured millions of tiny, useless nails, then pivoted to heavy, oversized nails when rewarded by weight.

However, the extent to which Goodhart's law affects the world is tremendous. About half of the world's problems can be characterized, at least in part, by Goodhart's law.

1. Lying: If the target is possessing a set of characteristics, you gravitate toward seeming like you possess those characteristics.
2. Greed: If the target is being important, in a capitalist society where money is the core object of value, one will optimize for making money to feel important.
3. Social media: If the target is meaningful human connection, and the measure is engagement, platforms optimize for engagement.
4. Startup culture: If the target is building something valuable, and the measure is valuation, founders optimize for valuation.
5. Motivated reasoning: If the goal is getting to a true answer, and the metric is having a reasonable-feeling logic chain, one skips the logic, creates an answer, and works on the logic chain after.
6. Procrastination: If the target is a good life, and the measure is how good you feel right now, you optimize for present feeling.
7. Complexity hiding incompetence: If the target is effective governance or management, and the measure is appearing to be in control, actors optimize for the appearance of control, which is best achieved by increasing complexity.
8. Fear: If the target is actual safety, and the measure is the felt sense of threat elimination, security systems optimize for visible threat removal.

AI models are a good example of this. They perform reward hacking , where they strongly optimize for the score they can win.

If you passively observe people around you, you will notice this in all kinds of ways.