As Usual, We Did This to Ourselves

Have you ever listened to someone explain something with absolute confidence, nodded along, and thought:

"This sounds like complete bullshit."

You know it, we've all been there. Something just doesn't add up.

A company celebrates record-breaking performance while quietly laying people off.
An AI model tops a benchmark under strangely specific conditions.
An organization announces its deep commitment to transparency, then carefully avoids every question anyone actually asked.

The slides are polished, and the more they explain, the less you believe them. Point out a contradiction and you get the classics:

"You misunderstood."

"That wasn't our intention."

"We take these concerns very seriously."

So it made me wonder: how did we end up in a world where managing the perception of reality pays better than dealing with reality? We used to be more honest than this. Right?

Were we, though?

It's tempting to imagine a golden age when a handshake was a contract. I'm not sure it existed. What surprised me is how long researchers have been describing the exact behaviors we complain about today.

Myth and ceremony, 1977

In 1977, sociologists John Meyer and Brian Rowan published a paper with a beautifully honest title: Institutionalized Organizations: Formal Structure as Myth and Ceremony.¹ Their observation: organizations adopt policies not only because they work, but because legitimate organizations are expected to have them. The structures earn trust even when they're disconnected from what actually happens. They called that gap decoupling.

An ethics policy without ethics. A feedback process without listening. A transparency report nobody can understand.

Having a process isn't the same as achieving its purpose.

The score becomes the goal

Around the same time, economist Charles Goodhart and social scientist Donald Campbell independently noticed what happens when you measure things.²³ Once an indicator decides rewards, people optimize the indicator, sometimes at the expense of the thing it was supposed to measure.

We want better education, so we measure test scores. We want better AI, so we build benchmarks. Now there are two ways to win: improve reality, or improve the number. Ideally they're the same thing. They don't have to be.

And recently? 🎵 Everybody was tokenmaxxing 🎵 Companies started leaderboards on how much people use AI, rather than how it improves productivity and quality. Usage climbed fast as lightning. In fact, it was a little bit frightening. Sorry, that song is too catchy. 🤣

We didn't eliminate subjectivity. We gave it a numerical interface.

Technically correct, the best kind of correct

You don't have to lie to mislead someone.

"Our new model outperforms competitors by 20%!" On which benchmark? At what cost? And what about the twenty evaluations that didn't make the press release?

Researchers call this paltering: using truthful statements to mislead. In 2017, Todd Rogers and colleagues found that palterers saw it as more acceptable than lying, while the people on the receiving end still felt deceived.⁴

A collection of carefully selected truths can tell a completely false story. And when challenged: "Technically, our statement was accurate." Congratulations, you won the argument about wording and skipped the argument about reality. 🏆

Nobody wants to be the difficult person

In 2000, Elizabeth Morrison and Frances Milliken described organizational silence: employees collectively keeping quiet about problems, because the organization has taught them that speaking up is unwise.⁵

You can have anonymous surveys, open-door policies, and a leadership team that loves feedback. If people believe honesty will cost them their careers, those tools will mostly collect compliments. The bad news stops traveling upward, and the people at the top grow more confident in a company that only exists in their dashboards.

This one is on leadership. If people in your organization can't speak their mind (professionally, of course) without worrying about the consequences, you might have a toxic culture. And the dashboard won't tell you.

So no, we weren't a society of saints. The myths, the gamed metrics, the clever truths and the polite silences were all there before the internet.

Clickbait is not for fish

Likes, shares, impressions. Once attention became a number with money attached, Goodhart's problem went global. Compare:

"The available evidence suggests several possible explanations, and more research is needed."

"EVERYTHING YOU KNOW ABOUT THIS IS WRONG."

I think we know which one wins the algorithm. 😉

It's not just a hunch. Likes on outraged posts teach people to post more outrage,⁶ and outrage gets misinformation shared, sometimes without anyone reading it.⁷

Here's an example: you see a clickbait headline and get angry. You lose a little more trust in "the media," because you know better than that. Then another post shows up that matches your views perfectly. Finally, someone who gets it. That's what I've been saying all along! You like it, share it, follow the account. Congratulations, you just got hooked. Now it's "your truth," which doesn't really make sense, but we say it anyway. And by sharing it, you feed the machine. Someone else now trusts a little less in what's true and what isn't.

It's a vicious cycle, and every lap feels like independent thinking. But feeling strongly about something is not evidence that it's true.

We hate cherry-picked statistics, until one supports what we already believe. We demand accountability from people we distrust, and find nuance for people we admire.

Now add AI that produces polished, authoritative-sounding explanations at almost no cost. The behaviors are old. The amplifier is new.

Hearing is not listening

There's an old American saying: money talks, bullshit walks. And it's true, it's out there walking around. You see it too, and you ignore it. I'm not saying we should all be heroes. But we could have a little less tolerance for BS, a little more tolerance for the people who question it, and ask questions ourselves instead of believing blindly.

Trust takes years to build and seconds to lose, but it pays off. If you lead people, you carry a bigger share of that responsibility, and it starts with listening.

Thank the colleague who says "I don't know yet." Thank the person who brings bad news early. Give credit to the leader who changes their mind when the evidence changes.

We built a world where being convincing can matter more than being correct. We can also build one where being honest pays.

As usual, we did this to ourselves. But we can undo it too. 🤝

John W. Meyer and Brian Rowan, "Institutionalized Organizations: Formal Structure as Myth and Ceremony", American Journal of Sociology, September 1977. ↩
Donald T. Campbell, "Assessing the Impact of Planned Social Change", Evaluation and Program Planning, 1979. Originally circulated as a 1976 paper. ↩
Mark Rodamar, "There Ought to Be a Law! Campbell versus Goodhart", Significance, November 2018. On how the two arrived at the idea independently. ↩
Todd Rogers, Richard Zeckhauser, Francesca Gino, Michael I. Norton and Maurice E. Schweitzer, "Artful Paltering: The Risks and Rewards of Using Truthful Statements to Mislead Others", Journal of Personality and Social Psychology, 2017. ↩
Elizabeth Wolfe Morrison and Frances J. Milliken, "Organizational Silence: A Barrier to Change and Development in a Pluralistic World", Academy of Management Review, October 2000. ↩
William J. Brady, Killian McLoughlin, Tuan N. Doan and Molly J. Crockett, "How Social Learning Amplifies Moral Outrage Expression in Online Social Networks", Science Advances, August 2021. ↩
Killian L. McLoughlin, William J. Brady et al., "Misinformation Exploits Outrage to Spread Online", Science, November 29, 2024. ↩