Our fears about AI mirror something older and more dangerous
Image: Image: Adobe Stock
As writer Ted Chiang has observed, “Most fears about AI are best understood as fears about capitalism.” When we imagine rogue AI systems optimizing the world to death, we’re actually describing what many corporations already do. If an organization’s ethos is misaligned, any technology platform it creates will amplify that misalignment.
No matter your formal title, you have influence over this organization.
Yet if you cannot answer these questions with confidence, you are not in control. You are a passenger in a vehicle you think you’re driving.
But if nobody is consciously driving, where is the organization going? If the behavior of superorganisms were random, we would see as many evolving toward flourishing as toward corruption.
Instead, the vast majority wind up in the same place, with indistinguishable values.
I recently had a conversation with the CEO of a multibillion‑dollar company who’d come to see me for help with a baffling problem. He had made a bold new bet on the promise of AI to revolutionize his business. He’d personally overseen the development of a new AI product and tested it with customers. They loved it. Satisfied that he had created something amazing, he turned it over to the rest of his team to commercialize and scale.
Six months later, this new product was going nowhere. The sales team wasn’t selling it. Developers were loath to work on it. Marketing would not promote it.
When confronted, the executives would insist that the product was part of their quarterly priorities. Each of them was telling their teams to focus on it. Yet, somehow, line employees and middle managers were consistently avoiding the AI product. It was a pattern throughout the organization.
When the CEO personally intervened, he could easily get individual customers to adopt the new product. But he could not get his team to do the same, even after investing extensively in training, performance bonuses, and top‑down directives.
He couldn’t identify any one person or group actively blocking the AI initiative. He even replaced several key executives with new ones explicitly chosen for their enthusiasm for this new direction. And yet, within months, the uneasy status quo prevailed.
He shook his head. “You know what’s driving me crazy? I founded this company fifteen years ago. When I first proposed the core idea, everyone thought I was nuts. Investors, potential employees, even my family.
“Now that we’re successful, when I suggest we need to pivot, many of the same people, including my own leadership team, resist because ‘we can’t risk what we’ve built.’ ”
He threw up his hands. “They’re protecting something that exists only because I took the ‘crazy’ risk they all thought was impossible. Now they think I’m reckless and can’t seem to appreciate how many times they’ve benefited from my pushing through the impossible to get us to new success.”
He could feel the stress his insistence was putting on his employees, he told me, as if they were being torn between two authorities. But who, exactly, was this other authority? Like so many founders, he had assumed that being in charge meant the organization would reflect his values. How could it be that he had all the control on paper but so little in daily practice? And if he wasn’t in control, who was?
John Steinbeck saw this predicament clearly in The Grapes of Wrath. Tenant farmers are being evicted from their land during the Dust Bowl, but when they confront the men carrying out the evictions, they can’t get a clear answer as to who is making the decision to devastate them.
The men sent to do the dirty work answer to the bank manager, who answers to distant owners, and those owners answer to shareholders. Each of them has a role in the organization, but somehow none of them is in charge.
This leads to the farmers’ bewildered question: Who is the bank?
Steinbeck portrays it as a faceless system in which no individual can be held accountable, even though real people are losing their homes and livelihoods. He calls the bank “the monster. Men made it, but they can’t control it.” My friend the CEO, grappling with resistance he couldn’t locate, experienced the same bewilderment. If the person at the top is not in charge, who is?
The CEO I was counseling wasn’t fighting his own people on the AI initiative. He was wrestling with a different kind of living being—one that had developed its own character, its own preferences, and its own will. His personal ethos of innovation was at war with the organization’s emergent character, which craved safety, certainty, and predictability. The fact that he had created it mattered very little.
Life is defined not by flesh and blood but by specific properties. Living things maintain boundaries between themselves and their environment. They metabolize resources into energy.
They grow, adapt, and reproduce. They exhibit behaviors that emerge from their parts but can’t be predicted by studying those parts in isolation. Most importantly, they display a will to survive that shapes every action they take.
Organizations exhibit every one of these properties. They maintain legal and cultural boundaries that define who’s inside and who’s out. They metabolize capital, talent, and raw materials into products and services.
They grow through hiring, adapt to market pressures, and spawn subsidiaries or merge with others. A company’s “culture” emerges from thousands of daily interactions but can’t be found in any employee handbook or org chart. And when threatened, organizations exhibit a tenacity that can override the preferences of any individual within them—even their nominal leader.
Each one is a superorganism unto itself, operating according to its own rules of existence.
Now we can answer Steinbeck’s question, Who is the bank? An emergent intelligence, every bit as alive as you or me. A superorganism.
This is who the bank is.
When we tell horror stories about AI—systems that optimize ruthlessly toward a narrow goal, blind to human costs—we are, in many ways, describing the bank. Men made it, but they can’t control it. Our fears about AI are, as Chiang suggests, often fears that we will pour unprecedented technical power into institutions whose ethos is already misaligned with human flourishing.
If we want different outcomes, we cannot just ask what our machines are optimizing for. We have to ask the same question of our organizations.
Excerpted from Incorruptible by Eric Ries, published by Authors Equity. Copyright © 2026 by Eric Ries. Reprinted by permission of Authors Equity.
ABOUT THE AUTHOR
Eric Ries is the author of "The Lean Startup," founder of the Long-Term Stock Exchange and Answer.AI, and author of the forthcoming book "Incorruptible: Why Good Companies Go Bad... and How Great Companies Stay Great."
Original article appeared on FAST COMPANY US
