If you'd like an essay-formatted version of this post to read or share, here's a link to it on pluralistic.net, my surveillance-free, ad-free, tracker-free blog:
https://pluralistic.net/2026/08/03/andor/#either
The greatest magic trick of them all is lying. The reason you can't figure out that coin vanish even after the conjurer performs it three times in a row is that they didn't do the same trick three times in a row! They did three different tricks: "Didn't catch it? Here, let me do it again!" is a lie:
There's times when it makes sense to treat two outcomes as the same, even if they were produced by very different means. As a reader, my enjoyment of your novel is the same whether it was dictated, typed on an Underwood Noiseless, keyed into a word processor, or scratched out with a fountain pen:
There's plenty of routes that arrive at the same place, and if the destination is all that matters to you, it's fine to ignore the journey. But often, those end-points have subtle differences that are only revealed when things go wrong. If all you care about is how things work, chances are good that you're in for an unpleasant surprise when things fail.
I recently found myself arguing with an interviewer about whether AI is, or could be, conscious. We weren't arguing about whether it might someday be possible to make an artificial consciousness – as a materialist, I'll happily stipulate to this. I think that everything we call "consciousness" is the result of a physical process occurring within our bodies (and possibly around them?), so I think it's perfectly reasonable to imagine that someday we might create another physical process that produces the same effect.
But that's not what the interviewer wanted to argue about. His point was that teaching more words to the word-guessing program would produce consciousness, an argument I always liken to "breeding horses to run faster and faster until one of them foals a locomotive." In support of this (outlandish) proposition, the interviewer performed a kind of cognitive coin-trick: "I can often predict what my wife is going to say, and so can a chatbot that's been trained on her words. Therefore, we're both doing the same conscious work – and therefore the chatbot will eventually be as conscious as I am."
"Predicting what you will say through an understanding based on a theory of your mind" and "predicting what you are going to say based on a statistical analysis of your utterances" might produce the same outputs, but they are not the same trick. You can tell by what happens when the trick fails.
My wife and I have been together for 23 years now, and there's plenty of times that we can finish each other's sentences – and so can the autocomplete on our phones. The autocomplete manages the trick by exploiting the fact that we often repeat ourselves. But we manage the trick by understanding each other (and by exploiting the fact of repetition).
When my wife says something surprising – because she is angry or delighted, sad or happy – I can make a reliable guess about what caused my prediction to misfire. Our "sentence completion" trick doesn't emerge from a rough, automatically generated mental table of the statistical likelihood that word A will follow word B. We also understand why those combinations appear in each other's speech and writing.
"Understanding" and "statistical extrapolation" can often lead to the same place, but when they don't, "understanding" provides a way forward, while "extrapolation" founders. Both work fine, but only one fails gracefully. The two tricks only appear the same, but they are fundamentally different.
AI's investor story – and the science fiction tales of AI's eventual capabilities that underpin that investor story – makes heavy use of this conjurer's trick, in which two different outcomes are equated to one another because they resemble each other.
This "ignore the journey, focus on the destination" idea is baked very deeply into the way we think about AI. Take the "Turing Test," a complicated and nuanced thought-experiment proposed in 1950. Over the ensuing 75 years, Turing's thought-experiment has been stripped down into a blunt metric: "Can a chatbot trick a human into thinking it is also human?"
"I mistook a chatbot for a human" and "I took a human for a human" arrive at near-identical places, but they are subtly and importantly different. The erroneous assumption that my phone's autocomplete is actually a person who understands me well enough to finish my sentences works fine, but the instant I turn to it for understanding, it will fail very badly. Autocomplete's predictions are always grounded in who you used to be, which means autocomplete knows very little about who you are now, and absolutely nothing about who you will become:
https://reallifemag.com/instant-recall/
The low-rez Turing Test that captured popular discourse is profoundly misleading. It's the unsound foundation of a worldview that renders you incapable of distinguishing your understanding of your spouse from their phone's autocomplete function. It's the self-serving rationale that leads you to declare yourself a proud stochastic parrot:
The AI bubble is (seemingly) full of contradictions, but – like those baffling coin-tricks – these contradictions often resolve themselves very neatly once you realize that the "contradiction" is actually just two things that appear to be one.
For example, some of the billionaires who put up the first several hundred million for AI are solipsists who just don't believe other people are entirely real and therefore find it easy to believe that AI can do their jobs. Other billionaires are cynics who think that bosses can be sold defective worker-replacing chatbots because they're credulous suckers for that pitch, the same way they believe that desperate young men are suckers for Joe Rogan's useless and/or dangerous supplements and peptides:
Billionaire AI true believers and billionaire AI cynics make for a powerful coalition. The roadblocks that might discourage the first group are easily hurdled by the second, and vice-versa. You don't have to believe AI works to believe it can be sold, and you don't have to be motivated by the sales opportunity to believe that AI is about to become god.
Almost every debate I get into about AI turns out to be an unjustified, unacknowledged conflation of two things that seem similar, but have profoundly different underlying characteristics. Take this argument: "Every time we extend rights to the nonhuman world – watersheds, endangered animals, ecosystems – the world gets better. Let's extend rights to AI – whether or not we think it's a 'person' and so reap those benefits."
This, too, is a coin trick. Extending rights to nature reliably makes the world better, but extending rights to constructs makes the world far worse (Exhibit A is corporate personhood) (obviously).
A few moments' thought reveals the difference. If we extend rights to a watershed, that might result in an AI data-center being killed. If we extend rights to AI, that might lead to sacrificing the watershed to cool the data-center:
Then there's AI and labor. The world is full of skilled workers who have found ways to use AI on the job that they insist have improved their work. It's also full of skilled workers who warn us that on-the-job AI is producing tech debt at unimaginable scale, seriously depreciating the quality of the tools we use today, and setting us up for painful reckonings in the future.
This (seeming) contradiction melts away once you realize that these workers only appear to be doing the same thing. The first group of workers, excited about their AI-assisted output, are "centaurs": people assisted by machines; workers who choose the time and manner of their AI adoption. The second group are "reverse centaurs": people recruited to serve as peripherals for machines, who direct their actions and workflow:
Note that this isn't the same thing as saying "A skilled worker who adopts a tool willingly is always right and will produce a better output as a result." Nor is it saying, "The tool is so flawed that workers who claim it works for them must be deluded."
That's another coin trick! The reality – again – is that this is two things: some workers whose AI-assisted work is measurably worse are wrong about AI making their work better (centaurs, but wrong), and; some workers are being forced to use AI and know damned well that it's making their work worse (reverse centaurs).
Finally, there's an economic coin-trick: "AI will destroy jobs." Sure, yes, AI is destroying jobs. But there's a vast difference between "You got fired because an AI can do your job" and "You got fired because your boss was convinced that the AI can do your job, even though it cannot."
This is one of the most consequential coin-tricks, because it's a real convincer for the investors who are funding the AI bubble. The difference is huge: "AI can do your job" means you're well and truly screwed. If an AI can really replace a contract lawyer, then everyone who needs a contract written or evaluated should be on the side of mass technological unemployment for contract lawyers. The point of contract lawyers is to produce contracts, not to pay contract lawyers' law-school debts and mortgages.
BUT! If some BigLaw's credulous partners can be suckered into firing their juniors and replacing them with chatbots who bill you $1,200/hour to produce unenforceable, error-riddled contracts, then everyone who needs a contract is on the same side as the contract lawyers – united in opposition to their bosses:
Every time we fail to draw this distinction, we help an AI boss raise another billion dollars. Every time we insist on this distinction, we hasten the day that the AI bubble pops, thus sparing a few more everyday savers and innocent bystanders from being wiped out in the crash we can all see on the horizon:
As "Cathy" so aptly put it: "The thing that is a good tool for the skilled people is being sold as a thing to reduce the number of skilled people hired":
The former is a normal technology. The latter is the root of a catastrophic folly that is destroying our environment, destroying workers' lives, destroying the quality of the goods and services we rely on, and which will shortly destroy our economy.
Object permanence: Collectible AOL CDs; Indefinite Gitmo; Circuit City rips DVDs; Real Names policies are abusive failures; Escher in a water drop; Maximally code-like bugs.
#20yrsago Gonzales: Gitmo prisoners can be held indefinitely https://www.dawn.com/news/204441/guantanamo-detainees-may-remain-indefinitely-us-attorney-general-s-warning
#20yrsago Circuit City offers DVD ripping service https://web.archive.org/web/20060811215644/https://consumerist.com/consumer/circuit-city/circuit-city-flouts-the-dmca-for-a-tenner-192049.php
#15yrsago Google Plus’s “Real Name” policy is abusive; Facebook is not a “Real Name” success story https://www.zephoria.org/thoughts/archives/2011/08/04/real-names.html
#15yrsago Photo: Escher painting refracted in a drop of falling water https://www.reddit.com/r/pics/comments/j5whr/water_drop_falling_in_front_of_an_mc_escher/
#15yrsago Getting digital copyright right: pay artists, but don’t break the Internet https://www.straight.com/article-415146/vancouver/interview-siggraph-2011-keynote-speaker-cory-doctorow-copyright-reform
#10yrsago After repeated budget cuts, Missouri’s underfunded Public Defender drafts the Governor to work for him https://web.archive.org/web/20160804061408/http://www.publicdefender.mo.gov/Newsfeed/Delegation_of_Representation.PDF
#10yrsago Spoofing GPS is surprisingly easy; detecting it is surprisingly hard https://spectrum.ieee.org/gps-spoofing
#10yrsago Decision to retain personally identifying information puts Australian census under threat https://web.archive.org/web/20160804124914/https://censusfail.com/
#10yrsago Residents of Silicon Valley homeless camp clear 48,000 Lbs of garbage from creek, ask for housing https://www.mercurynews.com/2016/08/03/san-jose-homeless-remove-24-tons-of-trash-from-coyote-creek/
#10yrsago Copyright Office to FCC: Hollywood should be able to killswitch your TV https://www.eff.org/deeplinks/2016/08/copyright-office-jumps-set-top-box-debate-says-hollywood-should-control-your-tv
#10yrsago Walking Tables: a strandbeest for your dining room https://www.youtube.com/watch?v=mBOdZ6nhDJg
#5yrsago Utilities governed like empires https://pluralistic.net/2021/08/04/eighty-sixed/#thank-you-come-again
#5yrsago Congress has allocated enough money to end the eviction crisis https://pluralistic.net/2021/08/04/eighty-sixed/#helicopter-not-found
#5yrsago Vaccine refusal and health insurance https://pluralistic.net/2021/08/04/eighty-sixed/#risk-management
#1yrago AI software assistants make the hardest kinds of bugs to spot https://pluralistic.net/2025/08/04/bad-vibe-coding/#maximally-codelike-bugs
If you'd like an essay-formatted version of this post to read or share, here's a link to it on pluralistic.net, my surveillance-free, ad-free, tracker-free blog:
https://pluralistic.net/2026/08/03/andor/#either
The greatest magic trick of them all is lying. The reason you can't figure out that coin vanish even after the conjurer performs it three times in a row is that they didn't do the same trick three times in a row! They did three different tricks: "Didn't catch it? Here, let me do it again!" is a lie:
There's times when it makes sense to treat two outcomes as the same, even if they were produced by very different means. As a reader, my enjoyment of your novel is the same whether it was dictated, typed on an Underwood Noiseless, keyed into a word processor, or scratched out with a fountain pen:
There's plenty of routes that arrive at the same place, and if the destination is all that matters to you, it's fine to ignore the journey. But often, those end-points have subtle differences that are only revealed when things go wrong. If all you care about is how things work, chances are good that you're in for an unpleasant surprise when things fail.
I recently found myself arguing with an interviewer about whether AI is, or could be, conscious. We weren't arguing about whether it might someday be possible to make an artificial consciousness – as a materialist, I'll happily stipulate to this. I think that everything we call "consciousness" is the result of a physical process occurring within our bodies (and possibly around them?), so I think it's perfectly reasonable to imagine that someday we might create another physical process that produces the same effect.
But that's not what the interviewer wanted to argue about. His point was that teaching more words to the word-guessing program would produce consciousness, an argument I always liken to "breeding horses to run faster and faster until one of them foals a locomotive." In support of this (outlandish) proposition, the interviewer performed a kind of cognitive coin-trick: "I can often predict what my wife is going to say, and so can a chatbot that's been trained on her words. Therefore, we're both doing the same conscious work – and therefore the chatbot will eventually be as conscious as I am."
"Predicting what you will say through an understanding based on a theory of your mind" and "predicting what you are going to say based on a statistical analysis of your utterances" might produce the same outputs, but they are not the same trick. You can tell by what happens when the trick fails.
My wife and I have been together for 23 years now, and there's plenty of times that we can finish each other's sentences – and so can the autocomplete on our phones. The autocomplete manages the trick by exploiting the fact that we often repeat ourselves. But we manage the trick by understanding each other (and by exploiting the fact of repetition).
When my wife says something surprising – because she is angry or delighted, sad or happy – I can make a reliable guess about what caused my prediction to misfire. Our "sentence completion" trick doesn't emerge from a rough, automatically generated mental table of the statistical likelihood that word A will follow word B. We also understand why those combinations appear in each other's speech and writing.
"Understanding" and "statistical extrapolation" can often lead to the same place, but when they don't, "understanding" provides a way forward, while "extrapolation" founders. Both work fine, but only one fails gracefully. The two tricks only appear the same, but they are fundamentally different.
AI's investor story – and the science fiction tales of AI's eventual capabilities that underpin that investor story – makes heavy use of this conjurer's trick, in which two different outcomes are equated to one another because they resemble each other.
This "ignore the journey, focus on the destination" idea is baked very deeply into the way we think about AI. Take the "Turing Test," a complicated and nuanced thought-experiment proposed in 1950. Over the ensuing 75 years, Turing's thought-experiment has been stripped down into a blunt metric: "Can a chatbot trick a human into thinking it is also human?"
"I mistook a chatbot for a human" and "I took a human for a human" arrive at near-identical places, but they are subtly and importantly different. The erroneous assumption that my phone's autocomplete is actually a person who understands me well enough to finish my sentences works fine, but the instant I turn to it for understanding, it will fail very badly. Autocomplete's predictions are always grounded in who you used to be, which means autocomplete knows very little about who you are now, and absolutely nothing about who you will become:
https://reallifemag.com/instant-recall/
The low-rez Turing Test that captured popular discourse is profoundly misleading. It's the unsound foundation of a worldview that renders you incapable of distinguishing your understanding of your spouse from their phone's autocomplete function. It's the self-serving rationale that leads you to declare yourself a proud stochastic parrot:
The AI bubble is (seemingly) full of contradictions, but – like those baffling coin-tricks – these contradictions often resolve themselves very neatly once you realize that the "contradiction" is actually just two things that appear to be one.
For example, some of the billionaires who put up the first several hundred million for AI are solipsists who just don't believe other people are entirely real and therefore find it easy to believe that AI can do their jobs. Other billionaires are cynics who think that bosses can be sold defective worker-replacing chatbots because they're credulous suckers for that pitch, the same way they believe that desperate young men are suckers for Joe Rogan's useless and/or dangerous supplements and peptides:
Billionaire AI true believers and billionaire AI cynics make for a powerful coalition. The roadblocks that might discourage the first group are easily hurdled by the second, and vice-versa. You don't have to believe AI works to believe it can be sold, and you don't have to be motivated by the sales opportunity to believe that AI is about to become god.
Almost every debate I get into about AI turns out to be an unjustified, unacknowledged conflation of two things that seem similar, but have profoundly different underlying characteristics. Take this argument: "Every time we extend rights to the nonhuman world – watersheds, endangered animals, ecosystems – the world gets better. Let's extend rights to AI – whether or not we think it's a 'person' and so reap those benefits."
This, too, is a coin trick. Extending rights to nature reliably makes the world better, but extending rights to constructs makes the world far worse (Exhibit A is corporate personhood) (obviously).
A few moments' thought reveals the difference. If we extend rights to a watershed, that might result in an AI data-center being killed. If we extend rights to AI, that might lead to sacrificing the watershed to cool the data-center:
Then there's AI and labor. The world is full of skilled workers who have found ways to use AI on the job that they insist have improved their work. It's also full of skilled workers who warn us that on-the-job AI is producing tech debt at unimaginable scale, seriously depreciating the quality of the tools we use today, and setting us up for painful reckonings in the future.
This (seeming) contradiction melts away once you realize that these workers only appear to be doing the same thing. The first group of workers, excited about their AI-assisted output, are "centaurs": people assisted by machines; workers who choose the time and manner of their AI adoption. The second group are "reverse centaurs": people recruited to serve as peripherals for machines, who direct their actions and workflow:
Note that this isn't the same thing as saying "A skilled worker who adopts a tool willingly is always right and will produce a better output as a result." Nor is it saying, "The tool is so flawed that workers who claim it works for them must be deluded."
That's another coin trick! The reality – again – is that this is two things: some workers whose AI-assisted work is measurably worse are wrong about AI making their work better (centaurs, but wrong), and; some workers are being forced to use AI and know damned well that it's making their work worse (reverse centaurs).
Finally, there's an economic coin-trick: "AI will destroy jobs." Sure, yes, AI is destroying jobs. But there's a vast difference between "You got fired because an AI can do your job" and "You got fired because your boss was convinced that the AI can do your job, even though it cannot."
This is one of the most consequential coin-tricks, because it's a real convincer for the investors who are funding the AI bubble. The difference is huge: "AI can do your job" means you're well and truly screwed. If an AI can really replace a contract lawyer, then everyone who needs a contract written or evaluated should be on the side of mass technological unemployment for contract lawyers. The point of contract lawyers is to produce contracts, not to pay contract lawyers' law-school debts and mortgages.
BUT! If some BigLaw's credulous partners can be suckered into firing their juniors and replacing them with chatbots who bill you $1,200/hour to produce unenforceable, error-riddled contracts, then everyone who needs a contract is on the same side as the contract lawyers – united in opposition to their bosses:
Every time we fail to draw this distinction, we help an AI boss raise another billion dollars. Every time we insist on this distinction, we hasten the day that the AI bubble pops, thus sparing a few more everyday savers and innocent bystanders from being wiped out in the crash we can all see on the horizon:
As "Cathy" so aptly put it: "The thing that is a good tool for the skilled people is being sold as a thing to reduce the number of skilled people hired":
The former is a normal technology. The latter is the root of a catastrophic folly that is destroying our environment, destroying workers' lives, destroying the quality of the goods and services we rely on, and which will shortly destroy our economy.
If you'd like an essay-formatted version of this post to read or share, here's a link to it on pluralistic.net, my surveillance-free, ad-free, tracker-free blog:
Neoclassical economics assumes rationality. The corollary of, "If you're so smart, why aren't you rich?" is "you're rich, so you must be very smart!" Thus it is that many people assume that if powerful, well-compensated CEOs insist that "AI is changing everything," well then, AI must be changing everything.
But the evidence for this "changing everything" thesis is thin on the ground. Despite a global mania that has reduced the real, pressing need for digital sovereignty to the imaginary need to create "sovereign AI," no one can really articulate the case for "sovereign AI." If Donald Trump ordered Big Tech to turn off all of your country's chatbots tomorrow, nothing would change. Every one of your country's ministries and corporations would chug on with nary a hitch. Households, too, though perhaps a few of the younger members of those families would have to do their own homework again.
(Contrast this with what would transpire if Trump directed his tech giants to switch off your country's Office 365 access, or to brick your Android and iOS phones, or to killswitch your John Deere tractors. Your country would effectively cease to exist. If "digital sovereignty" means anything, it means doing something about this urgent fact):
The world is full of people who insist that "AI is changing everything" but who – when pressed – have to admit that what they mean is that they're pretty sure that AI will change everything. Eventually. After we allow it to consume all the planet's energy, carbon, water and financial resources.
Maybe.
(They're pretty sure.)
One person who's had a lot of opportunity to observe the shear between the stated business/AI situation and the real business AI situation is Nikhil Suresh from Hermit Tech, a consulting firm of "radically ethical data wizards" (that is, tech consultants). Suresh reports on his experience talking with hundreds of executives (and, more importantly, their subordinates) about what (if anything) AI is doing for business in an essay entitled "AI Mania Is Eviscerating Global Decisionmaking":
Suresh has a good track record of writing trenchant, frank criticism of AI. You may know him from his 2024 essay, "I Will Fucking Piledrive You If You Mention AI Again":
Or possibly from his "Contra Ptacek's Terrible Article On AI," a stinging rebuttal to Thomas Ptacek's widely read "My AI Skeptic Friends Are All Nuts":
While those are important pieces of critical AI realpolitik, none of them have the heft or urgency of "AI Mania Is Eviscerating Global Decisionmaking," whose thesis can be summed up with this passage from halfway through this 6,000-word article:
[W]e’re facing a coordination problem around executives being honest around the AI gains they’ve witnessed – if they co-operate, they keep their jobs. If they defect, they will possibly be fired by their embarrassed peers (who have now been implicitly called liars, cowards, or incompetents) and then replaced with someone that will toe the line anyway. If they could all admit the truth at once there might be some hope, but there is no way to coordinate that event.
In other words, corporate leadership is starting from the premise that AI has (or will) radically change the business, and they're working backwards from that premise to find the evidence to support this article of faith.
In support of this thesis, Suresh cites "hundreds" of conversations with execs and employees who spoke to him on the condition that he would "file the serial numbers" off their stories. These, combined with his own experience consulting for large, multi-billion-dollar companies make it clear that "AI mania" is an absolutely justifiable label for the state of AI in corporate circles.
Here are a few highlights from this morning's read – moments where I had to look away from my screen and read out a passage to my wife so that we could share a "holy shit" moment.
A person worked for a division that "pivoted" to re-engineer its software to create interfaces that support AI agents. When it became apparent that only ten users had touched this expensive new technology, they "pivoted" again to support "agentic workflows." Why did they double down on AI agents after discovering such yawning market indifference for "agentic"? "Because every company has to do something agentic now."
Suresh describes this as a literal religious mania. In the 500+ employee businesses Suresh studied, the only people who were promoted – or even spared from being fired – were people who professed "religious declarations of faith" about "the transformative power of AI." Employees who voiced honest, informed objections to AI in the workplace were passed over for promotions or targeted for layoffs.
This has created a situation in which everyone – "boards, executives, employees, vendors, consultants" – has a strong incentive to lie about how much AI is delivering for their companies. Suresh says he's seen announcements from publicly traded companies about their AI triumphs that he knows for a fact never took place.
Suresh says he's never seen a successful enterprise AI project: "Every single one – we have seen 0% success in a year and a half." Not one of their clients would face a business challenge if OpenAI went out of business tomorrow. The problem most companies struggle with is that they're "terminally bad at running software projects effectively." Adding AI to the mix doesn't solve this problem – it just adds a whole new range of ways that software deployment can fail.
Chatbots don't help. The internally facing chatbot that's supposed to help employees figure out how to navigate the business sucks because it is only as good as its training data – the business's documentation of its own processes. Businesses suck at documenting their processes. Customer-facing chatbots also suck. They either can't solve your problem, or, when they seem to solve your problem, the "solution" goes nowhere.
Suresh recounts his sole positive customer service chatbot experience: a Mitsubishi chatbot with a natural sounding, responsive voice politely took all the details of an automotive failure and promised him a callback. That callback never came, but Suresh is certain that Mitsubishi has logged this as a chatbot success story, even though the experience convinced him not to buy a Mitsubishi car.
Suresh and his team at Hermit Tech now have a policy of not even asking about ongoing AI projects. They've learned that by the time an AI project has begun, no one will discuss it honestly until it reaches a crisis point.
Suresh says he frequently encounters people who reflexively utter the AI catechism: "AI is changing everything." But when he presses these people for details, they admit that their organization "does not currently use LLMs for anything, and indeed, that they cannot name a single thing that has changed other than they get some use out of ChatGPT."
This shear ("AI is changing everything"/"Well, OK, we're not using AI for anything") is so extreme that Suresh once met an exec who confessed to crafting an AI-centered AI strategy for a $2b/year business, even though that exec "had never even used ChatGPT or any AI tool in their life."
Some people have privately admitted to Suresh that they've embraced AI in order to earn a career-boosting corporate reputation for "thought leadership." But many other people (especially nontechnical people) sincerely believe that AI is about to "change everything." As Suresh says, if you're in business with a liar, you might be able to reason with them in private – but you can't reason with a true believer.
The true believers are in charge. Suresh points out that it would be very weird for the CEO of an engineering firm or a hospital to mandate "specific procedures or building techniques without explicit agreement from the professionals on staff." But when it comes to AI, business leaders will confidently demand that the skilled professionals who perform the business's core functions use AI, even if those professionals don't think it will help.
As an aside: I remember the dotcom era, when the business press was full of articles about the conflict between CEOs and a new workforce that demanded the right to use the web on the job. Today, the business press is full of articles about the conflict between the workforce and CEOs who demand that they use AI.
Suresh describes workers who feel they have to "AI wash" their work: "They just do the work, the same way they have for decades, and say Claude did it." To add verisimilitude to this sham, they write circular processes in which one chatbot prompts another, and then the process repeats itself in reverse, for the sole purpose of consuming AI tokens to score a high rank on corporate "token leaderboards."
How to account for this wildly, expensively irrational corporate leadership? Suresh places the blame in the hypnotizing, mesmerizing power of the AI demo. For example: Hermit Tech is often engaged to set up a database product called Snowflake for its customers. Snowflake has a useless, expensive AI bolt-on called Cortex, that Snowflake itself describes as being 92% accurate under ideal circumstances (that is, at least 8% of the time, it will mislead you, perhaps very badly).
Suresh describes sales meetings with execs who were lukewarm on the idea of retooling with Snowflake, but who were very interested in Cortex. Against their better judgment, Suresh and his team provided them with a Cortex demo, carefully explaining that this AI tool could not satisfy their requirements. Without fail, this resulted in the previously lukewarm customers insisting that they be allowed to purchase Cortex immediately. Sales prospects who'd been unmoved by a pitch for new technology that would result in millions in savings were hypnotized by demos of a product that was described as unsuitable and unreliable.
To their credit, Hermit Tech refused to sell these customers Cortex, and stopped doing Cortex demos altogether. Suresh describes the experience of "the total 180°, that shift from ice-cold to red-hot buying frenzy" as "deeply unsettling." What's more, the Cortex demos that Suresh and co performed were, by his account, pretty uninspiring. The thing that these demos had going for them is that they showed AI actually doing something marginally useful, to execs who'd already spent millions on AI without having anything to show for their money. The spectacle of AI that does something galvanizes corporate leaders who feel like they're the only bosses who can't find a revolutionary use for AI in their businesses.
This is the situation up and down the corporate org-chart. Suresh has a reader whose title is "Head of AI" at a billion-dollar firm who tells him "their job is totally fraudulent but it was the only promotion pathway remaining at the organisation." This exec is hardly alone. They're part of a cohort of executives at companies that have publicly announced "100x" productivity gains, but who confessed to Suresh that nothing of the sort has happened.
Why did these companies make these claims? Because their customers were making the claims. How could you hope to sell to a company that had 100x'ed its productivity with AI unless you, too had 100x'ed your productivity? If, as a vendor, you walked into a boardroom and said that this wasn't a plausible claim, you'd be calling your sales prospect a liar, with real consequences: "getting enterprise contracts cancelled because you wanted to opine on something that doesn’t really matter to your organisation’s mission is a great way to get fired."
With the state of the industry dominated by froth, lies and mutual destruction pacts, it's no wonder that companies are deploying "totally gameable metrics such as 'money spent on AI'" as a means of evaluating employees and divisions.
Between true believers and people who must find ways to plausibly tout their AI usage, there is now a gigantic market for "AI solutions." At best these are just traditional tech consulting contracts, like migrating a database from Oracle to Snowflake, with some kind of ornamental AI usage around the edges so that the person who commissions the work can claim to be "procuring AI-enabled services" for the business.
This isn't a harmless frippery: contracts are delayed and work is put off until the work can be made "sufficiently AI" to attain the minimum degree of buzzword compliance. Worse: every fake AI project that produces real results (because it's not really AI) adds credibility to the AI true believers, who view these projects as proof that AI can do anything, and therefore demand to know why everything isn't being done by AI.
Suresh ends his essay with a long section on how to "navigate AI mania" – advice for how to smile and nod politely when you're confronted with AI bullshit, while steering clear of the worst consequences and avoiding needless fights. This looks like very sound advice for anyone in a corporate environment, but thankfully, that isn't me.
Rather than summarize that advice, I want to reflect a little on two questions that Suresh's essay raises but doesn't answer. The first is why? Why are people in power such easy converts to this religious mania?
I have my own theory. The most important discomfort that powerful people experience is having ego-shattering conflicts with subordinates who know how to do things they do not know how to do. The fact that you're "in charge" is hard to reconcile with the fact that the people you're nominally in charge of tell you that all your ideas are impossible, illegal, immoral, or lethal:
Take that Cortex demo. Sure, Cortex is an expensive, unreliable way to address a Snowflake database. But (unlike Snowflake) Cortex is controlled via conversational, plain-language commands. With Cortex, a boss doesn't need to ask an underling to retrieve information from the company Snowflake system, an interaction that might come with unsolicited feedback about the technical or commercial incoherence of the boss's request. Cortex is the underling, except that unlike a human underling, Cortex never back-sasses you about your foolish questions. The fact that it grossly misleads you 8% of the time is a small price to pay for a life untroubled by uppity pismires who insist that your ideas be connected to base reality as they understand it.
The other question Suresh implicitly raises is, "How can you reconcile the failure of AI in the enterprise with the individual claims of skilled technologists who insist that AI is helping them do great work?" The answer is that these AI users are "centaurs" – experienced workers who are assisted by automation on terms that they set for themselves:
Thanks to their skill and experience, these workers possess discernment, the ability to tell good code from bad, and (more importantly) good uses of code-generation tools from bad. They demonstrate the adage that worker-driven automation improves quality, while capital-driven automation improves throughput:
An automation technique that requires close supervision by skilled and experienced workers isn't going to be a raw productivity powerhouse. You don't "100x" your code this way, at least, not in the sense of firing 99 of your coders and having the remaining programmer pick up all their work. Rather, an automation tool that requires the continuous and conscientious exercise of discernment will let individual practitioners improve their work in extremely satisfying and useful ways. It's a way to spend more on operations in order to produce better outputs. It's not a way to cut your workforce, realize a gigantic savings, and still produce comparable goods and services at a far lower cost.
That is why some individual coders report such delight with their AI tools. They engage with those tools on their own terms, to improve their work in the ways that they, in their expert judgment, consider beneficial. No one ranks them on a "token-maximization" scoreboard. No one tells them they can't do a project if it isn't "sufficiently AI." When they set out to do a project, no one makes them prove that it couldn't be "done by AI."
As ever, the most important fact about a given technology isn't "what it does," but "who it does it for" and "who it does it to."
All the pathologies Suresh observes and documents so well in this piece are hypertrophied versions of the buzzword-compliance dysfunctions from previous bubbles, but at a scale never before seen. Quantity has a quality all its own. These businesses aren't just wasting billions – they're replacing skilled workers with defective chatbots. As I've written before, AI is the asbestos we're shoveling into the walls of our technological society. Our descendants will spend generations digging it out again, and the longer the bubble goes on without popping, the longer it will take to repair the damage.
In other words, corporate leadership is starting from the premise that AI has (or will) radically change the business, and they're working backwards from that premise to find the evidence to support this article of faith.
I've seen this happening and it is terrifying. Being told that the organization you work for has made a huge investment in AI, and that your opportunities for advancement (or even getting to keep the job you have) are contingent on finding a way to use that AI....What are those workers realistically supposed to do? Stand up for accuracy and get fired? Point out the ecological and ethical impacts of increasing AI use and hope the executive who made the AI investment doesn't decide to save money by terminating the squeaky wheels? Try to unionize and get fired? They're too scared to do anything but comply.
And that enhanced tendency toward compliance is a big part of the appeal of AI, as mentioned above. AI doesn't talk back, which business leaders LOVE. So human workers are learning to also not talk back, because they can tell it's dangerous to their jobs to do so. No one will say that the emperor has no clothes when the AI is confidently describing the clothes (and being so complimentary about the emperor's taste in fabric!). If you can't see the clothes, then that's a defect in you; nothing else could possibly explain it.
The funniest thing about the old "emperor's new clothes" story is the part where no one in that story stopped to think about the logical conclusion according to the described properties of that magical fabric.
Some nonzero number of people were going to perceive the emperor as naked.
They were so worried about personally being thought simple, or unsuited for their own job, that no one stopped to think about the people who really were unsuited for their jobs. And anyone who lives in a society knows we've got a fair number of those.
By the description originally given, anyone who was incompetent at their job would get an eyeful of bare emperor, because the cloth would be rendered invisible. Who would choose to dress in something like that? Who would risk parading through the street, appearing to be naked to people who were bad at their jobs?
And we're all just staring in confusion and disbelief at these naked C-suiters, as they proudly slap each other on their (naked) backs, congratulating each other on their masterful implementation of AI.
It feels as if we're having to choose between having a roof over our heads versus using our principles to keep the rain off of us. I can't really blame front line workers for choosing to stay housed.
Without naming any names, about a year ago one of the mega companies behind one of the big AI things informed its marketing and advertising partners to stop asking for actual internal data for use cases to build commercials and campaigns around. Turns out their own internal research showed that inside their own company, no one used it, no one wanted to use it, and the people who were forced to use it were less productive and very upset about it. Legal wouldn't let them pretend otherwise publicly; too much documentation. All their spots are now aspirational and very vague.
One of the few aspects of this AI bullshit where it's clear the emperor has no clothes: lawyers insulating their clients against specific claims and liability over what the AI things can actually do.
Beginning in 1979, Games Workshop produced a number of box sets of dungeon tiles. The first one was D&D-branded and was basically a cardboard sheet of squares printed with a stone pattern in different colors that you could cut into corridors and rooms. Series 1, which ran across six boxes issued between 1982 and 1984, was basically the same, though new types of tiles were introduced, including water, roofing, boats, irregular caves. They are mostly cleanly designed and unexciting.
So is this one, Dungeon Floor Plans 1 (1986)! This is the start of Series 2, and is mostly the same as its predecessors. The exception is the inclusion of grunged up tiles and a sheet dedicated to unique features like rugs, pillars, corpses and such. That’s the stuff I’m looking for from Games Workshop! The previous boxes had some unique features (this box has part of a sheet from Dungeon Floor Plans 2, 1982), but they lack the personality of the later box, I think.
If you'd like an essay-formatted version of this post to read or share, here's a link to it on pluralistic.net, my surveillance-free, ad-free, tracker-free blog:
Neoclassical economics assumes rationality. The corollary of, "If you're so smart, why aren't you rich?" is "you're rich, so you must be very smart!" Thus it is that many people assume that if powerful, well-compensated CEOs insist that "AI is changing everything," well then, AI must be changing everything.
But the evidence for this "changing everything" thesis is thin on the ground. Despite a global mania that has reduced the real, pressing need for digital sovereignty to the imaginary need to create "sovereign AI," no one can really articulate the case for "sovereign AI." If Donald Trump ordered Big Tech to turn off all of your country's chatbots tomorrow, nothing would change. Every one of your country's ministries and corporations would chug on with nary a hitch. Households, too, though perhaps a few of the younger members of those families would have to do their own homework again.
(Contrast this with what would transpire if Trump directed his tech giants to switch off your country's Office 365 access, or to brick your Android and iOS phones, or to killswitch your John Deere tractors. Your country would effectively cease to exist. If "digital sovereignty" means anything, it means doing something about this urgent fact):
The world is full of people who insist that "AI is changing everything" but who – when pressed – have to admit that what they mean is that they're pretty sure that AI will change everything. Eventually. After we allow it to consume all the planet's energy, carbon, water and financial resources.
Maybe.
(They're pretty sure.)
One person who's had a lot of opportunity to observe the shear between the stated business/AI situation and the real business AI situation is Nikhil Suresh from Hermit Tech, a consulting firm of "radically ethical data wizards" (that is, tech consultants). Suresh reports on his experience talking with hundreds of executives (and, more importantly, their subordinates) about what (if anything) AI is doing for business in an essay entitled "AI Mania Is Eviscerating Global Decisionmaking":
Suresh has a good track record of writing trenchant, frank criticism of AI. You may know him from his 2024 essay, "I Will Fucking Piledrive You If You Mention AI Again":
Or possibly from his "Contra Ptacek's Terrible Article On AI," a stinging rebuttal to Thomas Ptacek's widely read "My AI Skeptic Friends Are All Nuts":
While those are important pieces of critical AI realpolitik, none of them have the heft or urgency of "AI Mania Is Eviscerating Global Decisionmaking," whose thesis can be summed up with this passage from halfway through this 6,000-word article:
[W]e’re facing a coordination problem around executives being honest around the AI gains they’ve witnessed – if they co-operate, they keep their jobs. If they defect, they will possibly be fired by their embarrassed peers (who have now been implicitly called liars, cowards, or incompetents) and then replaced with someone that will toe the line anyway. If they could all admit the truth at once there might be some hope, but there is no way to coordinate that event.
In other words, corporate leadership is starting from the premise that AI has (or will) radically change the business, and they're working backwards from that premise to find the evidence to support this article of faith.
In support of this thesis, Suresh cites "hundreds" of conversations with execs and employees who spoke to him on the condition that he would "file the serial numbers" off their stories. These, combined with his own experience consulting for large, multi-billion-dollar companies make it clear that "AI mania" is an absolutely justifiable label for the state of AI in corporate circles.
Here are a few highlights from this morning's read – moments where I had to look away from my screen and read out a passage to my wife so that we could share a "holy shit" moment.
A person worked for a division that "pivoted" to re-engineer its software to create interfaces that support AI agents. When it became apparent that only ten users had touched this expensive new technology, they "pivoted" again to support "agentic workflows." Why did they double down on AI agents after discovering such yawning market indifference for "agentic"? "Because every company has to do something agentic now."
Suresh describes this as a literal religious mania. In the 500+ employee businesses Suresh studied, the only people who were promoted – or even spared from being fired – were people who professed "religious declarations of faith" about "the transformative power of AI." Employees who voiced honest, informed objections to AI in the workplace were passed over for promotions or targeted for layoffs.
This has created a situation in which everyone – "boards, executives, employees, vendors, consultants" – has a strong incentive to lie about how much AI is delivering for their companies. Suresh says he's seen announcements from publicly traded companies about their AI triumphs that he knows for a fact never took place.
Suresh says he's never seen a successful enterprise AI project: "Every single one – we have seen 0% success in a year and a half." Not one of their clients would face a business challenge if OpenAI went out of business tomorrow. The problem most companies struggle with is that they're "terminally bad at running software projects effectively." Adding AI to the mix doesn't solve this problem – it just adds a whole new range of ways that software deployment can fail.
Chatbots don't help. The internally facing chatbot that's supposed to help employees figure out how to navigate the business sucks because it is only as good as its training data – the business's documentation of its own processes. Businesses suck at documenting their processes. Customer-facing chatbots also suck. They either can't solve your problem, or, when they seem to solve your problem, the "solution" goes nowhere.
Suresh recounts his sole positive customer service chatbot experience: a Mitsubishi chatbot with a natural sounding, responsive voice politely took all the details of an automotive failure and promised him a callback. That callback never came, but Suresh is certain that Mitsubishi has logged this as a chatbot success story, even though the experience convinced him not to buy a Mitsubishi car.
Suresh and his team at Hermit Tech now have a policy of not even asking about ongoing AI projects. They've learned that by the time an AI project has begun, no one will discuss it honestly until it reaches a crisis point.
Suresh says he frequently encounters people who reflexively utter the AI catechism: "AI is changing everything." But when he presses these people for details, they admit that their organization "does not currently use LLMs for anything, and indeed, that they cannot name a single thing that has changed other than they get some use out of ChatGPT."
This shear ("AI is changing everything"/"Well, OK, we're not using AI for anything") is so extreme that Suresh once met an exec who confessed to crafting an AI-centered AI strategy for a $2b/year business, even though that exec "had never even used ChatGPT or any AI tool in their life."
Some people have privately admitted to Suresh that they've embraced AI in order to earn a career-boosting corporate reputation for "thought leadership." But many other people (especially nontechnical people) sincerely believe that AI is about to "change everything." As Suresh says, if you're in business with a liar, you might be able to reason with them in private – but you can't reason with a true believer.
The true believers are in charge. Suresh points out that it would be very weird for the CEO of an engineering firm or a hospital to mandate "specific procedures or building techniques without explicit agreement from the professionals on staff." But when it comes to AI, business leaders will confidently demand that the skilled professionals who perform the business's core functions use AI, even if those professionals don't think it will help.
As an aside: I remember the dotcom era, when the business press was full of articles about the conflict between CEOs and a new workforce that demanded the right to use the web on the job. Today, the business press is full of articles about the conflict between the workforce and CEOs who demand that they use AI.
Suresh describes workers who feel they have to "AI wash" their work: "They just do the work, the same way they have for decades, and say Claude did it." To add verisimilitude to this sham, they write circular processes in which one chatbot prompts another, and then the process repeats itself in reverse, for the sole purpose of consuming AI tokens to score a high rank on corporate "token leaderboards."
How to account for this wildly, expensively irrational corporate leadership? Suresh places the blame in the hypnotizing, mesmerizing power of the AI demo. For example: Hermit Tech is often engaged to set up a database product called Snowflake for its customers. Snowflake has a useless, expensive AI bolt-on called Cortex, that Snowflake itself describes as being 92% accurate under ideal circumstances (that is, at least 8% of the time, it will mislead you, perhaps very badly).
Suresh describes sales meetings with execs who were lukewarm on the idea of retooling with Snowflake, but who were very interested in Cortex. Against their better judgment, Suresh and his team provided them with a Cortex demo, carefully explaining that this AI tool could not satisfy their requirements. Without fail, this resulted in the previously lukewarm customers insisting that they be allowed to purchase Cortex immediately. Sales prospects who'd been unmoved by a pitch for new technology that would result in millions in savings were hypnotized by demos of a product that was described as unsuitable and unreliable.
To their credit, Hermit Tech refused to sell these customers Cortex, and stopped doing Cortex demos altogether. Suresh describes the experience of "the total 180°, that shift from ice-cold to red-hot buying frenzy" as "deeply unsettling." What's more, the Cortex demos that Suresh and co performed were, by his account, pretty uninspiring. The thing that these demos had going for them is that they showed AI actually doing something marginally useful, to execs who'd already spent millions on AI without having anything to show for their money. The spectacle of AI that does something galvanizes corporate leaders who feel like they're the only bosses who can't find a revolutionary use for AI in their businesses.
This is the situation up and down the corporate org-chart. Suresh has a reader whose title is "Head of AI" at a billion-dollar firm who tells him "their job is totally fraudulent but it was the only promotion pathway remaining at the organisation." This exec is hardly alone. They're part of a cohort of executives at companies that have publicly announced "100x" productivity gains, but who confessed to Suresh that nothing of the sort has happened.
Why did these companies make these claims? Because their customers were making the claims. How could you hope to sell to a company that had 100x'ed its productivity with AI unless you, too had 100x'ed your productivity? If, as a vendor, you walked into a boardroom and said that this wasn't a plausible claim, you'd be calling your sales prospect a liar, with real consequences: "getting enterprise contracts cancelled because you wanted to opine on something that doesn’t really matter to your organisation’s mission is a great way to get fired."
With the state of the industry dominated by froth, lies and mutual destruction pacts, it's no wonder that companies are deploying "totally gameable metrics such as 'money spent on AI'" as a means of evaluating employees and divisions.
Between true believers and people who must find ways to plausibly tout their AI usage, there is now a gigantic market for "AI solutions." At best these are just traditional tech consulting contracts, like migrating a database from Oracle to Snowflake, with some kind of ornamental AI usage around the edges so that the person who commissions the work can claim to be "procuring AI-enabled services" for the business.
This isn't a harmless frippery: contracts are delayed and work is put off until the work can be made "sufficiently AI" to attain the minimum degree of buzzword compliance. Worse: every fake AI project that produces real results (because it's not really AI) adds credibility to the AI true believers, who view these projects as proof that AI can do anything, and therefore demand to know why everything isn't being done by AI.
Suresh ends his essay with a long section on how to "navigate AI mania" – advice for how to smile and nod politely when you're confronted with AI bullshit, while steering clear of the worst consequences and avoiding needless fights. This looks like very sound advice for anyone in a corporate environment, but thankfully, that isn't me.
Rather than summarize that advice, I want to reflect a little on two questions that Suresh's essay raises but doesn't answer. The first is why? Why are people in power such easy converts to this religious mania?
I have my own theory. The most important discomfort that powerful people experience is having ego-shattering conflicts with subordinates who know how to do things they do not know how to do. The fact that you're "in charge" is hard to reconcile with the fact that the people you're nominally in charge of tell you that all your ideas are impossible, illegal, immoral, or lethal:
Take that Cortex demo. Sure, Cortex is an expensive, unreliable way to address a Snowflake database. But (unlike Snowflake) Cortex is controlled via conversational, plain-language commands. With Cortex, a boss doesn't need to ask an underling to retrieve information from the company Snowflake system, an interaction that might come with unsolicited feedback about the technical or commercial incoherence of the boss's request. Cortex is the underling, except that unlike a human underling, Cortex never back-sasses you about your foolish questions. The fact that it grossly misleads you 8% of the time is a small price to pay for a life untroubled by uppity pismires who insist that your ideas be connected to base reality as they understand it.
The other question Suresh implicitly raises is, "How can you reconcile the failure of AI in the enterprise with the individual claims of skilled technologists who insist that AI is helping them do great work?" The answer is that these AI users are "centaurs" – experienced workers who are assisted by automation on terms that they set for themselves:
Thanks to their skill and experience, these workers possess discernment, the ability to tell good code from bad, and (more importantly) good uses of code-generation tools from bad. They demonstrate the adage that worker-driven automation improves quality, while capital-driven automation improves throughput:
An automation technique that requires close supervision by skilled and experienced workers isn't going to be a raw productivity powerhouse. You don't "100x" your code this way, at least, not in the sense of firing 99 of your coders and having the remaining programmer pick up all their work. Rather, an automation tool that requires the continuous and conscientious exercise of discernment will let individual practitioners improve their work in extremely satisfying and useful ways. It's a way to spend more on operations in order to produce better outputs. It's not a way to cut your workforce, realize a gigantic savings, and still produce comparable goods and services at a far lower cost.
That is why some individual coders report such delight with their AI tools. They engage with those tools on their own terms, to improve their work in the ways that they, in their expert judgment, consider beneficial. No one ranks them on a "token-maximization" scoreboard. No one tells them they can't do a project if it isn't "sufficiently AI." When they set out to do a project, no one makes them prove that it couldn't be "done by AI."
As ever, the most important fact about a given technology isn't "what it does," but "who it does it for" and "who it does it to."
All the pathologies Suresh observes and documents so well in this piece are hypertrophied versions of the buzzword-compliance dysfunctions from previous bubbles, but at a scale never before seen. Quantity has a quality all its own. These businesses aren't just wasting billions – they're replacing skilled workers with defective chatbots. As I've written before, AI is the asbestos we're shoveling into the walls of our technological society. Our descendants will spend generations digging it out again, and the longer the bubble goes on without popping, the longer it will take to repair the damage.
In other words, corporate leadership is starting from the premise that AI has (or will) radically change the business, and they're working backwards from that premise to find the evidence to support this article of faith.
I've seen this happening and it is terrifying. Being told that the organization you work for has made a huge investment in AI, and that your opportunities for advancement (or even getting to keep the job you have) are contingent on finding a way to use that AI....What are those workers realistically supposed to do? Stand up for accuracy and get fired? Point out the ecological and ethical impacts of increasing AI use and hope the executive who made the AI investment doesn't decide to save money by terminating the squeaky wheels? Try to unionize and get fired? They're too scared to do anything but comply.
And that enhanced tendency toward compliance is a big part of the appeal of AI, as mentioned above. AI doesn't talk back, which business leaders LOVE. So human workers are learning to also not talk back, because they can tell it's dangerous to their jobs to do so. No one will say that the emperor has no clothes when the AI is confidently describing the clothes (and being so complimentary about the emperor's taste in fabric!). If you can't see the clothes, then that's a defect in you; nothing else could possibly explain it.
The funniest thing about the old "emperor's new clothes" story is the part where no one in that story stopped to think about the logical conclusion according to the described properties of that magical fabric.
Some nonzero number of people were going to perceive the emperor as naked.
They were so worried about personally being thought simple, or unsuited for their own job, that no one stopped to think about the people who really were unsuited for their jobs. And anyone who lives in a society knows we've got a fair number of those.
By the description originally given, anyone who was incompetent at their job would get an eyeful of bare emperor, because the cloth would be rendered invisible. Who would choose to dress in something like that? Who would risk parading through the street, appearing to be naked to people who were bad at their jobs?
And we're all just staring in confusion and disbelief at these naked C-suiters, as they proudly slap each other on their (naked) backs, congratulating each other on their masterful implementation of AI.
It feels as if we're having to choose between having a roof over our heads versus using our principles to keep the rain off of us. I can't really blame front line workers for choosing to stay housed.
kianamaiart
snowghoul
discoholicmusic
origamiaround
yousharknotpass
