I'm on a tour with my new book, the international bestseller Enshittification: catch me next in Madison, CT; Hamburg and Denver! Full schedule here.
Last night, I gave a speech for the University of Washington's "Neuroscience, AI and Society" lecture series, through the university's Computational Neuroscience Center. It was called "The Reverse Centaur’s Guide to Criticizing AI," and it's based on the manuscript for my next book, "The Reverse Centaur’s Guide to Life After AI," which will be out from Farrar, Straus and Giroux next June:
The talk was sold out, but here's the text of my lecture. I'm very grateful to UW for the opportunity, and for a lovely visit to Seattle!
==
I'm a science fiction writer, which means that my job is to make up futuristic parables about our current techno-social arrangements to interrogate not just what a gadget does, but who it does it for, and who it does it to.
What I don't do is predict the future. No one can predict the future, which is a good thing, since if the future were predictable, that would mean that what we all do couldn't change it. It would mean that the future was arriving on fixed rails and couldn't be steered.
Jesus Christ, what a miserable proposition!
Now, not everyone understands the distinction. They think sf writers are oracles, soothsayers. Unfortunately, even some of my colleagues labor under the delusion that they can "see the future."
But for every sf writer who deludes themselves into thinking that they are writing the future, there are a hundred sf fans who believe that they are reading the future, and a depressing number of those people appear to have become AI bros. The fact that these guys can't shut up about the day that their spicy autocomplete machine will wake up and turn us all into paperclips has led many confused journalists and conference organizers to try to get me to comment on the future of AI.
That's a thing I strenuously resisted doing, because I wasted two years of my life explaining patiently and repeatedly why I thought crypto was stupid, and getting relentless bollocked by cryptocurrency cultists who at first insisted that I just didn't understand crypto. And then, when I made it clear that I did understand crypto, insisted that I must be a paid shill.
This is literally what happens when you argue with Scientologists, and life is Just. Too. Short.
So I didn't want to get lured into another one of those quagmires, because on the one hand, I just don't think AI is that important of a technology, and on the other hand, I have very nuanced and complicated views about what's wrong, and not wrong, about AI, and it takes a long time to explain that stuff.
But people wouldn't stop asking, so I did what I always do. I wrote a book.
Over the summer I wrote a book about what I think about AI, which is really about what I think about AI criticism, and more specifically, how to be a good AI critic. By which I mean: "How to be a critic whose criticism inflicts maximum damage on the parts of AI that are doing the most harm." I titled the book The Reverse Centaur's Guide to Life After AI, and Farrar, Straus and Giroux will publish it in June, 2026.
But you don't have to wait until then because I am going to break down the entire book's thesis for you tonight, over the next 40 minutes. I am going to talk fast.
#
Start with what a reverse centaur is. In automation theory, a "centaur" is a person who is assisted by a machine. You're a human head being carried around on a tireless robot body. Driving a car makes you a centaur, and so does using autocomplete.
And obviously, a reverse centaur is machine head on a human body, a person who is serving as a squishy meat appendage for an uncaring machine.
Like an Amazon delivery driver, who sits in a cabin surrounded by AI cameras, that monitor the driver's eyes and take points off if the driver looks in a proscribed direction, and monitors the driver's mouth because singing isn't allowed on the job, and rats the driver out to the boss if they don't make quota.
The driver is in that van because the van can't drive itself and can't get a parcel from the curb to your porch. The driver is a peripheral for a van, and the van drives the driver, at superhuman speed, demanding superhuman endurance. But the driver is human, so the van doesn't just use the driver. The van uses the driver up.
Obviously, it's nice to be a centaur, and it's horrible to be a reverse centaur. There are lots of AI tools that are potentially very centaur-like, but my thesis is that these tools are created and funded for the express purpose of creating reverse-centaurs, which is something none of us want to be.
But like I said, the job of an sf writer is to do more than think about what the gadget does, and drill down on who the gadget does it for and who the gadget does it to. Tech bosses want us to believe that there is only one way a technology can be used. Mark Zuckerberg wants you to think that it's technologically impossible to have a conversation with a friend without him listening in. Tim Cook wants you to think that it's technologically impossible for you to have a reliable computing experience unless he gets a veto over which software you install and without him taking 30 cents out of every dollar you spend. Sundar Pichai wants you think that it's impossible for you to find a webpage unless he gets to spy on you from asshole to appetite.
This is all a kind of vulgar Thatcherism. Margaret Thatcher's mantra was "There is no alternative." She repeated this so often they called her "TINA" Thatcher: There. Is. No. Alternative. TINA.
"There is no alternative" is a cheap rhetorical slight. It's a demand dressed up as an observation. "There is no alternative" means "STOP TRYING TO THINK OF AN ALTERNATIVE." Which, you know, fuck that.
I'm an sf writer, my job is to think of a dozen alternatives before breakfast.
So let me explain what I think is going on here with this AI bubble, and sort out the bullshit from the material reality, and explain how I think we could and should all be better AI critics.
#
Start with monopolies: tech companies are gigantic and they don't compete, they just take over whole sectors, either on their own on in cartels.
Google and Meta control the ad market. Google and Apple control the mobile market, and Google pays Apple more than $20 billion/year not to make a competing search engine, and of course, Google has a 90% Search market-share.
Now, you'd think that this was good news for the tech companies, owning their whole sector.
But it's actually a crisis. You see, when a company is growing, it is a "growth stock," and investors really like growth stocks. When you buy a share in a growth stock, you're making a bet that it will continue to grow. So growth stocks trade at a huge multiple of their earnings. This is called the "price to earnings ratio" or "P/E ratio."
But once a company stops growing, it is a "mature" stock, and it trades at a much lower P/E ratio. So for ever dollar that Target – a mature company – brings in, it is worth ten dollars. It has a P/E ratio of 10, while Amazon has a P/E ratio of 36, which means that for ever dollar Amazon brings in, the market values it at $36.
It's wonderful to run a company that's got a growth stock. Your shares are as good as money. If you want to buy another company, or hire a key worker, you can offer stock instead of cash. And stock is very easy for companies to get, because shares are manufactured right there on the premises, all you have to do is type some zeroes into a spreadsheet, while dollars are much harder to come by. A company can only get dollars from customers or creditors.
So when Amazon bids against Target for a key acquisition, or a key hire, Amazon can bid with shares they make by typing zeroes into a spreadsheet, and Target can only bid with dollars they get from selling stuff to us, or taking out loans. which is why Amazon generally wins those bidding wars.
That's the upside of having a growth stock. But here's the downside: eventually a company has to stop growing. Like, say you get a 90% market share in your sector, how are you gonna grow?
Once the market decides that you aren't a growth stock, once you become mature, your stocks are revalued, to a P/E ratio befitting a mature stock.
If you are an exec at a dominant company with a growth stock, you have to live in constant fear that the market will decide that you're not likely to grow any further. Think of what happened to Facebook in the first quarter of 2022. They told investors that they experienced slightly slower growth in the USA than they had anticipated, and investors panicked. They staged a one-day, $240B sell off. A quarter-trillion dollars in 24 hours! At the time, it was the largest, most precipitous drop in corporate valuation in human history.
That's a monopolist's worst nightmare, because once you're presiding over a "mature" firm, the key employees you've been compensating with stock, experience a precipitous pay-drop and bolt for the exits, so you lose the people who might help you grow again, and you can only hire their replacements with dollars. With dollars, not shares.
And the same goes for acquiring companies that might help you grow, because they, too, are going to expect money, not stock. This is the paradox of the growth stock. While you are growing to domination, the market loves you, but once you achieve dominance, the market lops 75% or more off your value in a single stroke if they don't trust your pricing power.
Which is why growth stock companies are always desperately pumping up one bubble or another, spending billions to hype the pivot to video, or cryptocurrency, or NFTs, or Metaverse, or AI.
I'm not saying that tech bosses are making bets they don't plan on winning. But I am saying that winning the bet – creating a viable metaverse – is the secondary goal. The primary goal is to keep the market convinced that your company will continue to grow, and to remain convinced until the next bubble comes along.
So this is why they're hyping AI: the material basis for the hundreds of billions in AI investment.
#
Now I want to talk about how they're selling AI. The growth narrative of AI is that AI will disrupt labor markets. I use "disrupt" here in its most disreputable, tech bro sense
The promise of AI – the promise AI companies make to investors – is that there will be AIs that can do your job, and when your boss fires you and replaces you with AI, he will keep half of your salary for himself, and give the other half to the AI company.
That's it.
That's the $13T growth story that MorganStanley is telling. It's why big investors and institutionals are giving AI companies hundreds of billions of dollars. And because they are piling in, normies are also getting sucked in, risking their retirement savings and their family's financial security.
Now, if AI could do your job, this would still be a problem. We'd have to figure out what to do with all these technologically unemployed people.
But AI can't do your job. It can help you do your job, but that doesn't mean it's going to save anyone money. Take radiology: there's some evidence that AIs can sometimes identify solid-mass tumors that some radiologists miss, and look, I've got cancer. Thankfully, it's very treatable, but I've got an interest in radiology being as reliable and accurate as possible
If my Kaiser hospital bought some AI radiology tools and told its radiologists: "Hey folks, here's the deal. Today, you're processing about 100 x-rays per day. From now on, we're going to get an instantaneous second opinion from the AI, and if the AI thinks you've missed a tumor, we want you to go back and have another look, even if that means you're only processing 98 x-rays per day. That's fine, we just care about finding all those tumors."
If that's what they said, I'd be delighted. But no one is investing hundreds of billions in AI companies because they think AI will make radiology more expensive, not even if it that also makes radiology more accurate. The market's bet on AI is that an AI salesman will visit the CEO of Kaiser and make this pitch: "Look, you fire 9/10s of your radiologists, saving $20m/year, you give us $10m/year, and you net $10m/year, and the remaining radiologists' job will be to oversee the diagnoses the AI makes at superhuman speed, and somehow remain vigilant as they do so, despite the fact that the AI is usually right, except when it's catastrophically wrong.
"And if the AI misses a tumor, this will be the human radiologist's fault, because they are the 'human in the loop.' It's their signature on the diagnosis."
This is a reverse centaur, and it's a specific kind of reverse-centaur: it's what Dan Davies calles an "accountability sink." The radiologist's job isn't really to oversee the AI's work, it's to take the blame for the AI's mistakes.
This is another key to understanding – and thus deflating – the AI bubble. The AI can't do your job, but an AI salesman can convince your boss to fire you and replace you with an AI that can't do your job. This is key because it helps us build the kinds of coalitions that will be successful in the fight against the AI bubble.
If you're someone who's worried about cancer, and you're being told that the price of making radiology too cheap to meter, is that we're going to have to re-home America's 32,000 radiologists, with the trade-off that no one will every be denied radiology services again, you might say, "Well, OK, I'm sorry for those radiologists, and I fully support getting them job training or UBI or whatever. But the point of radiology is to fight cancer, not to pay radiologists, so I know what side I'm on."
AI hucksters and their customers in the C-suites want the public on their side. They want to forge a class alliance between AI deployers and the people who enjoy the fruits of the reverse centaurs' labor. They want us to think of ourselves as enemies to the workers.
Now, some people will be on the workers' side because of politics or aesthetics. They just like workers better than their bosses. But if you want to win over all the people who benefit from your labor, you need to understand and stress how the products of the AI will be substandard. That they are going to get charged more for worse things. That they have a shared material interest with you.
Will those products be substandard? There's every reason to think so. Earlier, I alluded to "automation blindness, "the physical impossibility of remaining vigilant for things that rarely occur. This is why TSA agents are incredibly good at spotting water bottles. Because they get a ton of practice at this, all day, every day. And why they fail to spot the guns and bombs that government red teams smuggle through checkpoints to see how well they work, because they just don't have any practice at that. Because, to a first approximation, no one deliberately brings a gun or a bomb through a TSA checkpoint.
Automation blindness is the Achilles' heel of "humans in the loop."
Think of AI software generation: there are plenty of coders who love using AI, and almost without exception, they are senior, experienced coders, who get to decide how they will use these tools. For example, you might ask the AI to generate a set of CSS files to faithfully render a web-page across multiple versions of multiple browsers. This is a notoriously fiddly thing to do, and it's pretty easy to verify if the code works – just eyeball it in a bunch of browsers. Or maybe the coder has a single data file they need to import and they don't want to write a whole utility to convert it.
Tasks like these can genuinely make coders more efficient and give them more time to do the fun part of coding, namely, solving really gnarly, abstract puzzles. But when you listen to business leaders talk about their AI plans for coders, it's clear they're not looking to make some centaurs.
They want to fire a lot of tech workers – 500,000 over the past three years – and make the rest pick up their work with coding, which is only possible if you let the AI do all the gnarly, creative problem solving, and then you do the most boring, soul-crushing part of the job: reviewing the AIs' code.
And because AI is just a word guessing program, because all it does is calculate the most probable word to go next, the errors it makes are especially subtle and hard to spot, because these bugs are literally statistically indistinguishable from working code (except that they're bugs).
Here's an example: code libraries are standard utilities that programmers can incorporate into their apps, so they don't have to do a bunch of repetitive programming. Like, if you want to process some text, you'll use a standard library. If it's an HTML file, that library might be called something like lib.html.text.parsing; and if it's a DOCX file, it'll be lib.docx.text.parsing. But reality is messy, humans are inattentive and stuff goes wrong, so sometimes, there's another library, this one for parsing PDFs, and instead of being called lib.pdf.text.parsing, it's called lib.text.pdf.parsing.
Now, because the AI is a statistical inference engine, because all it can do is predict what word will come next based on all the words that have been typed in the past, it will "hallucinate" a library called lib.pdf.text.parsing. And the thing is, malicious hackers know that the AI will make this error, so they will go out and create a library with the predictable, hallucinated name, and that library will get automatically sucked into your program, and it will do things like steal user data or try and penetrate other computers on the same network.
And you, the human in the loop – the reverse centaur – you have to spot this subtle, hard to find error, this bug that is literally statistically indistinguishable from correct code. Now, maybe a senior coder could catch this, because they've been around the block a few times, and they know about this tripwire.
But guess who tech bosses want to preferentially fire and replace with AI? Senior coders. Those mouthy, entitled, extremely highly paid workers, who don't think of themselves as workers. Who see themselves as founders in waiting, peers of the company's top management. The kind of coder who'd lead a walkout over the company building drone-targeting systems for the Pentagon, which cost Google ten billion dollars in 2018.
For AI to be valuable, it has to replace high-wage workers, and those are precisely the experienced workers, with process knowledge, and hard0won intuition, who might spot some of those statistically camouflaged AI errors.
Like I said, the point here is to replace high-waged workers
And one of the reasons the AI companies are so anxious to fire coders is that coders are the princes of labor. They're the most consistently privileged, sought-after, and well-compensated workers in the labor force.
If you can replace coders with AI, who cant you replace with AI? Firing coders is an ad for AI.
Which brings me to AI art. AI art – or "art" – is also an ad for AI, but it's not part of AI's business model.
Let me explain: on average, illustrators don't make any money. They are already one of the most immiserated, precartized groups of workers out there. They suffer from a pathology called "vocational awe." That's a term coined by the librarian Fobazi Ettarh, and it refers to workers who are vulnerable to workplace exploitation because they actually care about their jobs – nurses, librarians, teachers, and artists.
If AI image generators put every illustrator working today out of a job, the resulting wage-bill savings would be undetectable as a proportion of all the costs associated with training and operating image-generators. The total wage bill for commercial illustrators is less than the kombucha bill for the company cafeteria at just one of Open AI's campuses.
The purpose of AI art – and the story of AI art as a death-knell for artists – is to convince the broad public that AI is amazing and will do amazing things. It's to create buzz. Which is not to say that it's not disgusting that former OpenAI CTO Mira Murati told a conference audience that "some creative jobs shouldn't have been there in the first place," and that it's not especially disgusting that she and her colleagues boast about using the work of artists to ruin those artists' livelihoods.
It's supposed to be disgusting. It's supposed to get artists to run around and say, "The AI can do my job, and it's going to steal my job, and isn't that terrible?"
Because the customers for AI – corporate bosses – don't see AI taking workers' jobs as terrible. They see it as wonderful.
But can AI do an illustrator's job? Or any artist's job?
Let's think about that for a second. I've been a working artist since I was 17 years old, when I sold my first short story, and I've given it a lot of thought, and here's what I think art is: it starts with an artist, who has some vast, complex, numinous, irreducible feeling in their mind. And the artist infuses that feeling into some artistic medium. They make a song, or a poem, or a painting, or a drawing, or a dance, or a book, or a photograph. And the idea is, when you experience this work, a facsimile of the big, numinous, irreducible feeling will materialize in your mind.
Now that I've defined art, we have to go on a little detour.
I have a friend who's a law professor, and before the rise of chatbots, law students knew better than to ask for reference letters from their profs, unless they were a really good student. Because those letters were a pain in the ass to write. So if you advertised for a postdoc and you heard from a candidate with a reference letter from a respected prof, the mere existence of that letter told you that the prof really thought highly of that student.
But then we got chatbots, and everyone knows that you generate a reference letter by feeding three bullet points to an LLM, and it'll barf up five paragraphs of florid nonsense about the student.
So when my friend advertises for a postdoc, they are flooded with reference letters, and they deal with this flood by feeding all these letters to another chatbot, and ask it to reduce them back to three bullet points. Now, obviously, they won't be the same bullet-points, which makes this whole thing terrible.
But just as obviously, nothing in that five-paragraph letter except the original three bullet points are relevant to the student. The chatbot doesn't know the student. It doesn't know anything about them. It cannot add a single true or useful statement about the student to the letter.
What does this have to do with AI art? Art is a transfer of a big, numinous, irreducible feeling from an artist to someone else. But the image-gen program doesn't know anything about your big, numinous, irreducible feeling. The only thing it knows is whatever you put into your prompt, and those few sentences are diluted across a million pixels or a hundred thousand words, so that the average communicative density of the resulting work is indistinguishable from zero.
It's possible to infuse more communicative intent into a work: writing more detailed prompts, or doing the selective work of choosing from among many variants, or directly tinkering with the AI image after the fact, with a paintbrush or Photoshop or The Gimp. And if there will every be a piece of AI art that is good art – as opposed to merely striking, or interesting, or an example of good draftsmanship – it will be thanks to those additional infusions of creative intent by a human.
And in the meantime, it's bad art. It's bad art in the sense of being "eerie," the word Mark Fisher uses to describe "when there is something present where there should be nothing, or is there is nothing present when there should be something."
AI art is eerie because it seems like there is an intender and an intention behind every word and every pixel, because we have a lifetime of experience that tells us that paintings have painters, and writing has writers. But it's missing something. It has nothing to say, or whatever it has to say is so diluted that it's undetectable.
The images were striking before we figured out the trick, but now they're just like the images we imagine in clouds or piles of leaves. We're the ones drawing a frame around part of the scene, we're the ones focusing on some contours and ignoring the others. We're looking at an inkblot, and it's not telling us anything.
Sometimes that can be visually arresting, and to the extent that it amuses people in a community of prompters and viewers, that's harmless.
I know someone who plays a weekly Dungeons and Dragons game over Zoom. It's transcribed by an open source model running locally on the dungeon master's computer, which summarizes the night's session and prompts an image generator to create illustrations of key moments. These summaries and images are hilarious because they're full of errors. It's a bit of harmless fun, and it bring a small amount of additional pleasure to a small group of people. No one is going to fire an illustrator because D&D players are image-genning funny illustrations where seven-fingered paladins wrestle with orcs that have an extra hand.
But bosses have and will fire illustrators, because they fantasize about being able to dispense with creative professionals and just prompt an AI. Because even though the AI can't do the illustrator's job, an AI salesman can convince the illustrator's boss to fire them and replace them with an AI that can't do their job.
This is a disgusting and terrible juncture, and we should not simply shrug our shoulders and accept Thatcherism's fatalism: "There is no alternative."
So what is the alternative? A lot of artists and their allies think they have an answer: they say we should extend copyright to cover the activities associated with training a model.
And I'm here to tell you they are wrong:w rong because this would inflict terrible collateral damage on socially beneficial activities, and it would represent a massive expansion of copyright over activities that are currently permitted – for good reason!.
Let's break down the steps in AI training.
First, you scrape a bunch of web-pages This is unambiguously legal under present copyright law. You do not need a license to make a transient copy of a copyrighted work in order to analyze it, otherwise search engines would be illegal. Ban scraping and Google will be the last search engine we ever get, the Internet Archive will go out of business, that guy in Austria who scraped all the grocery store sites and proved that the big chains were colluding to rig prices would be in deep trouble.
Next, you perform analysis on those works. Basically, you count stuff on them: count pixels and their colors and proximity to other pixels; or count words. This is obviously not something you need a license for. It's just not illegal to count the elements of a copyrighted work. And we really don't want it to be, not if you're interested in scholarship of any kind.
And it's important to note that counting things is legal, even if you're working from an illegally obtained copy. Like, if you go to the flea market, and you buy a bootleg music CD, and you take it home and you make a list of all the adverbs in the lyrics, and you publish that list, you are not infringing copyright by doing so.
Perhaps you've infringed copyright by getting the pirated CD, but not by counting the lyrics.
This is why Anthropic offered a $1.5b settlement for training its models based on a ton of books it downloaded from a pirate site: not because counting the words in the books infringes anyone's rights, but because they were worried that they were going to get hit with $150k/book statutory damages for downloading the files.
OK, after you count all the pixels or the words, it's time for the final step: publishing them. Because that's what a model is: a literary work (that is, a piece of software) that embodies a bunch of facts about a bunch of other works, word and pixel distribution information, encoded in a multidimensional array.
And again, copyright absolutely does not prohibit you from publishing facts about copyrighted works. And again, no one should want to live in a world where someone else gets to decide which truthful, factual statements you can publish.
But hey, maybe you think this is all sophistry. Maybe you think I'm full of shit. That's fine. It wouldn't be the first time someone thought that.
After all, even if I'm right about how copyright works now, there's no reason we couldn't change copyright to ban training activities, and maybe there's even a clever way to wordsmith the law so that it only catches bad things we don't like, and not all the good stuff that comes from scraping, analyzing and publishing.
Well, even then, you're not gonna help out creators by creating this new copyright. If you're thinking that you can, you need to grapple with this fact: we have monotonically expanded copyright since 1976, so that today, copyright covers more kinds of works, grants exclusive rights over more uses, and lasts longer.
And today, the media industry is larger and more profitable than it has ever been, and also: the share of media industry income that goes to creative workers is lower than its ever been, both in real terms, and as a proportion of those incredible gains made by creators' bosses at the media company.
So how it is that we have given all these new rights to creators, and those new rights have generated untold billions, and left creators poorer? It's because in a creative market dominated by five publishers, four studios, three labels, two mobile app stores, and a single company that controls all the ebooks and audiobooks, giving a creative worker extra rights to bargain with is like giving your bullied kid more lunch money.
It doesn't matter how much lunch money you give the kid, the bullies will take it all. Give that kid enough money and the bullies will hire an agency to run a global campaign proclaiming "think of the hungry kids! Give them more lunch money!"
Creative workers who cheer on lawsuits by the big studios and labels need to remember the first rule of class warfare: things that are good for your boss are rarely what's good for you.
The day Disney and Universal filed suit against Midjourney, I got a press release from the RIAA, which represents Disney and Universal through their recording arms. Universal is the largest label in the world. Together with Sony and Warner, they control 70% of all music recordings in copyright today.
It starts: "There is a clear path forward through partnerships that both further AI innovation and foster human artistry."
It ends: "This action by Disney and Universal represents a critical stand for human creativity and responsible innovation."
And it's signed by Mitch Glazier, CEO of the RIAA.
It's very likely that name doesn't mean anything to you. But let me tell you who Mitch Glazier is. Today, Mitch Glazier is the CEO if the RIAA, with an annual salary of $1.3m. But until 1999, Mitch Glazier was a key Congressional staffer, and in 1999, Glazier snuck an amendment into an unrelated bill, the Satellite Home Viewer Improvement Act, that killed musicians' right to take their recordings back from their labels.
This is a practice that had been especially important to "heritage acts" (which is a record industry euphemism for "old music recorded by Black people"), for whom this right represented the difference between making rent and ending up on the street.
When it became clear that Glazier had pulled this musician-impoverishing scam, there was so much public outcry, that Congress actually came back for a special session, just to vote again to cancel Glazier's amendment. And then Glazier was kicked out of his cushy Congressional job, whereupon the RIAA started paying more than $1m/year to "represent the music industry."
This is the guy who signed that press release in my inbox. And his message was: The problem isn't that Midjourney wants to train a Gen AI model on copyrighted works, and then use that model to put artists on the breadline. The problem is that Midjourney didn't pay RIAA members Universal and Disney for permission to train a model. Because if only Midjourney had given Disney and Universal several million dollars for training right to their catalogs, the companies would have happily allowed them to train to their heart's content, and they would have bought the resulting models, and fired as many creative professionals as they could.
I mean, have we already forgotten the Hollywood strikes? I sure haven't. I live in Burbank, home to Disney, Universal and Warner, and I was out on the line with my comrades from the Writers Guild, offering solidarity on behalf of my union, IATSE 830, The Animation Guild, where I'm a member of the writers' unit.
And I'll never forget when one writer turned to me and said, "You know, you prompt an LLM exactly the same way an exec gives shitty notes to a writers' room. You know: 'Make me ET, except it's about a dog, and put a love interest in there, and a car chase in the second act.' The difference is, you say that to a writers' room and they all make fun of you and call you a fucking idiot suit. But you say it to an LLM and it will cheerfully shit out a terrible script that conforms exactly to that spec (you know, Air Bud)."
These companies are desperate to use AI to displace workers. When Getty Images sues AI companies, it's not representing the interests of photographers. Getty hates paying photographers! Getty just wants to get paid for the training run, and they want the resulting AI model to have guardrails, so it will refuse to create images that compete with Getty's images for anyone except Getty. But Getty will absolutely use its models to bankrupt as many photographers as it possibly can.
A new copyright to train models won't get us a world where models aren't used to destroy artists, it'll just get us a world where the standard contracts of the handful of companies that control all creative labor markets are updated to require us to hand over those new training rights to those companies. Demanding a new copyright just makes you a useful idiot for your boss, a human shield they can brandish in policy fights, a tissue-thin pretense of "won't someone think of the hungry artists?"
When really what they're demanding is a world where 30% of the investment capital of the AI companies go into their shareholders' pockets. When an artist is being devoured by rapacious monopolies, does it matter how they divvy up the meal?
We need to protect artists from AI predation, not just create a new way for artists to be mad about their impoverishment.
And incredibly enough, there's a really simple way to do that. After 20+ years of being consistently wrong and terrible for artists' rights, the US Copyright Office has finally done something gloriously, wonderfully right. All through this AI bubble, the Copyright Office has maintained – correctly – that AI-generated works cannot be copyrighted, because copyright is exclusively for humans. That's why the "monkey selfie" is in the public domain. Copyright is only awarded to works of human creative expression that are fixed in a tangible medium.
And not only has the Copyright Office taken this position, they've defended it vigorously in court, repeatedly winning judgments to uphold this principle.
The fact that every AI created work is in the public domain means that if Getty or Disney or Universal or Hearst newspapers use AI to generate works – then anyone else can take those works, copy them, sell them, or give them away for free. And the only thing those companies hate more than paying creative workers, is having other people take their stuff without permission.
The US Copyright Office's position means that the only way these companies can get a copyright is to pay humans to do creative work. This is a recipe for centaurhood. If you're a visual artist or writer who uses prompts to come up with ideas or variations, that's no problem, because the ultimate work comes from you. And if you're a video editor who uses deepfakes to change the eyelines of 200 extras in a crowd-scene, then sure, those eyeballs are in the public domain, but the movie stays copyrighted.
But creative workers don't have to rely on the US government to rescue us from AI predators. We can do it ourselves, the way the writers did in their historic writers' strike. The writers brought the studios to their knees. They did it because they are organized and solidaristic, but also are allowed to do something that virtually no other workers are allowed to do: they can engage in "sectoral bargaining," whereby all the workers in a sector can negotiate a contract with every employer in the sector.
That's been illegal for most workers since the late 1940s, when the Taft-Hartley Act outlawed it. If we are gonna campaign to get a new law passed in hopes of making more money and having more control over our labor, we should campaign to restore sectoral bargaining, not to expand copyright.
Our allies in a campaign to expand copyright are our bosses, who have never had our best interests at heart. While our allies in the fight for sector bargaining are every worker in the country. As the song goes, "Which side are you on?"
OK, I need to bring this talk in for a landing now, because I'm out of time, so I'm going to close out with this: AI is a bubble and bubbles are terrible.
Bubbles transfer the life's savings of normal people who are just trying to have a dignified retirement to the wealthiest and most unethical people in our society, and every bubble eventually bursts, taking their savings with it.
But not every bubble is created equal. Some bubbles leave behind something productive. Worldcom stole billions from everyday people by defrauding them about orders for fiber optic cables. The CEO went to prison and died there. But the fiber outlived him. It's still in the ground. At my home, I've got 2gb symmetrical fiber, because AT&T lit up some of that old Worldcom dark fiber.
All things being equal, it would have been better if Worldcom hadn't ever existed, but the only thing worse than Worldcom committing all that ghastly fraud would be if there was nothing to salvage from the wreckage.
I don't think we'll salvage much from cryptocurrency, for example. Sure, there'll be a few coders who've learned something about secure programming in Rust. But when crypto dies, what it will leave behind is bad Austrian economics and worse monkey JPEGs.
AI is a bubble and it will burst. Most of the companies will fail. Most of the data-centers will be shuttered or sold for parts. So what will be left behind?
We'll have a bunch of coders who are really good at applied statistics. We'll have a lot of cheap GPUs, which'll be good news for, say, effects artists and climate scientists, who'll be able to buy that critical hardware at pennies on the dollar. And we'll have the open source models that run on commodity hardware, AI tools that can do a lot of useful stuff, like transcribing audio and video, describing images, summarizing documents, automating a lot of labor-intensive graphic editing, like removing backgrounds, or airbrushing passersby out of photos. These will run on our laptops and phones, and open source hackers will find ways to push them to do things their makers never dreamt of.
If there had never been an AI bubble, if all this stuff arose merely because computer scientists and product managers noodled around for a few year coming up with cool new apps for back-propagation, machine learning and generative adversarial networks, most people would have been pleasantly surprised with these interesting new things their computers could do. We'd call them "plugins."
It's the bubble that sucks, not these applications. The bubble doesn't want cheap useful things. It wants expensive, "disruptive" things: Big foundation models that lose billions of dollars every year.
When the AI investment mania halts, most of those models are going to disappear, because it just won't be economical to keep the data-centers running. As Stein's Law has it: "Anything that can't go on forever eventually stops."
The collapse of the AI bubble is going to be ugly. Seven AI companies currently account for more than a third of the stock market, and they endlessly pass around the same $100b IOU.
Bosses are mass-firing productive workers and replacing them with janky AI, and when the janky AI is gone, no one will be able to find and re-hire most of those workers, we're going to go from disfunctional AI systems to nothing.
AI is the asbestos in the walls of our technological society, stuffed there with wild abandon by a finance sector and tech monopolists run amok. We will be excavating it for a generation or more.
So we need to get rid of this bubble. Pop it, as quickly as we can. To do that, we have to focus on the material factors driving the bubble. The bubble isn't being driven by deepfake porn, oOr election disinformation, or AI image-gen, or slop advertising.
All that stuff is terrible and harmful, but it's not driving investment. The total dollar figure represented by these apps doesn't come close to making a dent in the capital expenditures and operating costs of AI. They are peripheral, residual uses: flashy, but unimportant to the bubble.
Get rid of all those uses and you reduce the expected income of AI companies by a sum so small it rounds to zero.
Same goes for all that "AI Safety" nonsense, that purports to concern itself with preventing an AI from attaining sentience and turning us all into paperclips. First of all, this is facially absurd. Throwing more words and GPUs into the word-guessing program won't make it sentient. That's like saying, "Well, we keep breeding these horses to run faster and faster, so it's only a matter of time until one of our mares gives birth to a locomotive." A human mind is not a word-guessing program with a lot of extra words.
I'm here for science fiction thought experiments, don't get me wrong. But also, don't mistake sf for prophesy. SF stories about superintelligence are futuristic parables, not business plans, roadmaps, or predictions.
The AI Safety people say they are worried that AI is going to end the world, but AI bosses love these weirdos. Because on the one hand, if AI is powerful enough to destroy the world, think of how much money it can make! And on the other hand, no AI business plan has a line on its revenue projections spreadsheet labeled "Income from turning the human race into paperclips." So even if we ban AI companies from doing this, we won't cost them a dime in investment capital.
To pop the bubble, we have to hammer on the forces that created the bubble: the myth that AI can do your job, especially if you get high wages that your boss can claw back; the understanding that growth companies need a succession of ever-more-outlandish bubbles to stay alive; the fact that workers and the public they serve are on one side of this fight, and bosses and their investors are on the other side.
Because the AI bubble really is very bad news, it's worth fighting seriously, and a serious fight against AI strikes at its roots: the material factors fueling the hundreds of billions in wasted capital that are being spent to put us all on the breadline and fill all our walls will high-tech asbestos.
If you'd like an essay-formatted version of this post to read or share, here's a link to it on pluralistic.net, my surveillance-free, ad-free, tracker-free blog:
If you'd like an essay-formatted version of this post to read or share, here's a link to it on pluralistic.net, my surveillance-free, ad-free, tracker-free blog:
"Gish Gallop" is the debating term for an opponent who makes so many claims that "it's impossible to address them in the time available" (it's named for Creationist Duane Gish, who was notorious for this tactic):
https://en.wikipedia.org/wiki/Gish_gallop
I think about the Gish Gallop whenever I'm asked to comment on AI.
Here's a recent example: last week, I had a pre-interview call with a radio producer who wanted me to come on a 13-minute segment to discusses "whether there's a problem with AI governance?"
I asked what the show meant by that: was it whether regulation of AI in commercial or public sector decision-making needed more oversight? Was it that the siting and provisioning of data-centers needed more democratic accountability? Was it that workers deserved more of a say in AI's impact on labor markets? Was it that customers and/or audiences should be able to opt out of AI customer service and AI slop? Was it about whether we needed some kind of system to prevent "runaway AI," in the event that we teach so many words to the word-guessing program that it wakes up, becomes God, and turns us all into paperclips?
"Oh," the producer said, "all of that."
In 13 minutes.
You see the problem, right? The AI industry has made so many claims about its past, present and future that it's almost impossible to have a reasonable critical conversation about it:
Shortly after I did the radio show, a newspaper editor who'd heard my segment got in touch to ask me if I'd write an 800-word op-ed about the subject, and also, could I address claims that "AI is the next Industrial Revolution?"
I keep finding myself on stages or panels where an AI-struck person says something like, "AI is the next industrial revolution. It will change everything we do. It will let anyone create important works of art. It will cure cancer. It will take us to space. It will solve the climate crisis."
Or sometimes it's an AI critic, but that person's criticism is really more "criti-hype," which is when you accept tech industry hype claims at face value, and then criticize them rather than questioning them:
AI criti-hype might ask what we'll do once AI takes all our jobs, or what we'll do when AI replaces the government or teachers or doctors, or what we'll do when AI can bypass our critical faculties and brainwash us or drive us all mad.
What do you say to that? I usually start by talking about whether there's any economic basis for keeping the AI servers running. AI is – by far – the money-losingest venture in human history, and it's practically impossible to overstate just how bad the AI business is. Not only does AI have terrible unit economics, those unit economics are getting worse over time:
AI's happiest customers cite cost-benefit calculations that depend on truly unimaginable subsidies from the AI companies, who are basically selling $100 bills for $5 apiece. It would be pretty amazing if you couldn't find people who'd extol the virtues of this arrangement. But when AI companies try to raise the price of those $100 bills to, say, $20 apiece, those ecstatic customers fly into a rage and start loudly proclaiming that AI is so inefficient that they will lose money on this arrangement:
Now, it shouldn't fall to me, a card-carrying member of the Democratic Socialists of America, to point out that capitalist enterprises require profits to be sustainable. You can't keep a business afloat by selling $100 bills for $5, nor for $20. You can't even make a profit selling $100 bills for $100 apiece! For a company to succeed, it needs to take in more than it expends.
AI is a money-furnace, and AI hustlers are clearly on the hunt for a way to force all of us to feed every dime we've got to it. Elon Musk's (now scuttled) gambit to make every pension saver in America bail out Grok (and Twitter, but at a mere $44b, the losses from Twitter are dwarfed by the titanic losses from Grok) was the most ambitious and shameless population-scale bag-holder scheme, but it's not the only one:
So before we ask about the capabilities AI will acquire in the future, we should at least give some consideration to the question of whether anyone will be willing to fund the development of those capabilities, and if so, where the money would come from? Likewise, before we ask whether AI can perform adequately in a job, we should at least consider the possibility that the company that sells that AI tool will be bankrupt in a year or two. When we fight about data-center buildout, we mostly talk about the (considerable) environmental downsides to them – but what about the question of what we will do with these data-centers after their owners go bankrupt, possibly even before they can be provisioned with electricity? How many laser-tag arenas do we actually need?
This is just one example of the questions that you could spend days unpacking, which make many of the other questions about AI a little silly. Like, even if you think there are limitless returns to scale for creating new AI capabilities, which means that if we keep the money-furnace burning it's only a matter of time until it powers a cure for cancer and the end of the climate emergency, how much money do we need to shovel into the furnace before that happens, and where will it come from? There are plenty of cancer researchers who have promising approaches they haven't been able to pursue due to funding shortfalls.
Unless there's some way to estimate how much money we have to give to AI companies before they cure cancer, we should at least consider the possibility that the true sum is "more money than exists now and that will ever exist." We should also consider that whatever benefits to cancer research that AI might deliver could come with a higher price-tag than the promising cancer research we're dropping because we can't find far more modest sums.
Likewise, it may be that the amount of CO2 that AI will generate atmosphere before it "solves climate change" will render Earth permanently unfit for humans, consuming the only habitable planet capable of sustaining human life in the known universe. I mean, I suppose that's one way to "solve" climate change, but it's a pretty drastic solution.
My next book (out later this month) is The Reverse Centaur's Guide to Life After AI. I wrote it because I was frustrated by other people demanding that I talk to them about AI, and then handing me 800 words or 13 minutes to address fifty nebulous, poorly supported claims about AI:
Now that I'm about to go out on the road with the book, I find myself frustrated anew by the need to try and pull together a compact way to address the broad, incoherent claims the industry uses to keep its bubble inflated and the money furnaces roaring. The series of essays I've developed here on Pluralistic are part of that effort:
But it occurred to me that this whole enterprise of making sense of AI needs to be framed in the context of the messiness of AI itself, and AI boosters' overwhelming, promiscuous and disjointed Gish Gallop.
I'm on a tour with my new book, the international bestseller Enshittification: catch me next in Miami, Burbank, Lisbon! Full schedule here (New dates just added in San Diego and Denver!).
Amazon made $35 billion in profit last year, so they're celebrating by laying off 14,000 workers (a number they say will rise to 30,000). This is the kind of thing that Wall Street loves, and this layoff comes after a string of pronouncements from Amazon CEO Andy Jassy about how AI is going to let them fire tons of workers.
That's the AI story, after all. It's not about making workers more productive or creative. The only way to recoup the $700 billion in capital expenditure to date (to say nothing of AI companies' rather fanciful coming capex commitments) is by displacing workers – a lot of workers. Bain & Co say the sector needs to be grossing $2 trillion by 2030 in order to break even, which is more than the combined grosses of Amazon, Google, Microsoft, Apple Nvidia and Meta:
Every investor who has put a nickel into that $700b capex is counting on bosses firing a lot of workers and replacing them with AI. Amazon is also counting on people buying a lot of AI from it after firing those workers. The company has sunk $120b into AI this year alone.
There's just one problem: AI can't do our jobs. Oh, sure, an AI salesman can convince your boss to fire you and replace you with an AI that can't do your job, but that's the world's easiest sales-call. Your boss is relentlessly horny for firing you:
What's Amazon to do? How do they convince you to buy enough AI to justify that $180b in capital expenditure? Somehow, they have to convince you that an AI can do your workers' jobs. One way to sell that pitch is to fire a ton of Amazon workers and announce that their jobs have been given to a chatbot. This isn't a production strategy, it's a marketing strategy – it's Amazon deliberately taking an efficiency loss by firing workers in a desperate bid to convince you that you can fire your workers:
Amazon does use a lot of AI in its production, of course. AI is the "digital whip" that Amazon uses to allow itself to control drivers who (nominally) work for subcontractors. This lets Amazon force workers into unsafe labor practices that endanger them and the people they share the roads with, while offloading responsibility onto "independent delivery service" operators and the drivers themselves:
Amazon leadership has announced that AI has or will shortly replace its coders as well. But chatbots can't do software engineering – sure, they can write code, but writing code is only a small part of software engineering. An engineer's job is to maintain a very deep and wide context window, one that considers how each piece of code interacts with the software that executes before it and after it, and with the systems that feed into it and accept its output.
There's one thing AI struggles with beyond all else: maintaining context. Each linear increase in context that you demand from AI results in an exponential increase in computational expense. AI has no object permanence. It doesn't know where it's been and it doesn't know where it's going. It can't remember how many fingers it's drawn, so it doesn't know when to stop. It can write a routine, but it can't engineer a system.
When tech bosses dream of firing coders and replacing them with AI, they're fantasizing about getting rid of their highest-paid, most self-assured workers and transforming the insecure junior programmers leftover into AI babysitters whose job it is to evaluate and integrate that code at a speed that no one – much less a junior programmer – can meet if they are to do a careful and competent job:
The jobs that can be replaced with AI are the jobs that companies already gave up on doing well. If you've already outsourced your customer service to an overseas call-center whose workers are not empowered to solve any of your customers' problems, why not fire those workers and replace them with chatbots? The chatbots also can't solve anyone's problems, and they're even cheaper than overseas call-center workers:
Amazon CEO Andy Jassy wrote that he "is convinced" that firing workers will make the company "AI ready," but it's not clear what he means by that. Does he mean that the mass firings will save money while maintaining quality, or that mass firings will help Amazon recoup the $180,000,000,000 it spent on AI this year?
Bosses really want AI to work, because they really, really want to fire you. As Allison Morrow writes for CNN bosses are firing workers in anticipation of the savings AI will produce…someday:
All this can feel improbable. Would bosses really fire workers on the promise of eventual AI replacements, leaving themselves with big bills for AI and falling revenues as the absence of those workers is felt?
The answer is a resounding yes. The AI industry has done such a good job of convincing bosses that AI can do their workers' jobs that each boss for whom AI fails assumes that they've done something wrong. This is a familiar dynamic in con-jobs.
The people who get sucked into pyramid schemes all think that they are the only ones failing to sell any of the "merchandise" they shell out every month to buy, and that no one else has a garage full of unsold leggings or essential oils. They don't know that, to a first approximation, the MLM industry has no sales, and relies entirely on "entrepreneurs" lying to themselves and one another about the demand for their wares, paying out of their own pocket for goods that no one wants.
The MLM industry doesn't just rely on this deception – they capitalize on it, by selling those self-flagellating "entrepreneurs" all kinds of expensive training courses that promise to help them overcome the personal defects that stop them from doing as well as all those desperate liars boasting about their incredible MLM sales success:
The AI industry has its own version of those sales coaching courses – there's a whole secondary industry of management consultancies and business schools offering high-ticket "continuing education" courses to bosses who think that the only reason the AI they've purchased isn't saving them money is that they're doing AI wrong.
Amazon really needs AI to work. Last week, Ed Zitron published an extensive analysis of leaked documents showing how much Amazon is making from AI companies who are buying cloud services from it. His conclusion? Take away AI and Amazon's cloud division is in steep decline:
https://www.wheresyoured.at/costs/
What's more, those big-money AI customers – like Anthropic – are losing tens of billions of dollars per year, relying on investors to keep handing them money to incinerate. Amazon needs bosses to believe they can fire workers and replace them with AI, because that way, investors will keep giving Anthropic the money it needs to keep Amazon in the black.
Amazon firing 30,000 workers in the run-up to Christmas is a great milestone in enshittification. America's K-shaped recovery means that nearly all of the consumption is coming from the wealthiest American households, and these households overwhelmingly subscribe to Prime. Prime-subscribing households do not comparison shop. After all, they've already prepaid for a year's shipping in advance. These households start and end nearly every shopping trip in the Amazon app.
If Amazon fires 30,000 workers and tanks its logistics network and e-commerce systems, if it allows itself to drown in spam and scam reviews, if it misses its delivery windows and messes up its returns, that will be our problem, not Amazon's. In a world of commerce where Amazon's predatory pricing, lock-in, and serial acquisitions has left us with few alternatives, Amazon can truly be "too big to care":
From that enviable position, Amazon can afford to enshittify its services in order to sell the big AI lie. Killing 30,000 jobs is a small price to pay if it buys them a few months before a reckoning for its wild AI overspending, keeping the AI grift alive for just a little longer.
If you'd like an essay-formatted version of this post to read or share, here's a link to it on pluralistic.net, my surveillance-free, ad-free, tracker-free blog:
ITHACA and NYC! I'm heading your way for a zillion events from Sept 11-17. Here's a list of open-to-all CORNELL activities including two major keynotes; a movie night with dinner and discussion; and a public event at CORNELL TECH in NYC. I'm also appearing at BUFFALO STREET BOOKS on Sept 11 and at AUTUMN LEAVES BOOKS on Sept 13. I'll also be at the BROOKLYN BOOK FESTIVAL on Sept 21:
The most ENSHITTIFICATION-PROOF way to get the Enshittification audiobook, ebook and hardcover is to pre-order them on my Kickstarter! Help me do AN END RUN around the AMAZON/AUDIBLE AUDIOBOOK MONOPOLY and DISENSHITTIFY your audiobook experience in the process.
My latest Locus column is "Reverse Centaurs," and it sets out to unravel a paradox: how is it that some AI's users describe their experience as a hellish ordeal, while others delight in the ways that AI is changing their lives for the better?
A "centaur" is a human being who is assisted by a machine (a human head on a strong and tireless body). A reverse centaur is a machine that uses a human being as its assistant (a frail and vulnerable person being puppeteered by an uncaring, relentless machine).
Let me give you an example: remember at the start of the summer, when Hearst published a summer reading guide that was full of nonexistent books that had been "hallucinated" by a chatbot?
But in a followup story, Koebler noticed something that the first round of dunks and memes about this poor guy had missed: this same writer had his name on many of these "best of the summer" lists in this supplement. He was practically the sole author of an entire 64-page insert:
And that's where it gets interesting. Koebler got his start in journalism as an intern at the Washington Monthly, where he worked on lists like these:
https://www.404media.co/podcast-ai-slop-summer/
When Koebler was doing this work, he'd be part of a team of three interns, overseen by an experienced journalist, backstopped by an extensive fact-checking department. Those little lists take a surprising amount of work, if you really care about their quality.
The freelance writer who authored this giant summer reading guide with all its lists had been tasked with doing the work of literally dozens of writers, editors and fact-checkers. We don't know whether his boss told him he had to use AI, but there's no way one writer could do all that work without AI.
In other words, that writer's job wasn't to write the article. His job was to be the "human in the loop" for an AI that wrote the articles, but on a schedule and with a workload that precluded his being able to do a good job. It's more true to say that his job was to be the AI's "accountability sink" (in the memorable phrasing of Dan Davies): he was being paid to take the blame for the AI's mistakes.
He was, in other words, a reverse centaur.
Now, I am a freelance writer as well, and not so long ago, I wanted to quote something smart I'd heard on a podcast in an article, but I couldn't remember where I heard it. So I downloaded Whisper, an open source AI transcription model from Openai, to my laptop. I threw the last 30 hours' worth of audio that I'd listened to at it, and worked away on other stuff for an hour or two. When I checked again, I had a folder full of pretty reliable transcripts. I searched the text, found the quote, and opened the audio to the supplied timecode to double-check it. I was a centaur. I got to decide how to use the AI, and I only had to use it in ways that made my work better and more satisfying.
This, I think, is the explanation for the paradox of AI: the AI users who are being immiserated and precaratized by bosses who have been convinced to fire their colleagues and pile their work on the terrorized survivors of the layoffs hate the AI, because it makes their life worse in every way.
Whereas the people who choose when and how to use AI – the centaurs – are only using AI to the extent that it is useful, and throwing it away when it's not. They may make poor choices about the AI, but those choices are theirs, they are not imposed from on high. A bicyclist who chooses to commute on two wheels can have a glorious ride, or they can ride like a maniac and end up eating dirt, but they are having a fundamentally different experience from, say, a gig delivery platform rider who has been given an impossible quota and is having their pay eroded by algorithmic wage discrimination:
I was very happy to put this analysis in the pages of Locus, the trade magazine for the science fiction field. The job of a science fiction writer is only incidentally to describe what a technology does – at its best, science fiction interrogates who the technology does it to and who the technology does it for.
This is a political act of resistance. Margaret Thatcher's motto, after all, was "There is no alternative," by which she meant, "Stop trying to think of alternatives." The bully's trick is to present your defeat as a fait accompli: "Resistance is futile."
Tech bosses practice a form of vulgar Thatcherism all the time: Mark Zuckerberg wants you to think there's no way to talk with your friends without letting him listen in; Sundar Pichai wants you to think there's no way to search the web without being spied on; Tim Cook wants you to think there's no way to have a safe and reliable computing experience without giving him a veto over which software you install; Satya Nadella wants you to think there's no way for you to edit a Word file without letting your boss compare your keystrokes-per-minute to your co-workers:
And AI bosses want you to think that the only way to use these tools is to displace and immiserate labor, because that's the promise they raise investment capital on:
AI is a bubble. If it wasn't a bubble – if it was just a bunch of computer scientists and product teams tinkering with possible uses for advancements in back-propagation, generative adversarial networks and machine learning – there wouldn't be any controversy here. A programmer who uses a chatbot to autogen a bunch of cross-browser CSS stylesheets that mostly work, after some tinkering, would maybe mention that fact over beers – but they wouldn't get sucked into a cult obsessed with outlandish scenarios in which the chatbot wakes up and turns us all into paperclips:
AI is a bubble. Bubbles burst. We're in for a near-total collapse of the AI investment mania. Most of these companies will fail. Many planned data-centers will never be opened. Many existing data-centers will be shuttered. When that happens, what will be left?
AI is a bubble, and when bubbles burst, they sometimes leave behind a productive residue. At home, I enjoy 2GB symmetrical fiber optic internet, because AT&T was able to light up some of the dark fiber that Worldcom fraudulently raised billions for. Worldcom's CEO died in prison after scamming the finances of ordinary people, and the world would be a better place if that had never happened, but there was some productive residue left behind, and many of us are reaping the benefit today:
Contrast that with the cryptocurrency bubble. When that bursts, we'll still have a smattering of programmers who've had a subsidized education in cryptography and secure programming in Rust, but mostly what crypto will leave behind is bad Austrian economics and worse monkey JPEGs. Like Enron, crypto will leave nothing much behind of any value.
All bubbles are bad, but some are more productive than others. When the AI bubble bursts, there will be stellar bargains on GPUs (it would be ironic if scientists snapped them up at pennies on the dollar and used them for climate modeling). We'll have a lot of technical people who are much better at applied statistics than they were a decade ago. And there will be the open source models, like Whisper, the tool I used to transcribe all those podcasts.
These open source models run on commodity hardware, and while the climate costs of creating those models is terrible, they're here now, and operating them isn't especially energy-intensive. When I used Whisper to transcribe 30 hours' worth of podcasts, my laptop's fan didn't even switch on.
What's more, open source hackers are doing amazing things with these tools – far more than the giant corporations that released them ever anticipated. These "toy" models were released as a way to entice programmers into specializing in cloud systems operated by the big tech companies, but it turns out that these standalone models can do amazing things, and aren't just a demo for a big, doomed foundation model:
It doesn't matter what happens to Openai; Whisper is here to stay. It's already being rolled into other standard tools – the latest version of ffmpeg integrates Whisper and can autogen captions:
The things these open source standalone models can do will only expand, and they will become a given for our computing applications. Your computer or phone will be able to transcribe audio and do cool image-editing stuff like erasing strangers from the background of a photo as a standard feature.
That's the good news. The bad news is all the damage the bubble is doing now and all the further damage that will come from its collapse. Today, we're getting the climate impact, obviously, and the immiseration of all those workers who are being reverse-centaured by an AI that can't do their job, but whose manufacturer's salesforce convinced their boss to fire them and replace them with an AI anyway.
After the bubble bursts, there will be the mass incineration of everyday people's retirement savings and the knock-on effects as the whole market craters. And long after that, there will be the terrible impact on our society's ability to do things, as defunct foundation models grind to a halt, after the people they replaced are long gone and can't step in to pick up the work they fumble. We are busily filling the walls of society with digital asbestos and we'll be digging it out for generations to come.
Every day the bubble persists, the harms of today and tomorrow increase. We need to burst that bubble as soon as possible. That's how I came to spend the summer writing a book for Farrar, Straus and Giroux with the working title The Reverse-Centaur's Guide to AI, whose goal is to improve the quality of AI criticism so that it inflicts maximum damage on AI swindlers and their terrible investment bubble.
It'll be out in 2026, but for now, you can have a look at my Locus column:
Click here to pre-order my next book, ENSHITTIFICATION: WHY EVERYTHING SUDDENLY GOT WORSE AND WHAT TO DO ABOUT IT
If you'd like an essay-formatted version of this post to read or share, here's a link to it on pluralistic.net, my surveillance-free, ad-free, tracker-free blog:
The future of Amazon coders is the present of Amazon warehouse workers
I'm on a 20+ city book tour for my new novel PICKS AND SHOVELS. Catch me in BURBANK with WIL WHEATON TONIGHT (Mar 13), and in SAN DIEGO at MYSTERIOUS GALAXY on Mar 24. More tour dates here.
My theory of the "shitty technology adoption curve" holds that you can predict the future impact of abusive technologies on you by observing the way these are deployed against people who have less social power than you:
When you have a new, abusive technology, you can't just aim it at rich, powerful people, because when they complain, they get results. To successfully deploy that abusive tech, you need to work your way up the privilege gradient, starting with people with no power, like prisoners, refugees, and mental patients. This starts the process of normalization, even as it sands down some of the technology's rough edges against their tender bodies. Once that's done, you can move on to people with more social power – immigrants, blue collar workers, school children. Step by step, you normalize and smooth out the abusive tech, until you can apply it to everyone – even rich and powerful people. Think of the deployment of CCTV, facial recognition, location tracking, and web surveillance.
All this means that blue collar workers are the pioneering early adopters of the bossware that will shortly be tormenting their white-collar colleagues elsewhere in the business. It's as William Gibson prophesied: "The future is here, it's just not evenly distributed" (it's pooled up thick and noxious around the ankles of blue-collar workers, refugees, mental patients, etc).
Nowhere is this rule more salient than in Big Tech firms. Tech companies have thoroughly segregated workforces. Delivery drivers, customer service reps, data-labelers, warehouse workers and other "green badge," low-status workers are the testing ground for their employer's own disciplinary technology, which monitors them down to the keystroke, the eye-movement, and the pee break. Meanwhile, the "blue badge" white-collar coders get stock options, gourmet cafeterias, free massages, day care and complimentary egg-freezing so they can delay fertility. Companies like Google not only use separate entrance for their different classes of workers – they stagger their shifts so that the elite workers don't even see their lower-status counterparts.
Importantly, almost none of these workers – whether low-status or high – are unionized. Tech union density is so thin, it's almost nonexistent. It's easy to see why elite tech workers wouldn't bother with unionizing: with such fantastic wages and so many perks, why endure the tedium of meetings and memos? But then there's the rest of the workers, who are subjected to endless "electronic whipping" by bossware and who take home wages that look like pocket change when compared to the tech division's compensation. These workers have every reason to unionize, living as they do in the dystopian future of labor.
At Amazon warehouses, workers are injured at three times the rate of warehouse workers at competing firms. They are penalized for "time off task" (like taking a piss break). They are made to stand in long, humiliating body-search lines when they go on- and off-shift, hours every week, without compensation. Variations on this theme play out in other blue-collar sectors of the Amazon empire, like Amazon delivery drivers and Whole Food shelf-stockers.
Those workers have every reason to unionize, and they have done their damndest, but Amazon has defeated worker union drives, again and again. How does Amazon win these battles? Simple: they cheat. They illegally fire union organizers:
They spend millions on anti-union tech, spying on workers and creating "heatmaps" that let them direct their anti-union efforts to specific stores and facilities:
That's just the tip of the iceberg. A new investigation by Northwestern University's Teke Wiggin draws on worker interviews and FOIA requests to the NLRB to assemble a first-of-its-kind catalog of Amazon's labor-disciplining, union-busting tactics:
Disciplining labor and busting unions go hand in hand. It's a simple equation: the harder it is for your workers to form a union, the worse you can treat them without facing labor reprisals, because individual workers' options are limited to a) quitting or b) sucking it up, while unionized workers can grieve, sue, and strike.
At the core of Amazon's labor discipline technology is "algorithmic management," which is exactly what it sounds like: replacing middle managers with software that counts your keystrokes, watches your eyeballs, or applies a virtual caliper to some other metric to decide whether you're a good worker or a rotten apple:
Automation theory describes two poles of workplace automation: centaurs (in which workers are assisted by technology) and "reverse-centaurs" (in which workers provide assistance to technology):
Amazon is a reverse-centaurism pioneer. Take the delivery drivers whose every maneuver, eyeball movement, and turn signal is analyzed and inevitably, found wanting, as workers seek to satisfy impossible quotas that can't even be met if you pee in a bottle instead of taking toilet breaks:
Then there's the warehouse workers who are also tormented with impossible, pisscall-annihilating quotas. Some of these workers are fitted with haptic wristbands that buzz to tell them they're being too slow at picking up an item and dropping it into a box, pushing them to faster, joint-destroying paces that account for Amazon's enduring position as the most worker-maiming warehouse employer in the nation:
In his paper, Wiggin does important work connecting these "electronic whips" to Amazon's arsenal of traditional union-busting weapons, like "captive audience" meetings where workers are forced to sit through hours of anti-union indoctrination. For Wiggin, bossware tools aren't just a stick to beat workers with – they're also a carrot that can be used to diffuse a worker's outrage ahead of a key union vote.
Algorithmic management isn't just software that wrings more work out of workers – it's software that replaces managers. By surveilling workers – both on the job and in social media spaces (like subreddits) where workers gather to talk, Amazon can tune the "electronic whip," reducing quotas and easing the pace of work so that workers view their jobs more favorably and are more receptive to anti-union propaganda.
This is "twiddling" – exploiting the digital flexibility of a system to "twiddle the knobs" governing its business logic, changing everything from prices to wages, search rankings to recommendations, in realtime, for every customer and worker:
https://pluralistic.net/2023/02/19/twiddler/
Twiddling combines surveillance data with flexible business logic to create an unbeatable house advantage. If you're an Amazon shopper, you get twiddled all the time, as Amazon replaces the best matches for your searches with paid results. If you buy that first product result, you'll pay an average of 29% more than the best match for your search:
Worker-side twiddling is even more dystopian. When a nurse is assigned a shift by an "Uber for nurses" app, the app checks whether the worker has overdue credit card bills, which trigger lower wages (on the theory that an indebted worker is a desperate worker):
When it comes to union-busting, Amazon's found a new use for twiddling: lessening the pace of work, which Wiggin calls "algorithmic slack-cutting." The important thing about algorithmic slack-cutting is that it's only temporary. The algorithm that reduces your work-load in the runup to a union vote can then dial the pace of work up afterward, by small, random increments that are below the threshold at which they register on the human sensory apparatus. They're not so much boiling the frog as poaching it.
Meanwhile, Amazon gets to flood the zone with anti-union messages, including mandatory messages on the app that assigns your shifts – a captive audience meeting in every pocket.
Between social media surveillance and on-the-job surveillance, Amazon has built a powerful training set for algorithms designed to crush workplace democracy. That's how things go for Amazon's warehouse workers and delivery drivers, and the shelf-stockers at Whole Foods.
But of course, the picture is very different for Amazon's techies, who enjoy the industry standard of high wages and lavish perks.
For now.
The tech industry is in the midst of three years' worth of mass layoffs: 260K in 2023, 150k in 2024, tens of thousands this year. None of this is due to a shortfall in profits, mind: Google laid off 12,000 workers just weeks after staging a stock buyback that would have funded their salaries for 27 years. Meta just announced a 5% across-the-board headcount cut and that it was doubling its executive bonuses.
In other words, tech is firing workers not because it must, but because it can. When workers depend on scarcity – instead of unions – as a source of power, they dig their own graves. For well-paid, scarcity-based coders, every new computer science graduate is the enemy, eroding the scarcity that your wages depend on.
Amazon coders get to come to work with pink mohawks, facial piercings, and black t-shirts that say things their bosses don't understand. They get to pee whenever they want to. That's not because Jeff Bezos is sentimentally attached to techies and bears personal animus toward warehouse workers. Jeff Bezos wants to pay his workforce as little as he can. He treats his tech workers with respect because he's afraid of them, because if they quit, he can't replace them, and without their work, he can't make money.
Once there's an army of unemployed coders who'll take your job, Jeff Bezos doesn't have to fear you anymore. He can fire you and replace you the next day.
Bezos is obviously incredibly horny for this. Like most tech bosses, he dreams of a world in which entitled hackers can't call their bosses dumbshits and decline to frog when they shout "jump!" That's why Amazon PR puts so much energy into trumpeting the business's use of AI to replace coders:
It's not just that they're excited about firing coders and saving money – they're even more excited about transforming the job of "Amazon coder," from someone who solves complex technical problems to someone who performs tedious code review on automatically generated code barfed up by a chatbot:
"Code reviewer" is a much less fulfilling job than "programmer." Code reviewers are also easier to replace than programmers. A code reviewer is a reverse-centaur, a servant to the machine. Every time you hear "AI-assisted programmer," you should substitute "programmer-assisted AI."
Programming is even more bossware-ready than working in a warehouse. The machines coders use are much easier to fit with surveillance technology that monitors their performance – and spies on their communications, looking for dissenting chatter – than a warehouse floor. The only thing that stopped Jeff Bezos from treating his programmers like his warehouse workers is their scarcity. That scarcity is now going away.
That's bad news for Amazon customers, too. Tech workers often feel a sense of duty to their users, a "vocational awe" that drives them to put in long hours to make things their users will enjoy. The labor power of tech workers has long served as a check on the impulse to enshittify those products:
As tech workers' power wanes, they don't just lose the ability to protect themselves from their bosses' greediest, most sadistic urges – they also lose the power to defend all of us. Smart tech workers know this. That's why Amazon tech workers walked out in support of Amazon warehouse workers:
Wiggin's report isn't just a snapshot of Amazon warehouse workers' dystopian present – it's a promise of Amazon tech workers' future. The future is here, in Amazon warehouses, and every day, it's getting closer to Amazon's technical offices.
If you'd like an essay-formatted version of this post to read or share, here's a link to it on pluralistic.net, my surveillance-free, ad-free, tracker-free blog:
I'm on a 20+ city book tour for my new novel PICKS AND SHOVELS. Catch me in SAN DIEGO at MYSTERIOUS GALAXY on Mar 24, and in CHICAGO with PETER SAGAL on Apr 2. More tour dates here.
AI can't do your job, but an AI salesman (Elon Musk) can convince your boss (the USA) to fire you and replace you (a federal worker) with a chatbot that can't do your job:
If you pay attention to the hype, you'd think that all the action on "AI" (an incoherent grab-bag of only marginally related technologies) was in generating text and images. Man, is that ever wrong. The AI hype machine could put every commercial illustrator alive on the breadline and the savings wouldn't pay the kombucha budget for the million-dollar-a-year techies who oversaw Dall-E's training run. The commercial market for automated email summaries is likewise infinitesimal.
The fact that CEOs overestimate the size of this market is easy to understand, since "CEO" is the most laptop job of all laptop jobs. Having a chatbot summarize the boss's email is the 2025 equivalent of the 2000s gag about the boss whose secretary printed out the boss's email and put it in his in-tray so he could go over it with a red pen and then dictate his reply.
The smart AI money is long on "decision support," whereby a statistical inference engine suggests to a human being what decision they should make. There's bots that are supposed to diagnose tumors, bots that are supposed to make neutral bail and parole decisions, bots that are supposed to evaluate student essays, resumes and loan applications.
The narrative around these bots is that they are there to help humans. In this story, the hospital buys a radiology bot that offers a second opinion to the human radiologist. If they disagree, the human radiologist takes another look. In this tale, AI is a way for hospitals to make fewer mistakes by spending more money. An AI assisted radiologist is less productive (because they re-run some x-rays to resolve disagreements with the bot) but more accurate.
In automation theory jargon, this radiologist is a "centaur" – a human head grafted onto the tireless, ever-vigilant body of a robot
Of course, no one who invests in an AI company expects this to happen. Instead, they want reverse-centaurs: a human who acts as an assistant to a robot. The real pitch to hospital is, "Fire all but one of your radiologists and then put that poor bastard to work reviewing the judgments our robot makes at machine scale."
No one seriously thinks that the reverse-centaur radiologist will be able to maintain perfect vigilance over long shifts of supervising automated process that rarely go wrong, but when they do, the error must be caught:
This is bad enough when we're talking about radiology, but it's even worse in government contexts, where the bots are deciding who gets Medicare, who gets food stamps, who gets VA benefits, who gets a visa, who gets indicted, who gets bail, and who gets parole.
That's because statistical inference is intrinsically conservative: an AI predicts the future by looking at its data about the past, and when that prediction is also an automated decision, fed to a Chaplinesque reverse-centaur trying to keep pace with a torrent of machine judgments, the prediction becomes a directive, and thus a self-fulfilling prophecy:
AIs want the future to be like the past, and AIs make the future like the past. If the training data is full of human bias, then the predictions will also be full of human bias, and then the outcomes will be full of human bias, and when those outcomes are copraphagically fed back into the training data, you get new, highly concentrated human/machine bias:
By firing skilled human workers and replacing them with spicy autocomplete, Musk is assuming his final form as both the kind of boss who can be conned into replacing you with a defective chatbot and as the fast-talking sales rep who cons your boss. Musk is transforming key government functions into high-speed error-generating machines whose human minders are only the payroll to take the fall for the coming tsunami of robot fuckups.
This is the equivalent to filling the American government's walls with asbestos, turning agencies into hazmat zones that we can't touch without causing thousands to sicken and die:
If you'd like an essay-formatted version of this post to read or share, here's a link to it on pluralistic.net, my surveillance-free, ad-free, tracker-free blog:
“Humans in the loop” must detect the hardest-to-spot errors, at superhuman speed
I'm touring my new, nationally bestselling novel The Bezzle! Catch me SATURDAY (Apr 27) in MARIN COUNTY, then Winnipeg (May 2), Calgary (May 3), Vancouver (May 4), and beyond!
If AI has a future (a big if), it will have to be economically viable. An industry can't spend 1,700% more on Nvidia chips than it earns indefinitely – not even with Nvidia being a principle investor in its largest customers:
https://news.ycombinator.com/item?id=39883571
A company that pays 0.36-1 cents/query for electricity and (scarce, fresh) water can't indefinitely give those queries away by the millions to people who are expected to revise those queries dozens of times before eliciting the perfect botshit rendition of "instructions for removing a grilled cheese sandwich from a VCR in the style of the King James Bible":
Eventually, the industry will have to uncover some mix of applications that will cover its operating costs, if only to keep the lights on in the face of investor disillusionment (this isn't optional – investor disillusionment is an inevitable part of every bubble).
Now, there are lots of low-stakes applications for AI that can run just fine on the current AI technology, despite its many – and seemingly inescapable - errors ("hallucinations"). People who use AI to generate illustrations of their D&D characters engaged in epic adventures from their previous gaming session don't care about the odd extra finger. If the chatbot powering a tourist's automatic text-to-translation-to-speech phone tool gets a few words wrong, it's still much better than the alternative of speaking slowly and loudly in your own language while making emphatic hand-gestures.
There are lots of these applications, and many of the people who benefit from them would doubtless pay something for them. The problem – from an AI company's perspective – is that these aren't just low-stakes, they're also low-value. Their users would pay something for them, but not very much.
For AI to keep its servers on through the coming trough of disillusionment, it will have to locate high-value applications, too. Economically speaking, the function of low-value applications is to soak up excess capacity and produce value at the margins after the high-value applications pay the bills. Low-value applications are a side-dish, like the coach seats on an airplane whose total operating expenses are paid by the business class passengers up front. Without the principle income from high-value applications, the servers shut down, and the low-value applications disappear:
Now, there are lots of high-value applications the AI industry has identified for its products. Broadly speaking, these high-value applications share the same problem: they are all high-stakes, which means they are very sensitive to errors. Mistakes made by apps that produce code, drive cars, or identify cancerous masses on chest X-rays are extremely consequential.
Some businesses may be insensitive to those consequences. Air Canada replaced its human customer service staff with chatbots that just lied to passengers, stealing hundreds of dollars from them in the process. But the process for getting your money back after you are defrauded by Air Canada's chatbot is so onerous that only one passenger has bothered to go through it, spending ten weeks exhausting all of Air Canada's internal review mechanisms before fighting his case for weeks more at the regulator:
There's never just one ant. If this guy was defrauded by an AC chatbot, so were hundreds or thousands of other fliers. Air Canada doesn't have to pay them back. Air Canada is tacitly asserting that, as the country's flagship carrier and near-monopolist, it is too big to fail and too big to jail, which means it's too big to care.
Air Canada shows that for some business customers, AI doesn't need to be able to do a worker's job in order to be a smart purchase: a chatbot can replace a worker, fail to their worker's job, and still save the company money on balance.
I can't predict whether the world's sociopathic monopolists are numerous and powerful enough to keep the lights on for AI companies through leases for automation systems that let them commit consequence-free free fraud by replacing workers with chatbots that serve as moral crumple-zones for furious customers:
But even stipulating that this is sufficient, it's intrinsically unstable. Anything that can't go on forever eventually stops, and the mass replacement of humans with high-speed fraud software seems likely to stoke the already blazing furnace of modern antitrust:
Of course, the AI companies have their own answer to this conundrum. A high-stakes/high-value customer can still fire workers and replace them with AI – they just need to hire fewer, cheaper workers to supervise the AI and monitor it for "hallucinations." This is called the "human in the loop" solution.
The human in the loop story has some glaring holes. From a worker's perspective, serving as the human in the loop in a scheme that cuts wage bills through AI is a nightmare – the worst possible kind of automation.
Let's pause for a little detour through automation theory here. Automation can augment a worker. We can call this a "centaur" – the worker offloads a repetitive task, or one that requires a high degree of vigilance, or (worst of all) both. They're a human head on a robot body (hence "centaur"). Think of the sensor/vision system in your car that beeps if you activate your turn-signal while a car is in your blind spot. You're in charge, but you're getting a second opinion from the robot.
Likewise, consider an AI tool that double-checks a radiologist's diagnosis of your chest X-ray and suggests a second look when its assessment doesn't match the radiologist's. Again, the human is in charge, but the robot is serving as a backstop and helpmeet, using its inexhaustible robotic vigilance to augment human skill.
That's centaurs. They're the good automation. Then there's the bad automation: the reverse-centaur, when the human is used to augment the robot.
Amazon warehouse pickers stand in one place while robotic shelving units trundle up to them at speed; then, the haptic bracelets shackled around their wrists buzz at them, directing them pick up specific items and move them to a basket, while a third automation system penalizes them for taking toilet breaks or even just walking around and shaking out their limbs to avoid a repetitive strain injury. This is a robotic head using a human body – and destroying it in the process.
An AI-assisted radiologist processes fewer chest X-rays every day, costing their employer more, on top of the cost of the AI. That's not what AI companies are selling. They're offering hospitals the power to create reverse centaurs: radiologist-assisted AIs. That's what "human in the loop" means.
This is a problem for workers, but it's also a problem for their bosses (assuming those bosses actually care about correcting AI hallucinations, rather than providing a figleaf that lets them commit fraud or kill people and shift the blame to an unpunishable AI).
Humans are good at a lot of things, but they're not good at eternal, perfect vigilance. Writing code is hard, but performing code-review (where you check someone else's code for errors) is much harder – and it gets even harder if the code you're reviewing is usually fine, because this requires that you maintain your vigilance for something that only occurs at rare and unpredictable intervals:
But for a coding shop to make the cost of an AI pencil out, the human in the loop needs to be able to process a lot of AI-generated code. Replacing a human with an AI doesn't produce any savings if you need to hire two more humans to take turns doing close reads of the AI's code.
This is the fatal flaw in robo-taxi schemes. The "human in the loop" who is supposed to keep the murderbot from smashing into other cars, steering into oncoming traffic, or running down pedestrians isn't a driver, they're a driving instructor. This is a much harder job than being a driver, even when the student driver you're monitoring is a human, making human mistakes at human speed. It's even harder when the student driver is a robot, making errors at computer speed:
This is why the doomed robo-taxi company Cruise had to deploy 1.5 skilled, high-paid human monitors to oversee each of its murderbots, while traditional taxis operate at a fraction of the cost with a single, precaratized, low-paid human driver:
The vigilance problem is pretty fatal for the human-in-the-loop gambit, but there's another problem that is, if anything, even more fatal: the kinds of errors that AIs make.
Foundationally, AI is applied statistics. An AI company trains its AI by feeding it a lot of data about the real world. The program processes this data, looking for statistical correlations in that data, and makes a model of the world based on those correlations. A chatbot is a next-word-guessing program, and an AI "art" generator is a next-pixel-guessing program. They're drawing on billions of documents to find the most statistically likely way of finishing a sentence or a line of pixels in a bitmap:
https://dl.acm.org/doi/10.1145/3442188.3445922
This means that AI doesn't just make errors – it makes subtle errors, the kinds of errors that are the hardest for a human in the loop to spot, because they are the most statistically probable ways of being wrong. Sure, we notice the gross errors in AI output, like confidently claiming that a living human is dead:
But the most common errors that AIs make are the ones we don't notice, because they're perfectly camouflaged as the truth. Think of the recurring AI programming error that inserts a call to a nonexistent library called "huggingface-cli," which is what the library would be called if developers reliably followed naming conventions. But due to a human inconsistency, the real library has a slightly different name. The fact that AIs repeatedly inserted references to the nonexistent library opened up a vulnerability – a security researcher created a (inert) malicious library with that name and tricked numerous companies into compiling it into their code because their human reviewers missed the chatbot's (statistically indistinguishable from the the truth) lie:
For a driving instructor or a code reviewer overseeing a human subject, the majority of errors are comparatively easy to spot, because they're the kinds of errors that lead to inconsistent library naming – places where a human behaved erratically or irregularly. But when reality is irregular or erratic, the AI will make errors by presuming that things are statistically normal.
These are the hardest kinds of errors to spot. They couldn't be harder for a human to detect if they were specifically designed to go undetected. The human in the loop isn't just being asked to spot mistakes – they're being actively deceived. The AI isn't merely wrong, it's constructing a subtle "what's wrong with this picture"-style puzzle. Not just one such puzzle, either: millions of them, at speed, which must be solved by the human in the loop, who must remain perfectly vigilant for things that are, by definition, almost totally unnoticeable.
This is a special new torment for reverse centaurs – and a significant problem for AI companies hoping to accumulate and keep enough high-value, high-stakes customers on their books to weather the coming trough of disillusionment.
This is pretty grim, but it gets grimmer. AI companies have argued that they have a third line of business, a way to make money for their customers beyond automation's gifts to their payrolls: they claim that they can perform difficult scientific tasks at superhuman speed, producing billion-dollar insights (new materials, new drugs, new proteins) at unimaginable speed.
However, these claims – credulously amplified by the non-technical press – keep on shattering when they are tested by experts who understand the esoteric domains in which AI is said to have an unbeatable advantage. For example, Google claimed that its Deepmind AI had discovered "millions of new materials," "equivalent to nearly 800 years’ worth of knowledge," constituting "an order-of-magnitude expansion in stable materials known to humanity":
It was a hoax. When independent material scientists reviewed representative samples of these "new materials," they concluded that "no new materials have been discovered" and that not one of these materials was "credible, useful and novel":
As Brian Merchant writes, AI claims are eerily similar to "smoke and mirrors" – the dazzling reality-distortion field thrown up by 17th century magic lantern technology, which millions of people ascribed wild capabilities to, thanks to the outlandish claims of the technology's promoters:
The fact that we have a four-hundred-year-old name for this phenomenon, and yet we're still falling prey to it is frankly a little depressing. And, unlucky for us, it turns out that AI therapybots can't help us with this – rather, they're apt to literally convince us to kill ourselves:
If you'd like an essay-formatted version of this post to read or share, here's a link to it on pluralistic.net, my surveillance-free, ad-free, tracker-free blog:
Support me this summer on the Clarion Write-A-Thon and help raise money for the Clarion Science Fiction and Fantasy Writers' Workshop!
When I took my kid to New Zealand with me on a book-tour, I was delighted to learn that grocery stores had special aisles where all the kids'-eye-level candy had been removed, to minimize nagging. What a great idea!
Related: countries around the world limit advertising to children, for two reasons:
1) Kids may not be stupid, but they are inexperienced, and that makes them gullible; and
2) Kids don't have money of their own, so their path to getting the stuff they see in ads is nagging their parents, which creates a natural constituency to support limits on kids' advertising (nagged parents).
There's something especially annoying about ads targeted at getting credulous people to coerce or torment other people on behalf of the advertiser. For example, AI companies spent millions targeting your boss in an effort to convince them that you can be replaced with a chatbot that absolutely, positively cannot do your job.
Your boss has no idea what your job entails, and is (not so) secretly convinced that you're a featherbedding parasite who only shows up for work because you fear the breadline, and not because your job is a) challenging, or b) rewarding:
That makes them prime marks for chatbot-peddling AI pitchmen. Your boss would love to fire you and replace you with a chatbot. Chatbots don't unionize, they don't backtalk about stupid orders, and they don't experience any inconvenient moral injury when ordered to enshittify the product:
Bosses are Bizarro-world Marxists. Like Marxists, your boss's worldview is organized around the principle that every dollar you take home in wages is a dollar that isn't available for executive bonuses, stock buybacks or dividends. That's why you boss is insatiably horny for firing you and replacing you with software. Software is cheaper, and it doesn't advocate for higher wages.
That makes your boss such an easy mark for AI pitchmen, which explains the vast gap between the valuation of AI companies and the utility of AI to the customers that buy those companies' products. As an investor, buying shares in AI might represent a bet the usefulness of AI – but for many of those investors, backing an AI company is actually a bet on your boss's credulity and contempt for you and your job.
But bosses' resemblance to toddlers doesn't end with their credulity. A toddler's path to getting that eye-height candy-bar goes through their exhausted parents. Your boss's path to realizing the productivity gains promised by an AI salesman runs through you.
A new research report from the Upwork Research Institute offers a look into the bizarre situation unfolding in workplaces where bosses have been conned into buying AI and now face the challenge of getting it to work as advertised:
96% of bosses expect that AI will make their workers more productive;
85% of companies are either requiring or strongly encouraging workers to use AI;
49% of workers have no idea how AI is supposed to increase their productivity;
77% of workers say using AI decreases their productivity.
Working at an AI-equipped workplaces is like being the parent of a furious toddler who has bought a million Sea Monkey farms off the back page of a comic book, and is now destroying your life with demands that you figure out how to get the brine shrimp he ordered from a notorious Holocaust denier to wear little crowns like they do in the ad:
Bosses spend a lot of time thinking about your productivity. The "productivity paradox" shows a rapid, persistent decline in American worker productivity, starting in the 1970s and continuing to this day:
The "paradox" refers to the growth of IT, which is sold as a productivity-increasing miracle. There are many theories to explain this paradox. One especially good theory came from the late David Graeber (rest in power), in his 2012 essay, "Of Flying Cars and the Declining Rate of Profit":
Graeber proposes that the growth of IT was part of a wider shift in research approaches. Research was once dominated by weirdos (e.g. Jack Parsons, Oppenheimer, etc) who operated with relatively little red tape. The rise of IT coincides with the rise of "managerialism," the McKinseyoid drive to monitor, quantify and – above all – discipline the workforce. IT made it easier to generate these records, which also made it normal to expect these records.
Before long, every employee – including the "creatives" whose ideas were credited with the productivity gains of the American century until the 70s – was spending a huge amount of time (sometimes the majority of their working days) filling in forms, documenting their work, and generally producing a legible account of their day's work. All this data gave rise to a ballooning class of managers, who colonized every kind of institution – not just corporations, but also universities and government agencies, which were structured to resemble corporations (down to referring to voters or students as "customers").
Even if you think all that record-keeping might be useful, there's no denying that the more time you spend documenting your work, the less time you have to do your work. The solution to this was inevitably more IT, sold as a way to make the record-keeping easier. But adding IT to a bureaucracy is like adding lanes to a highway: the easier it is to demand fine-grained record-keeping, the more record-keeping will be demanded of you.
But that's not all that IT did for the workplace. There are a couple areas in which IT absolutely increased the profitability of the companies that invested in it.
First, IT allowed corporations to outsource production to low-waged countries in the global south, usually places with worse labor protection, weaker environmental laws, and easily bribed regulators. It's really hard to produce things in factories thousands of miles away, or to oversee remote workers in another country. But IT makes it possible to annihilate distance, time zone gaps, and language barriers. Corporations that figured out how to use IT to fire workers at home and exploit workers and despoil the environment in distant lands thrived. Executives who oversaw these projects rose through the ranks. For example, Tim Cook became the CEO of Apple thanks to his successes in moving production out of the USA and into China.
https://archive.is/M17qq
Outsourcing provided a sugar high that compensated for declining productivity…for a while. But eventually, all the gains to be had from outsourcing were realized, and companies needed a new source of cheap gains. That's where "bossware" came in: the automation of workforce monitoring and discipline. Bossware made it possible to monitor workers at the finest-grained levels, measuring everything from keystrokes to eyeball movements.
What's more, the declining power of the American worker – a nice bonus of the project to fire huge numbers of workers and ship their jobs overseas, which made the remainder terrified of losing their jobs and thus willing to eat a rasher of shit and ask for seconds – meant that bossware could be used to tie wages to metrics. It's not just gig workers who don't score consistent five star ratings from app users whose pay gets docked – it's also creative workers whose Youtube and Tiktok wages are cut for violating rules that they aren't allowed to know, because that might help them break the rules without being detected and punished:
Bossware dominates workplaces from public schools to hospitals, restaurants to call centers, and extends to your home and car, if you're working from home (AKA "living at work") or driving for Uber or Amazon:
One way to think about how this works is through the automation-theory metaphor of a "centaur" and a "reverse centaur." In automation circles, a "centaur" is someone who is assisted by an automation tool – for example, when your boss uses AI to monitor your eyeballs in order to find excuses to steal your wages, they are a centaur, a human head atop a machine body that does all the hard work, far in excess of any human's capacity.
A "reverse centaur" is a worker who acts as an assistant to an automation system. The worker who is ridden by an AI that monitors their eyeballs, bathroom breaks, and keystrokes is a reverse centaur, being used (and eventually, used up) by a machine to perform the tasks that the machine can't perform unassisted:
But there's only so much work you can squeeze out of a human in this fashion before they are ruined for the job. Amazon's internal research reveals that the company has calculated that it ruins workers so quickly that it is in danger of using up every able-bodied worker in America:
Which explains the other major findings from the Upwork study:
81% of bosses have increased the demands they make on their workers over the past year; and
71% of workers are "burned out."
Bosses' answer to "AI making workers feel burned out" is the same as "IT-driven form-filling makes workers unproductive" – do more of the same, but go harder. Cisco has a new product that tries to detect when workers are about to snap after absorbing abuse from furious customers and then gives them a "Zen" moment in which they are showed a "soothing" photo of their family:
This is just the latest in a series of increasingly sweaty and cruel "workplace wellness" technologies that spy on workers and try to help them "manage their stress," all of which have the (totally predictable) effect of increasing workplace stress:
The only person who wouldn't predict that being closely monitored by an AI that snitches on you to your boss would increase your stress levels is your boss. Unfortunately for you, AI pitchmen know this, too, and they're more than happy to sell your boss the reverse-centaur automation tool that makes you want to die, and then sell your boss another automation tool that is supposed to restore your will to live.
The "productivity paradox" is being resolved before our eyes. American per-worker productivity fell because it was more profitable to ship American jobs to regulatory free-fire zones and exploit the resulting precarity to abuse the workers left onshore. Workers who resented this arrangement were condemned for having a shitty "work ethic" – even as the number of hours worked by the average US worker rose by 13% between 1976 and 2016:
AI is just a successor gimmick at the terminal end of 40 years of increasing profits by taking them out of workers' hides rather than improving efficiency. That arrangement didn't come out of nowhere: it was a direct result of a Reagan-era theory of corporate power called "consumer welfare." Under the "consumer welfare" approach to antitrust, monopolies were encouraged, provided that they used their market power to lower wages and screw suppliers, while lowering costs to consumers.
"Consumer welfare" supposed that we could somehow separate our identities as "workers" from our identities as "shoppers" – that our stagnating wages and worsening conditions ceased mattering to us when we clocked out at 5PM (or, you know, 9PM) and bought a $0.99 Meal Deal at McDonald's whose low, low price was only possible because it was cooked by someone sleeping in their car and collecting food-stamps.
But we're reaching the end of the road for consumer welfare. Sure, your toddler-boss can be tricked into buying AI and firing half of your co-workers and demanding that the remainder use AI to do their jobs. But if AI can't do their jobs (it can't), no amount of demanding that you figure out how to make the Sea Monkeys act like they did in the comic-book ad is doing to make that work.
As screwing workers and suppliers produces fewer and fewer gains, companies are increasingly turning on their customers. It's not just that you're getting worse service from chatbots or the humans who are reverse-centaured into their workflow. You're also paying more for that, as algorithmic surveillance pricing uses automation to gouge you on prices in realtime:
This is – in the memorable phrase of David Dayen and Lindsay Owens, the "age of recoupment," in which companies end their practice of splitting the gains from suppressing labor with their customers:
It's a bet that the tolerance for monopolies made these companies too big to fail, and that means they're too big to jail, so they can cheat their customers as well as their workers.
AI may be a bet that your boss can be suckered into buying a chatbot that can't do your job, but investors are souring on that bet. Goldman Sachs, who once trumpeted AI as a multi-trillion dollar sector with unlimited growth, is now publishing reports describing how companies who buy AI can't figure out what to do with it:
Fine, investment banks are supposed to be a little conservative. But VCs? They're the ones with all the appetite for risk, right? Well, maybe so, but Sequoia Capital, a top-tier Silicon Valley VC, is also publicly questioning whether anyone will make AI investments pay off:
I can't tell you how great it was to take my kid down a grocery checkout aisle from which all the eye-level candy had been removed. Alas, I can't figure out how we keep the nation's executive toddlers from being dazzled by shiny AI pitches that leave us stuck with the consequences of their impulse purchases.
If you'd like an essay-formatted version of this post to read or share, here's a link to it on pluralistic.net, my surveillance-free, ad-free, tracker-free blog:
