If you'd like an essay-formatted version of this post to read or share, here's a link to it on pluralistic.net, my surveillance-free, ad-free, tracker-free blog:
One of the best ways to evaluate your own understanding of a subject is to attempt to explain it to someone else. Through explaining things, we discover how much of the "totally obvious" world is actually full of ambiguity, mystery and contradiction.
There's a great bit in Rowan Atkinson's historical sitcom Blackadder that illustrates this principle. In "Ink and Incapability" Blackadder and friends have accidentally burned the only copy of Samuel Johnson's original dictionary of the English language. To cover up their mistake, they decide that they will recreate the dictionary themselves. However, they founder on the first word they try to define, "A":
Blackadder: Let's start at the beginning, shall we? First: 'A.' How would you define 'A'?
Prince George: Ohh…'A' (continues this in background). Oh, I love this! I love this! Quizzies! Erm, hang on, it’s coming. Ooh, crikey, erm, oh yes, I’ve got it!
B: What?
PG: Well, it doesn’t really mean anything, does it?
B: Good. So we're well on the way, then. "'A'; impersonal pronoun; doesn't really mean anything."
I mean, what does "A" mean? The Oxford English Dictionary has more than a dozen definitions, and just the first one runs to more than 1,500 words:
Now, normal life involves a lot of explaining things to other people. You have to explain your problems to customer service reps, who have to explain why they can't solve those problems to you. You need to explain to your loved ones why you want to leave your toothbrush in the shower, and they have to explain why they hate having your toothbrush in the shower. These explanation-exchanges teach you as much as they teach the person you're locked in dialog with. The reasons for leaving your toothbrush in the shower may seem totally obvious to you, and your partner's inability to understand this reveals the assumptions you've never even considered.
For the past four decades, an increasing proportion of the population have spent an increasing proportion of their lives explaining things to machines that have no assumptions or shared context: computers. What we call "programming a computer" is really "breaking down a thing that seems obvious to you into increasingly simple instructions that will be followed to the letter."
Computers are like the genies of legend, bloody-minded literalists who will do exactly what you say, in the way that is perversely furthest from what you mean. To get a computer to do anything, you must first understand it to a degree that far exceeds the understanding needed to explain something to any other human, even a small child.
To take just one example: yesterday, I was on a plane, and the seatback video started cycling through its video-on-demand offerings. All of the movie titles that began with "the" were rewritten to put "the" at the end of the title (for example, "The Sting" was written as "Sting, The"). It's obvious why the system's designer had done this: we expect to find movies whose titles begin with "The" alphabetized under their second word ("The Sting" should appear between "Star Wars" and "Story of a Love Affair"; not between "The Godfather" and "The Untouchables").
I remember when I learned this from my elementary school's teacher-librarian, when I was seven and my class got a tutorial on the school library's card catalog. The librarian explained this principle to us in a matter of minutes, as part of a longer set of instructions, and still, it stuck with me forever.
But here we are, 48 years later, and we still haven't standardized a way to get computers to grasp this foundational principle of alphabetization. Many different databases handle this, to be sure, but it's so inconsistent across so many platforms that someone at the head-end of the video distribution system that feeds American Airlines' VOD system decided, "Fuck it, I'm just gonna put the 'The' at the end of these titles."
Computers are stupid, in other words, which means that the people who program them have to have smarts enough for both of them. Unfortunately for our entire species and civilization, the software industry has historically valued skill at writing efficient and reliable software over writing software that adequately reflects reality. There is an entire genre of lists that illustrate the problem with this; the "falsehoods programmers believe" lists:
https://github.com/kdeldycke/awesome-falsehood
From "names of people" and "street addresses"; from "prices" to "time"; from "email addresses" to "phone numbers"; the "awesome falsehoods" lists are awesome because they reveal how much subtlety and complexity is lurking in these seemingly simple and intuitive concepts. This subtlety and complexity might never emerge through the process of trying to teach a person about them, but when you try to teach a computer about them, you have to confront them in all their awesome fuggliness.
That's because humans have context, agency and flexibility. Sure, the person who designs a form with a blank for "name" might never have met a Malagasy person whose first name is Randriamananjararadofabesata, but in the pre-digital world, when Madagascar Slim met a public official who had to transcribe his name onto a paper form, that official could simply draw an arrow in the margin next to the "name" blank, turn the form over, and write out all 28 characters on the reverse:
https://en.wikipedia.org/wiki/Madagascar_Slim
Computers can't do this. If the programmer doesn't know about Malagasy first names, the computer doesn't know about them either, and the only person who can "teach" the computer about these names is a programmer with access to the code for the database, who has to manually alter the code, compile it, and distribute it to everyone who uses it.
This is partly why digitization has been accompanied by a rise in people asserting that they exist on spectrums rather than in binaries. There were always people whose names, genders, races, and other biographic "immutables" changed, or failed to fit within the blanks on the forms. When those people's realities ran up against failures in the system's abstractions, they could petition a bureaucrat to turn the paper over and write an explanatory note, or to write really small to fill in a blank:
Getting a human official to turn the paper over and write something that didn't fit in the blank is a personal challenge. It requires that a subject convince the person who controls the form to make an exception. This isn't always easy, but officials on the front lines necessarily deal with reality, and they can't get their jobs done unless they're capable of interpreting the necessarily incomplete procedures they operate under to fit things as they really are.
But a computer doesn't have any agency or context or flexibility. If the computer says your name isn't valid, you can't argue the computer into accepting it. The only way to get a digital world to acknowledge your existence is to campaign for systemic change. A trans person might (with great difficulty, to be sure) convince the regional registrar to white-out an old X on one "gender" box and mark a new X in the other box. But the only way to make that change in a software system that has been programmed to treat the "gender" field as immutable is to change society itself.
In this way, computers are machines for teaching us what we don't know about ourselves. They require that we interrogate and faithfully recreate our personal tacit knowledge, and they require that our societies interrogate their tacit presumptions as well. When you are forced to turn your tacit knowledge into explicit knowledge, you're also forced to confront how many broken assumptions lurk inside your reasoning. At best, it's a clarifying process.
Computers don't just clarify what we know and how we organize our society: they also clarify what we are. There are lots of things that we have supposed that a computer would never do, because we believed that these things required something that only humans could do.
Take chess: there are more possible chess games than there are hydrogen atoms in the universe, so brute-forcing chess by running all possible games is a technological impossibility. The best human chess players do something we don't quite understand, mixing their recollections of previous games with rules-of-thumb about the best strategies, with "creativity" (whatever that is) that lets them spontaneously develop new strategies. We can easily get a computer to memorize all the known-good chess sequences and all the rules of thumb, but we don't know what "creativity" is, so we can't encode it as a series of instructions.
But thanks to breakthroughs in machine learning and its successor, "deep learning," we have created chess-playing software that can beat every human, partly by assaying gambits that we would term "creative" if they originated with a human player.
What we make of this new fact is controversial. For many people (myself included), this is a refinement: it tells me that behaviors that are indistinguishable from "creativity" can, at least some of the time, be created by mechanical processes, and the mere fact that a machine does something that appears "creative" doesn't mean that machines are human.
For others, the fact that a mechanical system can evince a behavior that we would call "creative" in a human doesn't mean that we defined "creativity" too broadly, it means that we defined "human" too narrowly, and now we have made a machine that is, at least partially, a person.
I think this is the wrong conclusion to draw, for reasons that Ted Chiang sets out with luminous brilliance in a recent Atlantic article entitled "No, Artificial Intelligence Is Not Conscious":
(If you're hitting the paywall on that one and you're on Firefox, you can try my favorite trick: switch to "Reader Mode" and hit "reload" – your mileage may vary.)
For all the reasons Chiang articulates, I think that drawing the "personhood" line to include machines is a technical mistake, but it's worse than that. Admitting machines to the "personhood" club is a tactical mistake, on par with the mistake we made when we admitted corporations to the personhood club. We should absolutely consider expanding personhood to incorporate living things, including animals and ecosystems, but at the same time, we must purge these dead, artificial constructs from the club:
There is a way in which the recognition of new capabilities in machines parallels the recognition of new capabilities in animals other than ourselves. When those animals manage to do things that we once thought were the exclusive province of humans, we (should) take that as an opportunity to refine our conception of humanity. We're not "the animals that use tools" or "the animals that make plans" or "the animals that recognize themselves in mirrors," because there are other animals that do those things. We are an "animal that uses tools"; not the animal that does so.
Likewise, if we thought that some activity was unique to humans, or to living beings, and we manage to get a machine to replicate that activity, we should revise our view of the activity – not our view of the machine. Creative breakthroughs in chess are not "a thing that requires a human mind," they're "things that can be done by human minds and by machines."
Edsger Dijkstra once famously asked "can a submarine swim?"
Submarines and fish and humans and dolphins all propel themselves through water by different means. But when an animal swims, it does something that is different from what a submarine does. The submarine has no intention, while (complex multicellular) animals swim to pursue goals. Building machines that propel themselves through water is very useful, but it's not the same thing as creating life. In some ways, it's better than creating life: for one thing, we owe other living things moral consideration that is not due to machines. Harnessing a machine to accomplish our own goals is more morally clear than controlling living things to achieve those goals. By the same token, creating machines that can do some of the tasks that we ask of other humans can be the superior moral course. I'd rather have a machine remove mines from a minefield than getting humans to do it.
But beyond this moral relief, creating machines is a fantastic way to learn more about ourselves – making explicit our tacit knowledge, our implicit social assumptions, and the limitations of our conception of what sets us apart from the rest of the universe.
One way in which AI is exceptional is in how it undermines this principle. Conventional software techniques struggled to produce a program that could identify objects in photographs. It turns out that defining all the visual correlates of "cat" is even harder than defining the letter "A." Deep learning techniques solved this previous insoluble problem by relieving us of the job of making explicit all the implicit factors that we deploy when distinguishing an image of a "cat" from an image of a "dog" or a "tiger" (or a "tractor").
Instead of forcing humans to engage in introspection until we'd made a list of every factor we use to identify cat pictures, we simply identified pictures of cats and fed them to a program that tried to find the commonalities among them. The more pictures we fed to that program, the better it got at identifying cats. Today, we have programs that can reliably distinguish an image of a cat from an image of a tiger cub!
This represents a major breakthrough in the power of computers to perform useful work for us, but it's also a huge regression in computers' role in forcing us to make our tacit thought processes explicit through systematic introspection. That's probably fine: we didn't create computers to make us introspect, we created them to do useful work for us. All things considered, it might be better to have genies who grant our wishes according to the spirit of our words, not their letter.
AI may not force us to render our implicit thoughts as explicit instructions, but it absolutely forces us to reconsider and narrow the realm of the numinous. Our own creativity is still delightful and important, but the fact that this squishy, amazing process can (sometimes) be replicated by procedural machines changes the definition of living things. We're "a thing that can produce creative outcomes" but not "the things that can produce creative outcomes." The machines aren't being creative (any more than a submarine is swimming) but they're outputting things that we used to only achieve by means of creativity.
An AI that does something that used to require creativity is fulfilling my favorite of Brian Eno and Peter Schmidt's Oblique Strategies: "Be the first person to not do something that no one else has not done before":
https://stoney.sb.org/eno/oblique.html
Just as bosses fantasize about AI bringing about a worksite without workers, and Zuckerberg is trying to build social media without socializing, and politicians want a bureaucracy without bureaucrats, we can sometimes use AI to produce creative outcomes without creativity:
But art isn't the only realm that we apply creativity to. There are plenty of outcomes that we've always believed we couldn't bring about without applying creativity. AI – like all software – is making us realize that an ingredient we once deemed uniquely essential turns out to have substitutes. AI can sometimes accomplish things without us explaining how we do them. That relieves us of a useful but difficult chore – but in so doing, it forces us (yet again!) to revisit what sorts of things are needed to do the things that matter to us, and therefore, what makes us special.
I used to like it in science fiction when a robot or a computer debatably had "consciousness." I frequently grew fond of that robot or character. But none of those stories involved megacorps manipulating people to think the robot was conscious so they could make more money and strip the earth of resources.
I think arguing that AI is conscious, or soon to become so, reflects levels of addiction that should concern everyone around that person.
The Atlantic piece linked makes a ton of good points - some of my favorites:
The part about needing other external circumstantial evidence to believe something resonated strongly with me, partly because it echoed my legal training, but also just in general.
The point about responsibility for one's actions is also excellent, and one I think more than half of Silicon Valley and all the richest people could stand a refresher course on.
Final note: using historical figure RP to make the point is hilarious and apt.
Quinn Slobodian and Ben Tarnoff’s “Muskism: A Guide for the Perplexed”
If you'd like an essay-formatted version of this post to read or share, here's a link to it on pluralistic.net, my surveillance-free, ad-free, tracker-free blog:
Quinn Slobodian and Ben Tarnoff's Muskism: A Guide for the Perplexed seeks to describe the ideology that gave rise to Elon Musk, the social forces that gave rise to that ideology, and the terrible future that ideology seeks to bring about:
The book's starting point is that "Muskism" isn't merely the things Musk says, believes and does. It's the ideology that coalesces around him, from the people in his wake and the people he follows. Just as Henry Ford neither defined "Fordism" nor precisely practiced it, "Muskism" is centered on Elon Musk, but it's not Elon Musk's creation.
So what is Muskism? To answer this question, Slobodian and Tarnoff enumerate the factors and influences that produced Musk himself. There's apartheid, with its "rational" system of technocratic authoritarianism, which blended together a life of luxury and plenty (for white settlers), brutal surveillance and state violence (for the Black majority) and fascist control over speech (for everyone), combined with a meat-grinder draft that saw young men of Musk's age being called up to suppress liberation uprisings.
Peak apartheid coincided with peak personal computing, the moment where PCs (and then, modems) were getting cheaper and faster, propagating like mushrooms, offering a young Musk access to a broad world outside of the fascist bubble of South Africa, inspiring global ambitions in Musk.
Closer to home, there's Musk's family: his grandfather, a grandiose and vicious white supremacist who moved to South Africa from Canada because of his love for apartheid and racial hierarchy. There's Musk's father, a violent and abusive fool.
Muskism is also a new variant on techno-libertarianism. Traditional techno-libertarianism seeks to dismantle the state – or better yet, exit from the state, in the manner of an Ayn Rand hero. Techno-libertarianism is intimately bound up with settler colonialism, ever on the hunt for an "empty land" (terra nullius) that can be settled without committing the original sin of expropriation, the gravest offense in a religion organized around the total sanctity of private property:
Muskism doesn't seek to exit the state, it seeks to colonize and control it. Long before DOGE, Musk was playing the organs of the state to his own tune, securing massive contracts and subsidies for his solar and rocketry businesses, relying on the massive, deep-pocketed government to keep his businesses afloat.
Obviously (DOGE!), Muskism also seeks to dismantle the state, but only the parts of it that can be transferred to Musk's own private hands. Muskism is about big government…for Musk, but not for you. It embodies that important conservative value summarized in Wilhoit's Law:
There must be in-groups whom the law protects but does not bind, alongside out-groups whom the law binds but does not protect.
This is Musk through and through – a man who demands the right to call innocent strangers "pedo guy" without legal consequence; and also wields the power of the state to shutter businesses that boycott his platform because of its shitty practices:
Musk grew up on science fiction novels and weaves stfnal tropes through his offerings (for example, calling his chatbot "Grok"). There's no shortage of reactionary politics in science fiction, but Musk doesn't confine his sf-inspired cosmology to reactionary literature. He's famously very fond of the Wachowskis' "Matrix" movies, and leans heavily into the metaphor of the Matrix in explaining his interest in wiring people directly into computers, in characterizing opposing political beliefs as "mind viruses," and in calling his political enemies "NPCs":
But Musk's relationship to this metaphor differs in a subtle and important way from the right's "Red Pill" rhetoric. Musk doesn't want to break out of the Matrix – he wants to control the Matrix. He wants to decide which opinions you're allowed to see and discuss (because "most people have weak firewalls for bad ideas"), he wants to beam ideas directly into your neural link, and he wants to abolish any form of workplace democracy, conquering the world with South African baasskap (boss-ism):
https://en.wikipedia.org/wiki/Baasskap
Throughout this slim volume, Slobodian and Tarnoff tease these strains of thought out of Musk's deeds and utterances, and in the systems that he has built or colonized through acquisition. The authors are offering more than a psychoanalysis, though – they're surfacing the material basis for Muskism, the benefits it delivers to its adherents, and the victories it has racked up.
They reveal the method in Musk's chaotic and bullying management style, and recount the times Musk has successfully shattered sclerotic processes to make real breakthroughs, especially in aerospace. You'd be hard pressed to read these passages and without feeling some grudging admiration.
Muskism gets stuff done…sometimes. At a cost. A high cost. Tarnoff and Slobodian count that cost, identify who pays it, and conjure up the world in which those costs continue to mount for all of us.
It's a chilling vision, a Torment Nexus dystopia run by someone who thinks cyberpunk was a suggestion, not a warning.
It's not just him, it's every single one of the too-rich motherfuckers who think they are destined to rule over us. Theil...how the fuck do you read Lord Of The Rings and end up saying "Y'know, that Sauron guy really had the right idea"???
These are damaged, sick-minded men who are all the product of getting their grubby mitts on too much money at too young an age. Overblown, never-tested egos. Never told "no"...never told they couldn't.
The hardest part of being as old as I am (67) is watching everything I was taught was true, right, and good as a child be dismantled by immature freakboys who have no right to be where they are, other than by birth into wealth and privilege.
dededos
theartofmadeline
turnnoffyourmind
janainaart
stormborns
origamiaround
ellievsbear
