The bureaucratic AI arms-race is mutually assured destruction
If you'd like an essay-formatted version of this post to read or share, here's a link to it on pluralistic.net, my surveillance-free, ad-free, tracker-free blog:
According to an Economist editorial, "AI is breaking the British state" by making it too easy to file complaints, demands and appeals, which will "drown the state" with "demands as well-crafted as a first-class lawyer's":
https://archive.is/VTrj9
Let's pause a moment to appreciate the Economist's touching credulity about AI's coming legal mastery. The law seems to be the area where AI is most prone to "hallucinate" (that is, "produce defective outputs"), which can only be sorted through by skilled practitioners whose experience gives them the discernment to distinguish useful arguments from foolish ones:
(And this requires those skilled practitioners to avoid the "automation blindness" that afflicts people who are asked to remain vigilant for things that seldom occur, a phenomenon that has turned every TSA agent into the water-bottle-detectingest motherfucker the human race has ever produced, who still misses 95% of the guns that red teams bring through the checkpoint):
More notable than the Economist's faith-based predictions about the impending army of hyper-competent robo-lawyers is the magazine's proposed solution to this looming crisis: "stop creating entitlements that are ripe for AI-fuelled claims…prune the mass of procedural rights." Above all, replace the bureaucrats who process your "complaints, demands and appeals" with more AI, which will arbitrarily decide who gets what, through "personalised welfare interventions" that are not based on any kind of guaranteed rights.
Writing on his blog, the political scientist Henry Farrell tells us where this will inevitably end up: with AI-based robot wars in which increasingly stingy and pernickety robo-bureaucrats create demand for progressively more aggressive robo-lawyers:
As Farrell writes, this end-time was foretold by the prophet Alan Moore with his 1980s 2000 AD character Abelard Snazz, "the man with the two-storey brain":
https://en.wikipedia.org/wiki/Abelard_Snazz
Snazz "solves" the street crime epidemic on the planet Twopp with "Big Police Robots," who spiral out of control, arresting the citizens of Twopp for trivial crimes like wearing brown shoes with a blue suit ("breaking the laws of good taste"). To solve this new problem, Snazz invents "Big Criminal Robots" whose "cunning, efficient" crimes "take up all the police's time."
Twopp is left in a state of high-stakes Big Robot crimewars, in which the most efficient criminals imaginable battle the most ruthless robocops science can deliver, with the Twoppians caught in the crossfire, collateral damage in a robotic forever war (on crime).
As Farrell writes, this is already afflicting the US health system, where an army of insurance company robo-claim-deniers have been countered with a doctors' army of robot-claim-appealers:
The point being that people need health care, people need public services, and while there will always be some waste at the margins (whether due to incompetence or dishonesty) responding to this by beefing up the system's defenses with more advanced red tape just requires the people who legitimately need these services to employ more aggressive tactics.
In support of this, Farrell points to a great, long essay by Dan "Accountability Sink" Davies for the Niskanen Center, "'The Problem Factory' – Preemptive risk aversion in infrastructure planning and the role of professional services":
Davies' essay describes how increasing bureaucratic defenses against frivolous or dishonest claims drives the participants in these processes to assume a war footing and approach the system as a battlefield, leading to the very runaway cost inflation that the bureaucratic process was instituted to prevent.
(Davies, a cybernetician, has some fascinating advice about how to structure planning processes to minimize this, but that's out of scope for this particular post.)
This reminds me of nothing so much as the spam wars. There was a time when it was very easy to set up a mail server and provide email access for anyone who wanted it – including spammers. Increased spam begat increased anti-spam countermeasures, notably the creation of blocklists that allowed mail administrators to automatically reject email from "insecure" mail servers.
Inevitably, spammers figured out how to send spam from "secure" servers, resulting in stricter, more onerous standards for mail server configuration. Spammers – for whom the ability to send spam is an existential matter – figured out how to meet these standards, so the security demands jumped again – and again, and again.
Today, sending and receiving mail is so technically challenging that most of the internet's email is run by a handful of giant, mostly US-based corporations. If any of these companies decides your mail server is spamming, you effectively disappear from the internet and good luck getting them to acknowledge an error. Meanwhile, these companies emit an avalanche of spam, but no one will ever block their servers, because to do so would be to cut off billions of legitimate email users:
https://pluralistic.net/2021/10/10/dead-letters/
And since most of these companies are US-based, they are liable to being weaponized by Trump, who has taken to ordering his tech giants to block foreign officials whose policy decisions make him angry:
Another parallel is the content moderation wars that saw the large platforms coming up with progressively more detailed rules about what constituted harassment and hate speech, only to have dedicated trolls master these rule-books. Trolls – for whom harassment was a full-time vocation – became the world's greatest experts on the platforms' speech policies, which let them skate right up to the line when abusing their victims, and to get those victims kicked off the platforms if they could be lured into putting a single toe over the line in response:
Farrell criticizes the Economist's answer to the (allege) looming robo-lawyer threat as "solutionism," Evgeny Morozov's word for "Recasting all complex social situations either as neat problems with definite, computable solutions":
https://en.wikipedia.org/wiki/Technological_fix
Using AI to root AI-generated bureaucratic appeals sacrifices the system's putative purpose – delivering services – in the name of defending that service from abuse and misuse of the system's resources. As the pioneering cybernetician Stafford Beer famously wrote, "the purpose of a system is what it does." If your bureaucracy is more concerned with fighting fraud than delivering service, then it isn't a service delivery system at all – it's a service denial system.
As Farrell writes, the people of Twopp can tell you how this ends – in a war of giant robots in which we are all collateral damage.
(A brief postscript: Farrell is a font of science fictional analogies to modern policy issues. This weekend in the FT, he and Dan Wang published an excellent editorial on the relevance of the paranoid, claustrophobic fiction of Philip K Dick to our present political reality:)
2000AD Regened roars back into newsagents and comic shops next Wednesday... along with the return of Abelard Snazz!
2000AD Regened roars back into newsagents and comic shops next Wednesday… along with the return of Abelard Snazz!
What’s not to love about an angry dinosaur on the cover of 2000AD – and one from ace artist Alex Ronald too, drawing us into the latest 2000AD Regened edition?
2000AD Prog 2206 is another “all ages” 52-page special, the fourth and final “Regened” issue of the year, a self-contained edition that’s perfect for new readers, crammed with a variety of strips, including “Abelard Snazz” written by…
