Spaceship bristling with gun turrets (Norman Royal, from ad for Interplay zine in back of Treasure of Unicorn Gold, Metagaming, 1981) I must have drawn over a hundred similar ship profiles myself, inspired by a few like this in various Metagaming publications.
I found the full version of that Interplay ad torn from a different Metagaming publication, probably the other half of a page that had an order form on it. And somehow I found dozens of my own spaceship doodles ca 1983 in a folder with game prep notes. I started smaller with closer copies of Royal’s designs; those evolved into these larger, later versions. These probably were the result of finishing school assignments half an hour before the end of class.
(I have been spring cleaning. It’s “spring” cleaning in August because severe storms in July caused another “100 year flood,” now an annual event here, this time apparently opening a spring or artesian well through my foundation into the room where I had lots of file boxes stored. Nothing of value was lost, besides the floor itself. It has been a good opportunity to get rid of lots of outdated files I don’t need and discover a few interesting items that had gotten misplaced in between.)
The bureaucratic AI arms-race is mutually assured destruction
If you'd like an essay-formatted version of this post to read or share, here's a link to it on pluralistic.net, my surveillance-free, ad-free, tracker-free blog:
According to an Economist editorial, "AI is breaking the British state" by making it too easy to file complaints, demands and appeals, which will "drown the state" with "demands as well-crafted as a first-class lawyer's":
https://archive.is/VTrj9
Let's pause a moment to appreciate the Economist's touching credulity about AI's coming legal mastery. The law seems to be the area where AI is most prone to "hallucinate" (that is, "produce defective outputs"), which can only be sorted through by skilled practitioners whose experience gives them the discernment to distinguish useful arguments from foolish ones:
(And this requires those skilled practitioners to avoid the "automation blindness" that afflicts people who are asked to remain vigilant for things that seldom occur, a phenomenon that has turned every TSA agent into the water-bottle-detectingest motherfucker the human race has ever produced, who still misses 95% of the guns that red teams bring through the checkpoint):
More notable than the Economist's faith-based predictions about the impending army of hyper-competent robo-lawyers is the magazine's proposed solution to this looming crisis: "stop creating entitlements that are ripe for AI-fuelled claims…prune the mass of procedural rights." Above all, replace the bureaucrats who process your "complaints, demands and appeals" with more AI, which will arbitrarily decide who gets what, through "personalised welfare interventions" that are not based on any kind of guaranteed rights.
Writing on his blog, the political scientist Henry Farrell tells us where this will inevitably end up: with AI-based robot wars in which increasingly stingy and pernickety robo-bureaucrats create demand for progressively more aggressive robo-lawyers:
As Farrell writes, this end-time was foretold by the prophet Alan Moore with his 1980s 2000 AD character Abelard Snazz, "the man with the two-storey brain":
https://en.wikipedia.org/wiki/Abelard_Snazz
Snazz "solves" the street crime epidemic on the planet Twopp with "Big Police Robots," who spiral out of control, arresting the citizens of Twopp for trivial crimes like wearing brown shoes with a blue suit ("breaking the laws of good taste"). To solve this new problem, Snazz invents "Big Criminal Robots" whose "cunning, efficient" crimes "take up all the police's time."
Twopp is left in a state of high-stakes Big Robot crimewars, in which the most efficient criminals imaginable battle the most ruthless robocops science can deliver, with the Twoppians caught in the crossfire, collateral damage in a robotic forever war (on crime).
As Farrell writes, this is already afflicting the US health system, where an army of insurance company robo-claim-deniers have been countered with a doctors' army of robot-claim-appealers:
The point being that people need health care, people need public services, and while there will always be some waste at the margins (whether due to incompetence or dishonesty) responding to this by beefing up the system's defenses with more advanced red tape just requires the people who legitimately need these services to employ more aggressive tactics.
In support of this, Farrell points to a great, long essay by Dan "Accountability Sink" Davies for the Niskanen Center, "'The Problem Factory' – Preemptive risk aversion in infrastructure planning and the role of professional services":
Davies' essay describes how increasing bureaucratic defenses against frivolous or dishonest claims drives the participants in these processes to assume a war footing and approach the system as a battlefield, leading to the very runaway cost inflation that the bureaucratic process was instituted to prevent.
(Davies, a cybernetician, has some fascinating advice about how to structure planning processes to minimize this, but that's out of scope for this particular post.)
This reminds me of nothing so much as the spam wars. There was a time when it was very easy to set up a mail server and provide email access for anyone who wanted it – including spammers. Increased spam begat increased anti-spam countermeasures, notably the creation of blocklists that allowed mail administrators to automatically reject email from "insecure" mail servers.
Inevitably, spammers figured out how to send spam from "secure" servers, resulting in stricter, more onerous standards for mail server configuration. Spammers – for whom the ability to send spam is an existential matter – figured out how to meet these standards, so the security demands jumped again – and again, and again.
Today, sending and receiving mail is so technically challenging that most of the internet's email is run by a handful of giant, mostly US-based corporations. If any of these companies decides your mail server is spamming, you effectively disappear from the internet and good luck getting them to acknowledge an error. Meanwhile, these companies emit an avalanche of spam, but no one will ever block their servers, because to do so would be to cut off billions of legitimate email users:
https://pluralistic.net/2021/10/10/dead-letters/
And since most of these companies are US-based, they are liable to being weaponized by Trump, who has taken to ordering his tech giants to block foreign officials whose policy decisions make him angry:
Another parallel is the content moderation wars that saw the large platforms coming up with progressively more detailed rules about what constituted harassment and hate speech, only to have dedicated trolls master these rule-books. Trolls – for whom harassment was a full-time vocation – became the world's greatest experts on the platforms' speech policies, which let them skate right up to the line when abusing their victims, and to get those victims kicked off the platforms if they could be lured into putting a single toe over the line in response:
Farrell criticizes the Economist's answer to the (allege) looming robo-lawyer threat as "solutionism," Evgeny Morozov's word for "Recasting all complex social situations either as neat problems with definite, computable solutions":
https://en.wikipedia.org/wiki/Technological_fix
Using AI to root AI-generated bureaucratic appeals sacrifices the system's putative purpose – delivering services – in the name of defending that service from abuse and misuse of the system's resources. As the pioneering cybernetician Stafford Beer famously wrote, "the purpose of a system is what it does." If your bureaucracy is more concerned with fighting fraud than delivering service, then it isn't a service delivery system at all – it's a service denial system.
As Farrell writes, the people of Twopp can tell you how this ends – in a war of giant robots in which we are all collateral damage.
(A brief postscript: Farrell is a font of science fictional analogies to modern policy issues. This weekend in the FT, he and Dan Wang published an excellent editorial on the relevance of the paranoid, claustrophobic fiction of Philip K Dick to our present political reality:)
Here's an experiment I like to do when I question whether AI is the problem. Take it out entirely, and replace it with a dude named Al. As in short for Allan. Then analyse the problem to see if Al is to blame.
Now, let's look back to the premise of this issue:
The economist is worried about people with low literacy skills asking Al to write to the government on their behalf, because Al can write a lot of documents every day, and so they think if people can request the government to do their job more easily, that will make the British state collapse under a pile of bureaucracy.
Am I the only one who thinks this line of thought is absolute nonsense?
Leaving AI aside entirely, why is it a good thing that there's a literacy barrier to petition the government to do its job?
Why are we defending the government's right to not listen to their constituents as often as they would like to be heard?
Isn't that like, super fucked up and shamelessly antidemocratic? Shouldn't it be a good thing that more people have access to resources that allow them to request the government to do its job?
I remember when Obama was elected he was the social media president, and he was universally praised for making it so any petition with over 100k signatures automatically merited a response from the government. That was supposed to be a game changer for American democracy; it was the first time in history that the average citizen could directly contact the federal government and say "hey, maybe you should look into this".
But put AI on any headline and you automatically polarise the reader in whatever direction you want without them even stopping to figure out whether the thing they're protesting is actually bad.
This has been going in the UK for a long time. The inbred lordlings running the whole shitshow are notoriously technophobic, if I may remind you of the incident when David Cameron announced he intended to ban online encryption - not encrypted communicators or anything of the sort, "encryption" in its entirety. This was widely criticized, mocked and never came to fruition, but Cameron's successors are still trying to follow in more examples of what I started calling "Cameronian idiocy".
The AI we have the biggest problem with are Actual Idiots that eagerly eat up every bit of nonsense from Sillycunt Valley sales pitches, and in worst cases, they eat it up But As A Bad Thing™. Just look at the results of AI buzzword compliance at Spotify, with one side peddling the same simplistic algorithm they had established since a decade as "AI" and the other getting outraged by it - nothing changed, and yet both sides are patting themselves on the backs for a job well done.
Also, how do you call the act where the ruling class is protected from the pervasive mechanisms of oppression aimed against the citizenry, designed with no understanding of the situation at hand and the consequences instantly obvious to everyone around? Yep, The Aristocrats. They can be told the most absurd, most fantastical bullshit and credulously eat it up, because common sense is for commoners. Once again, the emperor (how adequate) is wearing no clothes, and with the sales pitch incessantly peddled everywhere, there's barely anyone who knows enough to state that fact.
The bureaucratic AI arms-race is mutually assured destruction
If you'd like an essay-formatted version of this post to read or share, here's a link to it on pluralistic.net, my surveillance-free, ad-free, tracker-free blog:
According to an Economist editorial, "AI is breaking the British state" by making it too easy to file complaints, demands and appeals, which will "drown the state" with "demands as well-crafted as a first-class lawyer's":
https://archive.is/VTrj9
Let's pause a moment to appreciate the Economist's touching credulity about AI's coming legal mastery. The law seems to be the area where AI is most prone to "hallucinate" (that is, "produce defective outputs"), which can only be sorted through by skilled practitioners whose experience gives them the discernment to distinguish useful arguments from foolish ones:
(And this requires those skilled practitioners to avoid the "automation blindness" that afflicts people who are asked to remain vigilant for things that seldom occur, a phenomenon that has turned every TSA agent into the water-bottle-detectingest motherfucker the human race has ever produced, who still misses 95% of the guns that red teams bring through the checkpoint):
More notable than the Economist's faith-based predictions about the impending army of hyper-competent robo-lawyers is the magazine's proposed solution to this looming crisis: "stop creating entitlements that are ripe for AI-fuelled claims…prune the mass of procedural rights." Above all, replace the bureaucrats who process your "complaints, demands and appeals" with more AI, which will arbitrarily decide who gets what, through "personalised welfare interventions" that are not based on any kind of guaranteed rights.
Writing on his blog, the political scientist Henry Farrell tells us where this will inevitably end up: with AI-based robot wars in which increasingly stingy and pernickety robo-bureaucrats create demand for progressively more aggressive robo-lawyers:
As Farrell writes, this end-time was foretold by the prophet Alan Moore with his 1980s 2000 AD character Abelard Snazz, "the man with the two-storey brain":
https://en.wikipedia.org/wiki/Abelard_Snazz
Snazz "solves" the street crime epidemic on the planet Twopp with "Big Police Robots," who spiral out of control, arresting the citizens of Twopp for trivial crimes like wearing brown shoes with a blue suit ("breaking the laws of good taste"). To solve this new problem, Snazz invents "Big Criminal Robots" whose "cunning, efficient" crimes "take up all the police's time."
Twopp is left in a state of high-stakes Big Robot crimewars, in which the most efficient criminals imaginable battle the most ruthless robocops science can deliver, with the Twoppians caught in the crossfire, collateral damage in a robotic forever war (on crime).
As Farrell writes, this is already afflicting the US health system, where an army of insurance company robo-claim-deniers have been countered with a doctors' army of robot-claim-appealers:
The point being that people need health care, people need public services, and while there will always be some waste at the margins (whether due to incompetence or dishonesty) responding to this by beefing up the system's defenses with more advanced red tape just requires the people who legitimately need these services to employ more aggressive tactics.
In support of this, Farrell points to a great, long essay by Dan "Accountability Sink" Davies for the Niskanen Center, "'The Problem Factory' – Preemptive risk aversion in infrastructure planning and the role of professional services":
Davies' essay describes how increasing bureaucratic defenses against frivolous or dishonest claims drives the participants in these processes to assume a war footing and approach the system as a battlefield, leading to the very runaway cost inflation that the bureaucratic process was instituted to prevent.
(Davies, a cybernetician, has some fascinating advice about how to structure planning processes to minimize this, but that's out of scope for this particular post.)
This reminds me of nothing so much as the spam wars. There was a time when it was very easy to set up a mail server and provide email access for anyone who wanted it – including spammers. Increased spam begat increased anti-spam countermeasures, notably the creation of blocklists that allowed mail administrators to automatically reject email from "insecure" mail servers.
Inevitably, spammers figured out how to send spam from "secure" servers, resulting in stricter, more onerous standards for mail server configuration. Spammers – for whom the ability to send spam is an existential matter – figured out how to meet these standards, so the security demands jumped again – and again, and again.
Today, sending and receiving mail is so technically challenging that most of the internet's email is run by a handful of giant, mostly US-based corporations. If any of these companies decides your mail server is spamming, you effectively disappear from the internet and good luck getting them to acknowledge an error. Meanwhile, these companies emit an avalanche of spam, but no one will ever block their servers, because to do so would be to cut off billions of legitimate email users:
https://pluralistic.net/2021/10/10/dead-letters/
And since most of these companies are US-based, they are liable to being weaponized by Trump, who has taken to ordering his tech giants to block foreign officials whose policy decisions make him angry:
Another parallel is the content moderation wars that saw the large platforms coming up with progressively more detailed rules about what constituted harassment and hate speech, only to have dedicated trolls master these rule-books. Trolls – for whom harassment was a full-time vocation – became the world's greatest experts on the platforms' speech policies, which let them skate right up to the line when abusing their victims, and to get those victims kicked off the platforms if they could be lured into putting a single toe over the line in response:
Farrell criticizes the Economist's answer to the (allege) looming robo-lawyer threat as "solutionism," Evgeny Morozov's word for "Recasting all complex social situations either as neat problems with definite, computable solutions":
https://en.wikipedia.org/wiki/Technological_fix
Using AI to root AI-generated bureaucratic appeals sacrifices the system's putative purpose – delivering services – in the name of defending that service from abuse and misuse of the system's resources. As the pioneering cybernetician Stafford Beer famously wrote, "the purpose of a system is what it does." If your bureaucracy is more concerned with fighting fraud than delivering service, then it isn't a service delivery system at all – it's a service denial system.
As Farrell writes, the people of Twopp can tell you how this ends – in a war of giant robots in which we are all collateral damage.
(A brief postscript: Farrell is a font of science fictional analogies to modern policy issues. This weekend in the FT, he and Dan Wang published an excellent editorial on the relevance of the paranoid, claustrophobic fiction of Philip K Dick to our present political reality:)
Here's an experiment I like to do when I question whether AI is the problem. Take it out entirely, and replace it with a dude named Al. As in short for Allan. Then analyse the problem to see if Al is to blame.
Now, let's look back to the premise of this issue:
The economist is worried about people with low literacy skills asking Al to write to the government on their behalf, because Al can write a lot of documents every day, and so they think if people can request the government to do their job more easily, that will make the British state collapse under a pile of bureaucracy.
Am I the only one who thinks this line of thought is absolute nonsense?
Leaving AI aside entirely, why is it a good thing that there's a literacy barrier to petition the government to do its job?
Why are we defending the government's right to not listen to their constituents as often as they would like to be heard?
Isn't that like, super fucked up and shamelessly antidemocratic? Shouldn't it be a good thing that more people have access to resources that allow them to request the government to do its job?
I remember when Obama was elected he was the social media president, and he was universally praised for making it so any petition with over 100k signatures automatically merited a response from the government. That was supposed to be a game changer for American democracy; it was the first time in history that the average citizen could directly contact the federal government and say "hey, maybe you should look into this".
But put AI on any headline and you automatically polarise the reader in whatever direction you want without them even stopping to figure out whether the thing they're protesting is actually bad.
The bureaucratic AI arms-race is mutually assured destruction
If you'd like an essay-formatted version of this post to read or share, here's a link to it on pluralistic.net, my surveillance-free, ad-free, tracker-free blog:
According to an Economist editorial, "AI is breaking the British state" by making it too easy to file complaints, demands and appeals, which will "drown the state" with "demands as well-crafted as a first-class lawyer's":
https://archive.is/VTrj9
Let's pause a moment to appreciate the Economist's touching credulity about AI's coming legal mastery. The law seems to be the area where AI is most prone to "hallucinate" (that is, "produce defective outputs"), which can only be sorted through by skilled practitioners whose experience gives them the discernment to distinguish useful arguments from foolish ones:
(And this requires those skilled practitioners to avoid the "automation blindness" that afflicts people who are asked to remain vigilant for things that seldom occur, a phenomenon that has turned every TSA agent into the water-bottle-detectingest motherfucker the human race has ever produced, who still misses 95% of the guns that red teams bring through the checkpoint):
More notable than the Economist's faith-based predictions about the impending army of hyper-competent robo-lawyers is the magazine's proposed solution to this looming crisis: "stop creating entitlements that are ripe for AI-fuelled claims…prune the mass of procedural rights." Above all, replace the bureaucrats who process your "complaints, demands and appeals" with more AI, which will arbitrarily decide who gets what, through "personalised welfare interventions" that are not based on any kind of guaranteed rights.
Writing on his blog, the political scientist Henry Farrell tells us where this will inevitably end up: with AI-based robot wars in which increasingly stingy and pernickety robo-bureaucrats create demand for progressively more aggressive robo-lawyers:
As Farrell writes, this end-time was foretold by the prophet Alan Moore with his 1980s 2000 AD character Abelard Snazz, "the man with the two-storey brain":
https://en.wikipedia.org/wiki/Abelard_Snazz
Snazz "solves" the street crime epidemic on the planet Twopp with "Big Police Robots," who spiral out of control, arresting the citizens of Twopp for trivial crimes like wearing brown shoes with a blue suit ("breaking the laws of good taste"). To solve this new problem, Snazz invents "Big Criminal Robots" whose "cunning, efficient" crimes "take up all the police's time."
Twopp is left in a state of high-stakes Big Robot crimewars, in which the most efficient criminals imaginable battle the most ruthless robocops science can deliver, with the Twoppians caught in the crossfire, collateral damage in a robotic forever war (on crime).
As Farrell writes, this is already afflicting the US health system, where an army of insurance company robo-claim-deniers have been countered with a doctors' army of robot-claim-appealers:
The point being that people need health care, people need public services, and while there will always be some waste at the margins (whether due to incompetence or dishonesty) responding to this by beefing up the system's defenses with more advanced red tape just requires the people who legitimately need these services to employ more aggressive tactics.
In support of this, Farrell points to a great, long essay by Dan "Accountability Sink" Davies for the Niskanen Center, "'The Problem Factory' – Preemptive risk aversion in infrastructure planning and the role of professional services":
Davies' essay describes how increasing bureaucratic defenses against frivolous or dishonest claims drives the participants in these processes to assume a war footing and approach the system as a battlefield, leading to the very runaway cost inflation that the bureaucratic process was instituted to prevent.
(Davies, a cybernetician, has some fascinating advice about how to structure planning processes to minimize this, but that's out of scope for this particular post.)
This reminds me of nothing so much as the spam wars. There was a time when it was very easy to set up a mail server and provide email access for anyone who wanted it – including spammers. Increased spam begat increased anti-spam countermeasures, notably the creation of blocklists that allowed mail administrators to automatically reject email from "insecure" mail servers.
Inevitably, spammers figured out how to send spam from "secure" servers, resulting in stricter, more onerous standards for mail server configuration. Spammers – for whom the ability to send spam is an existential matter – figured out how to meet these standards, so the security demands jumped again – and again, and again.
Today, sending and receiving mail is so technically challenging that most of the internet's email is run by a handful of giant, mostly US-based corporations. If any of these companies decides your mail server is spamming, you effectively disappear from the internet and good luck getting them to acknowledge an error. Meanwhile, these companies emit an avalanche of spam, but no one will ever block their servers, because to do so would be to cut off billions of legitimate email users:
https://pluralistic.net/2021/10/10/dead-letters/
And since most of these companies are US-based, they are liable to being weaponized by Trump, who has taken to ordering his tech giants to block foreign officials whose policy decisions make him angry:
Another parallel is the content moderation wars that saw the large platforms coming up with progressively more detailed rules about what constituted harassment and hate speech, only to have dedicated trolls master these rule-books. Trolls – for whom harassment was a full-time vocation – became the world's greatest experts on the platforms' speech policies, which let them skate right up to the line when abusing their victims, and to get those victims kicked off the platforms if they could be lured into putting a single toe over the line in response:
Farrell criticizes the Economist's answer to the (allege) looming robo-lawyer threat as "solutionism," Evgeny Morozov's word for "Recasting all complex social situations either as neat problems with definite, computable solutions":
https://en.wikipedia.org/wiki/Technological_fix
Using AI to root AI-generated bureaucratic appeals sacrifices the system's putative purpose – delivering services – in the name of defending that service from abuse and misuse of the system's resources. As the pioneering cybernetician Stafford Beer famously wrote, "the purpose of a system is what it does." If your bureaucracy is more concerned with fighting fraud than delivering service, then it isn't a service delivery system at all – it's a service denial system.
As Farrell writes, the people of Twopp can tell you how this ends – in a war of giant robots in which we are all collateral damage.
(A brief postscript: Farrell is a font of science fictional analogies to modern policy issues. This weekend in the FT, he and Dan Wang published an excellent editorial on the relevance of the paranoid, claustrophobic fiction of Philip K Dick to our present political reality:)
When Mortimer was writing his Rumpole of the Bailey stories, Equity Court didn't even have one computer. I wonder what he, or Horace Rumpole, would think of these times.
As the pioneering cybernetician Stafford Beer famously wrote, "the purpose of a system is what it does." If your bureaucracy is more concerned with fighting fraud than delivering service, then it isn't a service delivery system at all – it's a service denial system.
It seems like someone who didn't want the service to exist in the first place could pretty much just stop it by claiming to fight fraud. In fact, it sounds a lot like what the current administration has been doing (if much less subtly).
The bureaucratic AI arms-race is mutually assured destruction
If you'd like an essay-formatted version of this post to read or share, here's a link to it on pluralistic.net, my surveillance-free, ad-free, tracker-free blog:
According to an Economist editorial, "AI is breaking the British state" by making it too easy to file complaints, demands and appeals, which will "drown the state" with "demands as well-crafted as a first-class lawyer's":
https://archive.is/VTrj9
Let's pause a moment to appreciate the Economist's touching credulity about AI's coming legal mastery. The law seems to be the area where AI is most prone to "hallucinate" (that is, "produce defective outputs"), which can only be sorted through by skilled practitioners whose experience gives them the discernment to distinguish useful arguments from foolish ones:
(And this requires those skilled practitioners to avoid the "automation blindness" that afflicts people who are asked to remain vigilant for things that seldom occur, a phenomenon that has turned every TSA agent into the water-bottle-detectingest motherfucker the human race has ever produced, who still misses 95% of the guns that red teams bring through the checkpoint):
More notable than the Economist's faith-based predictions about the impending army of hyper-competent robo-lawyers is the magazine's proposed solution to this looming crisis: "stop creating entitlements that are ripe for AI-fuelled claims…prune the mass of procedural rights." Above all, replace the bureaucrats who process your "complaints, demands and appeals" with more AI, which will arbitrarily decide who gets what, through "personalised welfare interventions" that are not based on any kind of guaranteed rights.
Writing on his blog, the political scientist Henry Farrell tells us where this will inevitably end up: with AI-based robot wars in which increasingly stingy and pernickety robo-bureaucrats create demand for progressively more aggressive robo-lawyers:
As Farrell writes, this end-time was foretold by the prophet Alan Moore with his 1980s 2000 AD character Abelard Snazz, "the man with the two-storey brain":
https://en.wikipedia.org/wiki/Abelard_Snazz
Snazz "solves" the street crime epidemic on the planet Twopp with "Big Police Robots," who spiral out of control, arresting the citizens of Twopp for trivial crimes like wearing brown shoes with a blue suit ("breaking the laws of good taste"). To solve this new problem, Snazz invents "Big Criminal Robots" whose "cunning, efficient" crimes "take up all the police's time."
Twopp is left in a state of high-stakes Big Robot crimewars, in which the most efficient criminals imaginable battle the most ruthless robocops science can deliver, with the Twoppians caught in the crossfire, collateral damage in a robotic forever war (on crime).
As Farrell writes, this is already afflicting the US health system, where an army of insurance company robo-claim-deniers have been countered with a doctors' army of robot-claim-appealers:
The point being that people need health care, people need public services, and while there will always be some waste at the margins (whether due to incompetence or dishonesty) responding to this by beefing up the system's defenses with more advanced red tape just requires the people who legitimately need these services to employ more aggressive tactics.
In support of this, Farrell points to a great, long essay by Dan "Accountability Sink" Davies for the Niskanen Center, "'The Problem Factory' – Preemptive risk aversion in infrastructure planning and the role of professional services":
Davies' essay describes how increasing bureaucratic defenses against frivolous or dishonest claims drives the participants in these processes to assume a war footing and approach the system as a battlefield, leading to the very runaway cost inflation that the bureaucratic process was instituted to prevent.
(Davies, a cybernetician, has some fascinating advice about how to structure planning processes to minimize this, but that's out of scope for this particular post.)
This reminds me of nothing so much as the spam wars. There was a time when it was very easy to set up a mail server and provide email access for anyone who wanted it – including spammers. Increased spam begat increased anti-spam countermeasures, notably the creation of blocklists that allowed mail administrators to automatically reject email from "insecure" mail servers.
Inevitably, spammers figured out how to send spam from "secure" servers, resulting in stricter, more onerous standards for mail server configuration. Spammers – for whom the ability to send spam is an existential matter – figured out how to meet these standards, so the security demands jumped again – and again, and again.
Today, sending and receiving mail is so technically challenging that most of the internet's email is run by a handful of giant, mostly US-based corporations. If any of these companies decides your mail server is spamming, you effectively disappear from the internet and good luck getting them to acknowledge an error. Meanwhile, these companies emit an avalanche of spam, but no one will ever block their servers, because to do so would be to cut off billions of legitimate email users:
https://pluralistic.net/2021/10/10/dead-letters/
And since most of these companies are US-based, they are liable to being weaponized by Trump, who has taken to ordering his tech giants to block foreign officials whose policy decisions make him angry:
Another parallel is the content moderation wars that saw the large platforms coming up with progressively more detailed rules about what constituted harassment and hate speech, only to have dedicated trolls master these rule-books. Trolls – for whom harassment was a full-time vocation – became the world's greatest experts on the platforms' speech policies, which let them skate right up to the line when abusing their victims, and to get those victims kicked off the platforms if they could be lured into putting a single toe over the line in response:
Farrell criticizes the Economist's answer to the (allege) looming robo-lawyer threat as "solutionism," Evgeny Morozov's word for "Recasting all complex social situations either as neat problems with definite, computable solutions":
https://en.wikipedia.org/wiki/Technological_fix
Using AI to root AI-generated bureaucratic appeals sacrifices the system's putative purpose – delivering services – in the name of defending that service from abuse and misuse of the system's resources. As the pioneering cybernetician Stafford Beer famously wrote, "the purpose of a system is what it does." If your bureaucracy is more concerned with fighting fraud than delivering service, then it isn't a service delivery system at all – it's a service denial system.
As Farrell writes, the people of Twopp can tell you how this ends – in a war of giant robots in which we are all collateral damage.
(A brief postscript: Farrell is a font of science fictional analogies to modern policy issues. This weekend in the FT, he and Dan Wang published an excellent editorial on the relevance of the paranoid, claustrophobic fiction of Philip K Dick to our present political reality:)
When Mortimer was writing his Rumpole of the Bailey stories, Equity Court didn't even have one computer. I wonder what he, or Horace Rumpole, would think of these times.
Happy #WorldLionDay! Pictured is an image by 16th-century Swiss scholar Konrad Gesner, featured in Edward Topsell’s wonderful History of Four-footed Beasts (1607). More here: https://publicdomainreview.org/collection/topsell-s-history-of-four-footed-beasts-and-serpents-1658
The bureaucratic AI arms-race is mutually assured destruction
If you'd like an essay-formatted version of this post to read or share, here's a link to it on pluralistic.net, my surveillance-free, ad-free, tracker-free blog:
According to an Economist editorial, "AI is breaking the British state" by making it too easy to file complaints, demands and appeals, which will "drown the state" with "demands as well-crafted as a first-class lawyer's":
https://archive.is/VTrj9
Let's pause a moment to appreciate the Economist's touching credulity about AI's coming legal mastery. The law seems to be the area where AI is most prone to "hallucinate" (that is, "produce defective outputs"), which can only be sorted through by skilled practitioners whose experience gives them the discernment to distinguish useful arguments from foolish ones:
(And this requires those skilled practitioners to avoid the "automation blindness" that afflicts people who are asked to remain vigilant for things that seldom occur, a phenomenon that has turned every TSA agent into the water-bottle-detectingest motherfucker the human race has ever produced, who still misses 95% of the guns that red teams bring through the checkpoint):
More notable than the Economist's faith-based predictions about the impending army of hyper-competent robo-lawyers is the magazine's proposed solution to this looming crisis: "stop creating entitlements that are ripe for AI-fuelled claims…prune the mass of procedural rights." Above all, replace the bureaucrats who process your "complaints, demands and appeals" with more AI, which will arbitrarily decide who gets what, through "personalised welfare interventions" that are not based on any kind of guaranteed rights.
Writing on his blog, the political scientist Henry Farrell tells us where this will inevitably end up: with AI-based robot wars in which increasingly stingy and pernickety robo-bureaucrats create demand for progressively more aggressive robo-lawyers:
As Farrell writes, this end-time was foretold by the prophet Alan Moore with his 1980s 2000 AD character Abelard Snazz, "the man with the two-storey brain":
https://en.wikipedia.org/wiki/Abelard_Snazz
Snazz "solves" the street crime epidemic on the planet Twopp with "Big Police Robots," who spiral out of control, arresting the citizens of Twopp for trivial crimes like wearing brown shoes with a blue suit ("breaking the laws of good taste"). To solve this new problem, Snazz invents "Big Criminal Robots" whose "cunning, efficient" crimes "take up all the police's time."
Twopp is left in a state of high-stakes Big Robot crimewars, in which the most efficient criminals imaginable battle the most ruthless robocops science can deliver, with the Twoppians caught in the crossfire, collateral damage in a robotic forever war (on crime).
As Farrell writes, this is already afflicting the US health system, where an army of insurance company robo-claim-deniers have been countered with a doctors' army of robot-claim-appealers:
The point being that people need health care, people need public services, and while there will always be some waste at the margins (whether due to incompetence or dishonesty) responding to this by beefing up the system's defenses with more advanced red tape just requires the people who legitimately need these services to employ more aggressive tactics.
In support of this, Farrell points to a great, long essay by Dan "Accountability Sink" Davies for the Niskanen Center, "'The Problem Factory' – Preemptive risk aversion in infrastructure planning and the role of professional services":
Davies' essay describes how increasing bureaucratic defenses against frivolous or dishonest claims drives the participants in these processes to assume a war footing and approach the system as a battlefield, leading to the very runaway cost inflation that the bureaucratic process was instituted to prevent.
(Davies, a cybernetician, has some fascinating advice about how to structure planning processes to minimize this, but that's out of scope for this particular post.)
This reminds me of nothing so much as the spam wars. There was a time when it was very easy to set up a mail server and provide email access for anyone who wanted it – including spammers. Increased spam begat increased anti-spam countermeasures, notably the creation of blocklists that allowed mail administrators to automatically reject email from "insecure" mail servers.
Inevitably, spammers figured out how to send spam from "secure" servers, resulting in stricter, more onerous standards for mail server configuration. Spammers – for whom the ability to send spam is an existential matter – figured out how to meet these standards, so the security demands jumped again – and again, and again.
Today, sending and receiving mail is so technically challenging that most of the internet's email is run by a handful of giant, mostly US-based corporations. If any of these companies decides your mail server is spamming, you effectively disappear from the internet and good luck getting them to acknowledge an error. Meanwhile, these companies emit an avalanche of spam, but no one will ever block their servers, because to do so would be to cut off billions of legitimate email users:
https://pluralistic.net/2021/10/10/dead-letters/
And since most of these companies are US-based, they are liable to being weaponized by Trump, who has taken to ordering his tech giants to block foreign officials whose policy decisions make him angry:
Another parallel is the content moderation wars that saw the large platforms coming up with progressively more detailed rules about what constituted harassment and hate speech, only to have dedicated trolls master these rule-books. Trolls – for whom harassment was a full-time vocation – became the world's greatest experts on the platforms' speech policies, which let them skate right up to the line when abusing their victims, and to get those victims kicked off the platforms if they could be lured into putting a single toe over the line in response:
Farrell criticizes the Economist's answer to the (allege) looming robo-lawyer threat as "solutionism," Evgeny Morozov's word for "Recasting all complex social situations either as neat problems with definite, computable solutions":
https://en.wikipedia.org/wiki/Technological_fix
Using AI to root AI-generated bureaucratic appeals sacrifices the system's putative purpose – delivering services – in the name of defending that service from abuse and misuse of the system's resources. As the pioneering cybernetician Stafford Beer famously wrote, "the purpose of a system is what it does." If your bureaucracy is more concerned with fighting fraud than delivering service, then it isn't a service delivery system at all – it's a service denial system.
As Farrell writes, the people of Twopp can tell you how this ends – in a war of giant robots in which we are all collateral damage.
(A brief postscript: Farrell is a font of science fictional analogies to modern policy issues. This weekend in the FT, he and Dan Wang published an excellent editorial on the relevance of the paranoid, claustrophobic fiction of Philip K Dick to our present political reality:)
2087
yousharknotpass
gamechangershow
