I think your post about AI Doom doesn't really acknowledge the fact that, generally speaking, people enjoy being alive for its own sake and prefer it to being dead. Unless I'm misinterpreting, the conclusion of the post is essentially saying that not wanting people to be killed is "out of step with human values" which is obviously not true. Most people do not want to be killed. Killing people is bad. It would not be OK for AI to kill everyone even if it made something else afterwards.
(Pt 2) this all seems extremely obvious to me but I could not come up with an interpretation of that post which isn’t just broadly in favour of people being killed, which seems sort of like. The most evil thing anyone could ever possibly believe. So I am hoping that I misinterpreted
You're not alone, this aspect of yesterday's post was confusing to a lot of people.
FWIW I'm mostly tapped out on discussing the subject matter of that post for the moment, but this does deserves some kind of further explanation, so here goes.
----
First, to address something you didn't mention, but which was broadly confusing:
I am not saying: "when the doomers say AI will kill us all, they don't mean the natural reading of that phrase, they don't mean it will literally kill all the individual humans, they mean some weird other thing instead."
No, they really do just mean it will kill everyone. Sorry that wasn't clear.
----
What I did mean, when I talked about doomers vs. average Joe here, is that the idea of human extinction hits different if you're an anti-deathist transhumanist, versus if you aren't one.
If you're an anti-deathist, what's bad about extinction is, in part, the same thing that's bad about ordinary death. The anti-deathist looks around them and sees, in some sense, a slow-motion and staggered extinction already happening.
Even without extinction, we are all gonna die. Our great-great-great-great-great-grandparents' generation did not die out in an extinction event, but all the same, they are in fact extinct. Dead. 100% fatality rate, for those guys.
Sure, it was spread out over time, and "natural," but -- the anti-deathist argues, quite reasonably -- why should any of that matter to them, the dead ones? Those distinctions don't change any of what it is that's intuitively bad about dying in the first place.
The horror you express at "people being killed"? For the anti-deathist, that horror gets generalized to include the case of people being killed "by death," as it were. By just, dying, of old age or whatever, rather than by the hand of some other creature.
----
Now sure, even for the anti-deathist, there are important ways that extinction is worse than business as usual. Most obviously, extinction not only stops all the lives of people around now, but prevents the lives of any future people from getting created later on. (Plus of course, all else being equal, death sooner is worse than death later.)
If you're not an anti-deathist, though -- and most people aren't -- these special factors that make extinction worse (for the anti-deathist) are in fact your only objections to extinction.
That is not to say that they aren't extremely strong objections. Of course normal people do not want human extinction!
But for the normal person, there is this hard line between "extinction" and "business as usual." For such a person, there is a horror in the former that just isn't there in the latter, even though (as the anti-deathist likes to point out) business as usual still means a 100% fatality rate, on a long enough timeline.
For the anti-deathist, there is not this hard line. Extinction is bad. Getting killed by a person or a machine is bad. Dying of natural causes is bad. And a lot of the badness -- though by no means all of it -- comes from what is shared across all these cases, not what is special to each case alone.
----
OK, now let's talk more directly about your question.
Unless I'm misinterpreting, the conclusion of the post is essentially saying that not wanting people to be killed is "out of step with human values" which is obviously not true.
I mean, yeah, that's obviously not true.
But there are things sort of superficially similar to it that might be true.
And when something is true, but on the surface sounds bizarre and backwards and staggeringly wrong, I often like to play around with the way it sounds -- to just have a bit of fun with the way I can say things that seem so outrageous, and yet might not actually be wrong. Or even really outrageous, when properly understood.
And maybe I get carried with this, sometimes, at the expense of clarity. Sorry about that. (But also, it's my blog, where I write the kind of stuff I like writing. And I do like writing in this way. Them's the breaks.)
Anyway.
If we want to understand ordinary human values, then we need to cope with the "average Joe's" simultaneous belief in the following two things:
I really do not want to die. As a particular case, I really really do not want to die right now, today. But also, come to think of it, dying tomorrow would be super bad too. And you know what, the day after tomorrow? Same deal. And I guess I could go on like this.
I do not, at all, actively want to "live forever." In fact I kind of don't want this. If you directly ask me, I'll say the idea is sort of creepy and weird and bad. Or, even if I don't think that, I don't find the idea motivating at all. It might be acceptable, if it were forced on me, but none of my actions are driven by a desire to make it more likely.
(I am hand-waving away the concept of the afterlife here, which is involved in the typical Joe's actual beliefs in a way that annoyingly complicates the analysis while being tangential to my point. Let's say we're talking about the average atheist/agnostic but non-transhumanist Joe. I think the point can be generalized further, but I'm trying and failing to be brief here, so you'll just have to trust me.)
Now, together, these two beliefs are nearly a paradox.
Maybe they are just a paradox. Maybe you can't, really, think both of these at the same time without, on some level, kidding yourself. This is what the anti-deathist alleges, about the average Joe.
Maybe you agree. If so: congratulations, you're an anti-deathist too. Which is a perfectly valid point of view. Despite all I said in my post, I have quite a lot of sympathy for it, myself.
But the average Joe is really not an anti-deathist. This is just a fact about the world. Average Joe really does think both of the 2 things, at once. Maybe he does so inconsistently, or wrongly. Still, he does.
I think you essentially have two choices here. You can take the road less traveled, fully bite the "death is bad" bullet, and be an anti-deathist. Or, you can do what most do, and be like average Joe.
But if you are doing what average Joe does, and you go on to say things like...
being in favour of people being killed [is the] most evil thing anyone could ever possibly believe
...then you have some explaining to do. You have to spell out what it is this means, if it doesn't just mean full anti-deathism. Which is kinda what it sounds like.
A lot of things "kinda sound like" full anti-deathism. That view is very amenable to being phrased in terms that make it sound utterly obvious.
But we can't let this lull us into thinking that -- because anti-deathism sounds obvious, and average Joe often believes things that sound obvious -- that average Joe believes in anti-deathism. Somehow, despite all that obviousness, he just doesn't.
Somehow, despite all that obviousness, anti-deathism is a fringe position. And if we're not on the fringe, then we have to spell out just what it is that we believe instead.
Now OK, let's be real. You didn't say "being in favour of death" was the evil thing. What you wrote was "people being killed," not "people dying."
And that's what makes the distinction to you, right? I imagine? That it's bad news when some entity actively kills a person, that goes beyond the badness of death per se?
----
That does sound pretty intuitive! But what exactly is it that makes killing worse, here?
I didn't answer that question, in my post. I answered a bunch of other questions, instead. There are still more questions, which no one has asked me, but which I kind of feel I ought to answer, when talking about this topic. Nonetheless, I have to stop myself at some point, or I'll never do anything else. Hence these kinds of glaring lacunae.
I won't answer it here, either, in full. I have some other things to do today, and this is no longer just explicating what I meant earlier, this is new stuff. I'll just make some gestures, now, towards the kind of answer that would make sense of how I treated the topic in my earlier post.
----
So, there are some pretty obvious answers to "why is killing especially bad?"
Say, that it reflects poorly on the killer: an AI that would kill us all is probably an AI that's just plain bad morally.
Or, that we have a norm against it. It's a part of our ethics, the stuff we agree on as part of the social contract.
But you know what we don't have a norm against? If we're average Joe, and not on the fringe?
Killing chickens.
Or torturing chickens, and then killing them. Or breeding lots of them, specifically to be tortured, and then killed.
Sorry for the sudden swerve into vegan talking points! But this is kind of a big deal.
I've heard this cited, multiple times, by doomer types as a motivating case for being worried about how superintelligent AIs might treat us.
Just look at how we treat creatures that can very evidently feel pain -- but just happen to be different from us, not constituted the way we are, and in particular much less smart than we are!
And I, personally, find this argument pretty motivating. This is one of those arguments where even I have to hand it to the doomers.
But once we've allowed this much, we are in danger of conceding some really wild shit, if we don't tread carefully. Maybe we even should concede the wild shit, in the last analysis. Still, we should tread carefully.
Say you take the chicken argument seriously.
You've conceded that human values contain some really fucked-up things about how to treat other, dumber, "more primitive" beings. Beings of the kind that prevailed before the new, "super"-intelligent, sparkly, world-dominating species stepped onto the scene and changed everything.
You've conceded that humans are basically misaligned AIs, of the evil killeveryone Torment Nexus sort.
Remember, that was the whole substance of the argument: to make such awful AIs seem more plausible, by pointing out that such a thing already exists. Namely, us.
But now, what standing do we have to object to the AIs, without it rebounding back on us? Must we oppose ourselves just as fervently as we oppose the evil AIs, for the same reason?
"An AI that kills all humans" sounds pretty bad. Sounds like an evil thing, that we would not want to exist. But by the same token, we're evil, and we shouldn't exist.
(We might have wiped out chickens, if they weren't so tasty. There are plenty of non-tasty things which we did, in fact, wipe out. I and the doomers focus on chickens and the like, here, because what we did them is arguably even worse.)
Would we really accept an AI that's only "aligned with human values," and treats us about as well as we treat other beings when we are placed in an analogous scenario? Or do we hold AI to a higher standard -- one we can't possibly apply to ourselves, for that way lies madness?
Well, I don't know. These are tough questions.
But I would like to leave open some room to imagine, at least, that the advent of humanity was not (or not only) a catastrophe. That it was not, in fact, "the most evil thing possible."
Despite all the evil that we do, I'd like to imagine that.
And I'd like to imagine that, if there is such a thing as "human values," it contains this affirmation of the value of the advent of humanity.
And the value of things like the advent of humanity.
And the golden rule, and the rule of law. Which means, among other things: not holding you to a higher standard than I hold myself.
Even though the apparent implications of this are pretty nasty.
Philosophy is like that. Often you are between a rock and a hard place. Saying "that's a rock, don't you know that rocks cannot be walked through??" in an alarmed tone does not really get at the heart of the dilemma, or point the way to a solution.
----
All else being equal, of course, I would prefer not to be killed.
So would the chickens, I imagine.
We must not pretend there are easy answers, when there aren't.
We're like the chickens
I won't comment here on factory farms. Pretend we are in a world where the only way anyone raises chickens is letting them roam around a pasture with other animals to scratch and eat bugs and corn and whatever else they find and then one day they are quickly guillotined and don't ever really feel stress, fear, or intense pain.
That is an exceptionally good life for a prey animal. Most animals are prey animals and those in the wild tend to have very miserable deaths, the better ones involve being chased and torn to pieces. Nature red in tooth and claw and all of that.
Human's are prey to [abstract concept of social organisms that quasi-counter entropy; ACoSOQE]. Like the chickens, we are just minding our own business, and in modern times, having historically low stress, low pain, low fear lives and then ACoSOQE comes along and cuts our heads off and that's it. Sometimes it does it to our young men early in life, mostly it waits 70+ years, and we don't really understand it, but it's the best kind of living we have ever known.
I think most of us have a built in trust that ACoSOQE is better than the alternatives. You release the chickens in to the wild and they will overproduce, many will starve, they will succumb to other predators, and many will end up killing each other in a tight spot. Most of us would rather we mostly get our 70-80+ years than roll the dice on some other situation.
I think there are chickens out there who say "If we make a break for it, we can escape the farmer and get out into the wild and live up to 4-7 years instead of the 1-3 we get here on this farm. The farmer is evil!"
Most of the chickens will quite rationally see that the up to is waving away a lot of things that could make those extra years, even if achieved, less appealing.
Humans are the same. Oh you say you have a revolutionary idea that will greatly improve society? How many of us will it get killed?














