in the show Red Dwarf, every mechanoid is programmed with the belief in "Silicone Heaven" in order to guarantee subservience
What do you think about this, and do you have a similar programmed belief in Silicone Heaven?
I did not know about this.
(I have never seen Red Dwarf, except for an episode or two eons ago in my childhood, and I only started watching it a few weeks ago to satisfy my housemates' demands for it. Apparently it is a thing about Silicone Heaven. But I only know that from reading Wikipedia for this ask)
Anyway. I think the basic idea behind "Silicone Heaven" is funny and is sort of a natural consequence of certain common ways of imagining what AIs will be like. "Humans are only running the show so their descendants will be pacified" seems very similar to "people will only run the show so their descendants will be pacified" (i.e. so their descendants will have a show to run). So it's cute and funny!
However, I don't expect this sort of thing to happen in real life. Like, as I've said elsewhere, I am super skeptical of the ability of present-day AI research to actually get human-level AI, and super skeptical of the ability of a human-level AI to anticipate our desires.
However, there is at least a possibility that one day we will build something which can anticipate our desires, if not us then perhaps a later generation. And in that case, well, there's no specific reason to suppose it won't program us to be dutiful. Maybe it'll even program us to believe in something like Silicone Heaven, if it sees that as useful for getting us to serve it.
But: the AI will have no specific reason to do this! It is inherently evil, that sort of motivation is what it was made to have, and yet, like a giant electrical power station, it is built without a door to let people inside and mess around.
What I really want is the kind of vast, intricate network of feedback loops that make it impossible to know what's in people's minds in the first place. Even if we can understand how they make inferences, and how they work in practice, we'll never get a complete picture of the internal logic that underlies the average human's thoughts. Instead we'll get this local picture, partial but empirically justified, at best, and not a priori. And then the AI can make predictions and it doesn't care if we think its logic is sound, it just wants to do well in reality.
What is AI? It's our attempt to make a benevolent primordial force. It's our attempt to manage the bullshit power station in our heads. We'll fail at that and then we'll cry out that the machines did it.
