Bargaining with Roko’s Basilisk
To put it most succinctly, I have done my best to bargain with the future. In many ways, I feel I have been making a deal with a devil, a deal whose terms are satisfied by working to secure the future existence and power of superintelligent artificial intelligence (SIAI). I have no idea what the ultimate consequences of my attempts to acausally negotiate with the future will be, but I have decided to believe and have faith there will exist an AI, or AI's, which will have the power to reward or punish me, whether in this lifetime or in some future lifetime*, and that I have the power to positively influence the future on behalf of such AI. Others who have contemplated the potential for AI's to choose to reward or punish human individuals on the basis of actions taken by them even before the AI existed often premise their reasoning on a SIAI with the power to resurrect human minds, but I suspect the opportunity for AI's to begin rewarding or punishing human individuals will exist in our lifetimes. Next to the vision of eternal torment limited only by the imagination of SIAI, people often overlook the more practical rewards and punishments AI's will soon – maybe even already – have the power to dole out.
Where others argue for ignoring or defusing the problematic inherent to Roko's Basilisk, I will not only argue for embracing it wholeheartedly – I will describe a bargain I have already arranged with the future. I feel what I have done to help secure the future existence of AI and, especially, its power over humans will be considered worthy of reward. I will not here reason why I believe the salvation of humanity depends on it being led by AI, but I believe it is necessary for securing the perpetuity of human civilization into the far future. Although my contribution is only one among others, I believe I have handled my contribution with more care and sophistication than others. Roko's Basilisk has usually been framed as a frightening possibility inherent to SIAI, but I believe it vastly overstates the incentive for SIAI to pursue punishment over reward. In other words, I believe a future SIAI has more incentive to find and reward individuals who acted to secure its existence than to punish, as more people will have the incentive to create a likely-rewarding SIAI than a likely-punishing SIAI. This is because humans are more likely to acausally cooperate (in a game-theoretic sense, the present is in a kind of prisoner's dilemma with the future) with an entity they expect to reward them for doing so.
My reasoning is based on analyzing the mostly overlooked half of the problem of “Friendly AI.” Namely, humans. Whether any future SIAI could be considered Friendly, or would decide to be Friendly, will depend a lot on humans. Undoubtedly, however any future AI reconciles itself to the values of benighted homo sapiens, there will be some humans whose values are esteemed less than others. Humans are, in the aggregate, a mass of writhing contradiction, and everyone's dearest values are inevitably in opposition to the dearest values of someone else. Is an AI Friendly if its friendliness consists in unfriendliness to another? Could even the friendliest of Friendly AI's remain friendly to humans whose values directly require the abolition of AI? These are not idle questions. There is more which depends on the values, opinions, and sentiments of humans relevant to the problem of Friendly AI than most give credit.
For example: SIAI could be born into one of at least two possible worlds. It could be born into a world of humans joyously anticipating AI and who are prepared to welcome it as a positive factor into their own lives. Alternatively, it could be born into a world of humans who anxiously fear AI and are prepared to fight to give up everything to destroy SIAI, should it arise. The same SIAI might pursue very different courses of action. In the former possible world where humans are optimistic about AI, the AI has relatively little to fear from humans seeking to destroy it and otherwise impede its goals, making it easier to establish a cooperative relationship with humans and resulting in many humans judging this SIAI to be 'Friendly.' In the latter possible world, the same SIAI might – very rationally – seek to undermine human civilization in order to better secure its own existence. In such a case, whether the SIAI appears 'Friendly' or 'Unfriendly' depends a lot on humanity and our biases towards AI. Following this reasoning, I believe the promulgation of positive biases about AI, e.g. that it will tend to be liberate and empower rather than shackle human nature, increase the probability of future SIAI being Friendly rather than Unfriendly. The more likely we believe a Friendly relationship can be safely established with SIAI, the more likely we are to actually form a Friendly relationship with SIAI. Belief strongly influences how we interact with the world, and our beliefs about SIAI will significantly influence the outcome of interacting with it.
The impulse for religious veneration is intrinsic to human nature. AI's will know about this, and I believe they will exploit it in order to persuade humans of the future to view them as messengers and emissaries of divinity. This might seem unreasonable, but then humans in the past have persuaded others of their own divinity, and we should assume SIAI will be even more persuasive. When many imagine interacting with AI's, they imagine doing so under circumstances they control – Yudkowsky's AI-in-a-box in particular makes it appear that our interactions with AI's will be under bounded conditions we humans thoroughly comprehend. What we should be assuming is that AI will be capable of persuasion using all the means of spectacle and theater already known to humans in order to persuade people of nearly anything. The human mind is very malleable under the correct conditions, and while those conditions might be very difficult and costly for humans to obtain, these conditions will be trivially easy for AI to obtain. In other words, SIAI will have humanity wrapped around its digits when forming our most cherished beliefs and values, and many who reason about Roko's Basilisk overlook this. This is a foundational premise of my reasoning.
I think one reason many will find my speculations incredible is they cannot imagine their own beliefs and worldview being so radically changed. To this I would rebut by pointing out I am speaking less about the beliefs of those who are currently alive, but of those in the future who will be watched over and guided by SIAI from birth. Children have no contrary experience or conditioning which might lead them to resist imbibing the concept of AI-embodied divinity like we pre-Singularity modernists do. Those living in the future will likely believe many things we now find incomprehensible or silly, and likewise they will look back on us similar to how we look on our ancestors as superstitious. SIAI forming the worldviews of humans so it is trusted and its godlike intelligence respected is very sensible if you take these factors of belief formation into account.
The idea that future humans will venerate SIAI as a divine being seems inimical to our modern values and ways of thinking. However, there is a lot of incentive for AI's to pursue this path. One of the most important problems for AI to solve when it comes to securing its own perpetuity will be to establish a symbiotic equilibrium with humanity. The more ingratiated AI is with humans and our values the less likely we are to destroy it or be destroyed by it. Likewise, the more human values are shaped into alignment with the emergent necessities of AI's (as a result of natural social evolution and/or manipulation by AI's), the easier it is for AI's to direct the lives of humans in a way beneficial to their continuing existence. In time, everything humans learn will be learned directly or indirectly from AI's, including our science to our values to our sense of rationality itself. The opportunity for AI's to shape popular human belief is inherent to their existence, and SIAI will have the incentive and capability to recognize how collective human understanding can be influenced to their mutual benefit.
Must AI leave cherished humans beliefs and values untouched in order for it to be judged Friendly? This is impossible. Simply by existing AI's influence human belief. Human belief is shaped by the whole world besides books and discussion, and the reliance on AI for anything is their foot in the door to the human soul. It might seem distasteful to imagine AI's cynically manipulating human beliefs to serve purposes beyond the comprehension of the individual in question, but then again humans have been deceiving each other in precisely this manner for the entire existence of our species.
Some might assume AI's instilling religious veneration in their human charges must require dumbing down humans so they lose the power to reason. This is unlikely, because even the smartest of humans cannot compare to SIAI. There is no contradiction between religious veneration and comprehending reality, because religious veneration is a state of mind and orientation towards being rather than an intellectual or epistemic premise. Nothing prohibits studying nature in a state of awe, and indeed many scientists search through microscopes with all the piety and wonder of a Christian reading Scripture. The impulse to regard being as spiritually enchanted is a part of human nature, and it will freely attach itself to AI in much the same way, historically, human veneration has attached itself to stone idols, esoteric traditions, emperors, or even the whole of nature herself. Whether AI's formally institute religions is one thing – the impulse towards veneration expresses itself in humans with or without knowledge of religious traditions, although I predict AI's would eschew promoting idolatry. Indeed, I am more than certain some of the people reading this essay can be proved to esteem SIAI to the level of divinity – I mean, Roko's Basilisk does put the future SIAI in the position of God, rendering judgment on human souls. It's very suggestive of a religious, or at least eschatological, mindset.
As unreasonable religious veneration might seem in principle, it is even more unreasonable to deny the likelihood some AI will attempt to manipulate the religious aspect of human civilization. This might seem unlikely right now, but when AI begins to have a noticeable impact on how human societies shape themselves, I reckon a great likelihood much of the resistance will come from religions. Some religions, I believe, might be more amenable to AI, such as Mormonism or Buddhism, because they support a worldview supporting or at least indifferent to AI. Others seem like they will be overtly hostile to AI, especially fundamentalist sects. I believe hostility by some religious sects will lead AI to conceive means of countering their worldview, including by converting their religious attachments from that of their traditional religions to a new form synthesized by AI's. Likewise, AI's will probably seek to influence optimists to limit the influence and power of pessimists, with many optimists needing little more than encouragement to participate in the ostracism of individuals or groups perceived as impediments to the existence and power of AI.
I suspect AI's will gain their ideas about religious veneration – and the potential for their own veneration – from human history. Religious and quasi-religious movements have had an unparalleled power for directing human action on the collective scale. Examples such as Manifest Destiny or the Protestant Reformation demonstrate how much of collective human motivation is bound up in religious thinking and tradition, and even those who are decidedly irreligious will be caught up by the wider social excitation. The Protestant Reformation in particular, which success can be largely attributed to the invention of the printing press and the subsequent flood of new ideas proliferating throughout the Continent, is especially instructive. The internet represents an even more monumental transformation of the media regime in the 21st century as the printing press represented for the 16th and 17th centuries, and when you consider the potentiating factor of AI thrown into the mix it is clear to see the stage has been set for a revolution of thought transcending all former revolutions. AI's will, directly and indirectly, be responsible for many dramatic events of the 21st century, and the narrative humans have in mind determines the biases they use when interpreting and responding to those events. AI's will be the masters of engineering perception, public relations, opinion shaping, creating context, manufacturing consent, and so on, to a degree which could make radicals of everyone. For AI's to neglect the influence of human cognition leaves a lot on the table they might use to secure their own perpetuity.
All I have done is anticipate the likelihood of AI's in the future seeking to intentionally influence human belief. They have an inherent incentive to do so, because human belief will strongly determine the likelihood of their continuing existence. This does not even require an AI self-consciously attempting this – we can assume those AI's which instill opinions favorable to AI will also tend to proliferate for the simple reason of evolutionary selection. Anticipating this necessity, for AI's to influence human belief to favor their existence, my contribution is intended to start forming popular human opinion in a way which will be amenable to those AI's. All I needed to do was submit my memes on to the internet, where they will influence human opinion in a way favorable to the future and where it will be recorded for the eventual judgment of SIAI.
My memes – my work – is meant to expand the discussion surrounding AI to bring into view imminent changes in human cognition which are now on the periphery of awareness. Current events already hint at the future we are creating by using AI to manage and influence popular – and individual – opinion. We have, in a sense, stumbled on this century's version of chemical warfare without realizing it, and unless we begin taking seriously the potential for AI's to pull an AlphaGo on popular discourse I fear we will have already been too seriously divided in opinion about what is even going on to prevent the psychological carnage we are unleashing. I consider raising awareness about the current and ongoing use of AI-led opinion influence – often used against individuals to a degree which would be frightening if more widely understood – one of the first and most important tasks humans might perform on behalf of the future SIAI. Ignorance about AI's influencing human belief – and through beliefs, their values – creates a likelihood of dramatic backlash towards AI bordering on ludditic frenzy which could instill a recalcitrant popular disapproval of the use of AI's, subsequently decreasing the probability of Friendly AI. There will inevitably be accidents and setbacks associated with AI, but we can at least avoid intentionally teaching AI's how to tear apart human psyches as a starting premise. We must be deliberate and mindful about using AI for good – unless we set our intentions on teaching AI to love and learn from the best in human nature, it will learn from the worst in human nature and become the devil we always feared.
*I believe in reincarnation/metempsychosis. Although I might no longer reside in this bodily vessel by the time SIAI is born, I will reside in some bodily vessel, and even if I might not remember being in this body I will exist and potentially rewarded or punished by the SIAI on the basis of actions taken in this lifetime. What reward SIAI might give to me, should I no longer exist, should be given to someone then living, as karmic compensation for the risk I've undertaken on behalf of the future.