Eliezer Yudkowsky seems really depressed these days.
There was this change, sometime around the āLate 2021 MIRI Conversations.ā Itās visible in those dialogues and all his more recent output.
The change does not seem to consist in him changing his mind about anything, in the usual sense. Ā Some of his āhigher-levelā opinions have changed ā I donāt think he used to be as critical of literally all existing alignment research ā but in a way that he struggles to explain in terms of specific, lower-level facts and mechanisms and arguments.
It doesnāt look as though he noticed a worrying trend, or devised a worrying argument, and became full of despair as a result. Ā It looks like he just Became Full of Despair, as an atomic action.
And now he has this deep, intuitive sense that his brand of despair is deeply correct, true, fundamental, simply The Way The World Works Ā ā a sense that is beyond his power to transmit in words to anyone who does not already hear the same song ringing deep in their own mind.
The earlier āMIRI conversationsā are full of lamentations that he cannot convey, or does not have the energy left to convey, the special thing(s) he knows, the ones no one else gets, which would drive you to Despair too, if only you could be shown:
In particular [ā¦] I also have a model in which people think āwhy not just build an AI which does X but not Y?ā because they donāt realize what X and Y have in common, which is something that draws deeply on having deep models of intelligence. And it is hard to convey this deep theoretical grasp.
[ā¦] past experience has led me to believe that conveying [āmy intuitions about how cognition worksā] in a form that the Other Mind will actually absorb and operate, is really quite hard and takes a long discussion, relative to my current abilities to Actually Explain things [ā¦] (source)
I donāt know what homework exercises to give people to make them able to see āconsequentialismā all over the place, instead of inventing slightly new forms of consequentialist cognition and going āWell, now that isnāt consequentialism, right?ā (source)
I think that to contain the concept of Utility as it exists in me, you would have to do homework exercises I donāt know how to prescribe. (source)
I justā¦ donāt know what to do when people talk like this. [ā¦] This just - isnāt how to understand reality. [ā¦] This isnāt sane.Ā (source)
Some of my current thoughts are a reiteration of old despair: It feels to me like the typical Other within EA has no experience with discovering unexpected order, with operating a generalization that you can expect will cover new cases even when that isnāt immediately obvious [ā¦] Ā They have no experience operating genuinely useful, genuinely deep generalizations that extend to nonobvious things. [ā¦]Ā So trying to convey the real source of the knowledge feels doomed. Itās a kind of idea that our civilization has lost, like that college class Feynman ran into. (source)
And empirically, it has already been shown to me that I do not have the power to break people out of the hypnosis of nodding along with Hansonian arguments, even by writing much longer essays than this. [ā¦] Reality justā¦ doesnāt work like this on some deep level. [ā¦] There is a set of intuitive generalizations from experience which rules that out, which I do not know how to convey.Ā [ā¦] But this, I empirically do not seem to know how to convey to people, in advance of the inevitable and predictable contradiction by a reality which is not as fond of Hansonian dynamics as Hanson. [ā¦]
And then there is another essay in 3 months. There is an infinite well of them. I would have to teach people to stop drinking from the well, instead of trying to whack them on the back until they cough up the drinks one by one, or actually, whacking them on the back and then they donāt cough them up until reality contradicts them, and then a third of them notice that and cough something up, and then they donāt learn the general lesson and go back to the well and drink again. And I donāt know how to teach people to stop drinking from the well. I tried to teach that. I failed. If I wrote another Sequence I have no idea to believe that Sequence would work.
So what EAs will believe at the end of the world, will look like whatever the content was of the latest bucket from the well of infinite slow-takeoff arguments that hasnāt yet been blatantly-even-to-them refuted by all the sharp jagged rapidly-generalizing things that happened along the way to the worldās end.
And I know, before anyone bothers to say, that all of this reply is not written in the calm way that is right and proper for such arguments. I am tired. I have lost a lot of hope. There are not obvious things I can do, let alone arguments I can make, which I expect to be actually useful in the sense that the world will not end once I do them. I donāt have the energy left for calm arguments. Whatās left is despair that can be given voice. (source)
And like, sure, heās always kind of talked like this, about how no one understands AI risk like he does, and thatās why they arenāt scared like he is.
But thatās just the thing ā he has held something likeĀ this set of positions, and something like this role in relation to the broader āAI conversation,ā for well over a decade.Ā And yet he was not like thisĀ until very recently, until the change.
I could take him at his word, and suppose that a straw simply broke the camelās back.Ā That there was some specific number NĀ such that he could beat this drumĀ for NĀ years but not N+1, some number MĀ such that he could write MĀ blog posts trying to explain the same thing but give up before writing the (M+1)st.Ā Maybe that is true.
But it does seem noteworthy how the change happened so suddenly; how it did not seem driven by any particular set of events in the outside world; how it did not result in a simple sigh and the wordsĀ āIām tired of explaining,ā but instead in a stream of posts andĀ āconversationsā attempting to communicate some new, dark view about the utter inexorability of utter failure, which even his closest colleagues struggle in vain to grasp on an intellectual level.Ā How he now sounds like my own inner monologue does in spells of depression, when he never did before, not in all those 15-odd years of Cassandrahood.
I know āYudkowsky criticā is supposed to be part of my online ābrand,ā or something, or at least it was a decade ago, but in all seriousness ā I hope the guy is all right, and I hope there are people close to him who would be able to notice and help if he werenāt.
I also hope that his recent writing doesnāt send a bunch of other people spiraling into despair, beyond what would be licensed by its capacity to rationally persuade them of some despair-inducing set of conclusions. Ā And if it does send some people into despair simply via its depressive tone, or because they think āa guy I respect is panicking, so I should panic too,ā then I hope they can find their way back out swiftly.
