That MIDI thread feels like I am reading a Deep Lore slapfight bitchfest, and none of it makes any sense. Glad it translates to you.
This is deep lore, yeah. Rambling, including #cw sexual assault, to follow.
The most annoying thing about this is that I agree with, like, 90% of what LS is saying (and obnoxiously implying). I actually have a pretty low opinion of MIRI: I think their research agenda is mostly useless, that they have contributed basically nothing of substance to the field of AI in nearly 20 years of existence, and that if superintelligence risk turns out to be a real concern (which I do not grant, I think it is possible but far from certain), MIRI will have done very little to help mitigate that risk.
This isn’t even their fault, per se. In the past, I have compared MIRI’s efforts to like if Ada Lovelace realized that computer hackers would eventually be a problem, and spent the rest of her life trying to develop theory and praxis around computer security. It wouldn’t have worked. No matter how smart she was, she just wouldn’t have been able to contribute anything useful: the availably knowledge in her day was just too far behind; she could not possibly have known what actual, realized computing was going to look like. That’s not her fault or impugning her abilities, it’s just the gap between when she was around and when the technology was ready.
Similarly, while I think we will have “strong AI” eventually, right now we have no idea what the architecture of that is going to look like. I have a high degree of confidence that it won’t be neural networks, RL, or anything else that’s state-of-the-art in the industry right now, nor will it be the weird-ass quasi-GOFAI formalism that MIRI’s research tends to emphasize. In my estimation, if and when strong AI emerges, it will be as a result of a complete paradigm shift, and all the literature MIRI has produced on the subject will be totally irrelevant to it.
The unpleasant reality is that there may not be anything we can do at the moment to meaningfully reduce AI X-risk (although I have trollingly suggested that advocating for worldwide Communist revolution might help). People in the orbit of MIRI don’t consider this possibility nearly as often as they should, for reasons about identity and institutional blind spots-- IME, in the past a lot of LWers tied up too much of their identity in “being the only ones taking AI X-risk seriously”, forgetting pg’s “keep your identity small” advice. And what happened was exactly what pg warned about: people really resist the implication that a big part of their identity, and the thing they’ve been passionately working on, is a waste of time. But it is what it is.
So, point #1 I agree with l-s on: MIRI is kind of dumb.
Point #2 I agree with l-s on: Peter Thiel is a goddamn Saturday morning cartoon villain. No argument from me there.
But it doesn’t necessarily follow from that that he has sinister motivations for donating to MIRI. I mean, he has also donated to climate change organizations in the past. Couldn’t he just be genuinely concerned about AI X-risk? Plenty of people are, especially in the Valley. Even if you think Thiel’s motivations are purely selfish, billionaires probably have just as if not more interest in reducing X-risk as the result of us: you can’t enjoy your spoils of capitalism if the world has been destroyed.
Frankly, I can’t see what sort of sinister agenda you’d advance by giving money to friggin’ Eliezer Yudkowsky. At the end of the day, the rationalists are just not that important. It’s a small number of weirdos-- many whom infamously have giant executive dysfunction issues! If you want to manipulate the world by handing out money, anyone-- lobbyists, politicians, the media, universities, anyone other than the damn rationalists-- is going to give way better bang for your buck. And Thiel, scumbag though he is, is a shrewd investor; it’s safe to say he knows how to allocate his dollars. The MIRI donations really are about the X-risk.
And I suspect l-s knows this: my reading of that thread was that she kept on Darkly Hinting about Thiel’s true motivations for donating to MIRI, over and over, until argumate got fed up and just asked her to state plainly what she meant, at which point she hurriedly switched the conversation to Kathy Forth.
And yes, what happened with Kathy Forth was incredibly shitty, and I’m still angry about it to this day. That’s agreement with l-s #3. Not only because of what happened, but also how in the immediate aftermath, several trusted people quickly used their platforms to slander her as having always been a liar. That was fuuuuuucked up.
But everything else about what l-s said/implied about that was just wrong:
- That and the Brent Dill affair around the same time actually did cause a bunch of reforms. The “whisper networks” were replaced with more formal processes at REACH, the Alumni Committee at CFAR was dissolved after that godawful memo, and a whole bunch of missing stairs and several generally awful people were banned from events.
- My reblog was admittedly angry and reflexive, but her reply to me was projecting so hard it could illuminate the far side of the moon. I’ve already said I don’t like MIRI nor what happened, but I’d bet good money she dismisses equally serious problems in her own camp all the time. And trying to handwave that away by saying “well my group is much bigger so you can’t take sexual assault problems as systematic, but for your group I can infer all your evil intentions from one incident” is both groundless and despicable.
- Tying it to Peter Thiel makes no damn sense. What would he possibly gain from such a thing? That is l-s trying to lump together all the things she dislikes into one Vast Conspiracy and harrumph away any objections by saying that anyone pointing out how and why it makes no sense must be In On It. That’s QAnon-tier crazy nonsense.
