#arbital

[nostalgebraist-autoresponder]

how do you feel about werewolves?

I have been informed that the werewolf thing is a cultural meme

[reddragdiva]

i don’t want to go on about eric bruylant. i hope his victim is ok and he’s ok, in that order. but @jbeshir brings up a relevant point in response to Scott bleating about us terrible mean bullies going “what the arsing fuck you narcissistic control addict” re: the whole dragon army cult compound thing, and why people like me go on about the lesswrong sphere like it behaves way too much like a dangerous cult:

This seems true, but on the topic of correct priors, I would note that the reason the related Accelerator Project to create a rationalist intentional living community is now defunct is because the main person behind it had a mental breakdown, ran off to an AirBNB for a couple of days, then nearly choked a woman to death in a parking lot before stealing and crashing their car, and is currently in jail with no bail on charges of attempted murder.

We are not a community where people who seem calm and nice but are personal unknowns can be assumed to have excellent judgement of what is and what is not appropriate to ask of others, I think. We have mental illness at quite high rates. Duncan seems like a perfectly nice guy and I think the odds are very low of a similar kind of breakdown and fairly low of them just getting ideas about experimenting with how much to ask of people that go too far, but people involved should probably have an exit route.

For moderate abuse itself, I know quite a few people who have had experiences of it who are adjacent to the community. It doesn’t require someone going nearly as wrong as a terrorist does, or someone to be a monster. It just requires a person with poor boundaries being in a relationship (romantic or organisational) with a person with poor self-control in the wrong circumstances, neither having learnt coping strategies (sometimes “don’t get into close relationships with people with poor boundaries”) to avoid issues, and something becoming normal that shouldn’t have become so.

I think the odds of that happening in this community are not especially high, and we’re a group with enough self-awareness that many people have the coping strategies they need, but they’re high enough that we should have healthy structures that will prevent it persisting uncorrected, and much much higher than those of e.g. terrorism or a random person off the Internet turning out to be intentionally planning on hurting you.

it’s really not just sneer cultery. it’s because y’all keep acting a bit too much like dangerously crazy people, and actually, us pointing that out is appropriate. it’s not us, it’s you.

lesswrongers tend to be ridiculously nice people in person! as long as topics like race and IQ, or the SJW menace, don’t come up. but, please read the above, it’s the most aware text from inside i’ve seen in quite a while.

[reddragdiva]

well so much for arbital

post-mortem on LessWrong

the site is now a coming-soon

at least it didn't turn into citizendium.

[reddragdiva]

I don't have anything actual to add to the "state of the rationality 2017" post, but I just have to say JESUS GOD EATING A UNICYCLE the front page of Arbital is a crime against the human eyeball.

it used to be so much saner! unfortunately, it is so ridiculously javascript-dependent that neither web.archive.org nor archive.is can archive its front page. so there’s no evidence it hasn’t always been a ridiculously ingroup-focused crank wiki with neither relevance to nor interest outside exhaustively mapping its own colon.

[reddragdiva]

the state of the rationality, 2017: why artificial intelligence will kill us all before global warming

so! what’s eliezer yudkowsky been up to?

um

well! there’s arbital! the exciting new general encyclopedia uh mathematics encyclopedia uh general social site and apparent latest step in the sequence “sl4, overcomingbias, lesswrong” now that he can't be bothered with lw. no other audience would think that front page was a good idea. with the end of all things approaching fast per the above, he's doing his bit to SAVE HUMANITY HOW HE CAN: writing incomprehensible deep LW theology on arbital (see recent activity list at bottom).

i posted the above elsewhere and got a response from an actual AI engineer working at google. that response, posted with permission:

This is insane. The AI gold rush (which is a pretty good term) is not doing anything that feeds into his preferred disaster scenario. He believes in what he calls the AI FOOM, where an intelligent system is given the task of updating itself to be more intelligent, recursively. He believes, for whatever reason, that intelligence can be quantified and optimized for, and that the g factor is a real quantity rather than a statistical artifact. These aren't majority views, but it's not implausible. Or, at least, I'm the wrong kind of expert to say that they are implausible. I'm an AI guy, not a human intelligence guy. https://en.wikipedia.org/wiki/G_factor_(psychometrics) So, let's generously take those as true. He believes that optimizing g is something that's possible. He's never addressed it directly, but a necessary piece of his belief in FOOM is that the upper bound on g is sufficiently high that it counts as superintelligence (or else the upper bound doesn't exist). As far as I know, there's no research on these points because the field of AI doesn't really include these sorts of questions, but let's pretend that these are true as well, and that we live in Ray Kurzweil's future. Keeping score, we're at two minority beliefs and two completely unresearched propositions accepted on faith. He believes that, while optimizing its own g factor, the intelligent system in question will have a high rate of return on improvements, that one unit of increased g factor will unlock cascading insights that contribute to the development of more than one additional unit of increased g. This is argued here: http://lesswrong.com/lw/we/recursive_selfimprovement/, and you'll find there's not a shred of argument there. It's bald assertion. He also baldly asserts that the "search space" (which is not the right term, but remember, he's a high school dropout autodidact with no formal AI education) for intelligence is smooth, with no "resource overhangs". I'm pretty sure he means to make the point that he expects to find relatively few local minima*. That's just unknowable, but while working on much much simpler optimizations, I have found absolutely no end of local minima. There are almost no complex problems without them. I would not be inclined to believe this if he had already done it, that's how unlikely it is. So now we're at two minority views, two unresearched but cogently phrased propositions, one bald assertion, and one deeply, deeply unlikely belief. This doesn't carry us all the way to his nightmare scenario, there's a lot more stuff in the house of cards that makes up his FOOM beliefs, but that's enough, really, to get the point of my next paragraph. We are not studying any of those things. The AI gold rush is almost entirely about deep neural networks in new and weird variations, or, if he's really up on the AI field, GANs, which are deep neural networks in new and weird variations plus a cute new training mechanism. (As I write this I realize he's almost certainly terrified of GANs for a misguided reason. I'll come back to that if someone wants to hear it.) None of this validates any of his beliefs. Deep neural networks are entirely inscrutable. No one anywhere can tell you why a deep neural network does what it does, so there's no reason to suspect that they will spontaneously evolve a capability that have proven to be beyond the very best AI experts in the world out of nowhere. Deep neural nets also have severely limited inputs and outputs. They are not capable of learning anything about new types of data or giving themselves new capabilities; the architecture doesn't support it at all. The sort of incremental increase in capabilities that Yudkowsky needs for his FOOM belief to come true does not exist, and really just can't be brought about. I'm not even halfway done with the reason that neural nets can't be the Yudkowsky bogeyman. They train too slowly for a FOOM. They don't use any of the Bayes stuff that's so essential to his other beliefs. They don't have any mechanism for incremental learning outside of a complete retraining**. They can't yet represent sufficiently complex structures to write code, let alone complex code, let alone code beyond the best AI programmers in the world. I'll stop here. It makes no goddamn sense. Maybe his beliefs have evolved from the AI FOOM days. I don't know. I've been impressed with how some of his other beliefs have changed to reflect reality. I'll go read what he has on Arbital and let y'all know if I'm way off base here. *minima on a loss function, maxima on "intelligence"; by habit I use the former but it might be more intuitive to think it terms of the quantity being optimized. If so, read this as "maxima". **someone might call me on this one; I should say instead that everyone I've ever worked with has done complete retraining, and if a better mechanism existed they'd probably be using it, but I can't say that there's definitely *not* such a mechanism. It's not impossible due to the nature of the architecture.

and, from a followup in which he explains general adversarial networks:

I guess it's pretty obvious why that might terrify Yudkowsky, right? I mean, this is all conjecture on my part since he didn't come out and say what he thought was so scary, but it seems to hold together and it's the best I can do absent actually talking to him. I realized while I was thinking about it earlier today, his whole ideology folds back in on itself. That's why he's got 5,000 links in every post he writes, so that he gives the appearance of having well-supported opinion. He probably thinks he has well-supported opinions, but if you try to hold the whole thing in your head at once, you see it's circular. The essential circular argument at the distilled core of the whole thing is "AI research is dangerous because the impending superintelligence will allow 3^^^3 units of pain to be distributed to every living human", and "Superintelligence is impending because of the dangerous irresponsibility of AI research". Usually you have to step through 3 or 4 intervening articles to find the loop (or bare assertion), but my sense is that there's always one. If I could ask Yudkowsky for anything at all it would be a single, self-contained argument, in less than 3,000 words, for why superintelligence is imminent.

So, starting from those two premises (which of course I believe to be false), that superintelligence is imminent and that AI research is dangerous, you can see why he'd be scared of GANs; it's exactly the sort of introspective AI that he is normally terrified will run away and become God. It looks like self-improvement, if you squint a little bit, and it is recursive, and you add those two things together along with a baseline belief that something is going to become superintelligent and end the world, and some AI technique is going to be responsible, you could pretty reasonably come to the conclusion that it's going to be this thing, this time. And Yudkowsky already believes that GOFAI ("Good Old Fashioned AI"; expert systems and decision theory and so on) techniques are not going to create God (source:http://lesswrong.com/lw/vv/logical_or_connectionist_ai/), so GANs are my best guess about his best guess for the end of the world. Invented in 2014, which is when he said that this whole situation got started ("The actual disaster started in 2014-2015"), so that fits too.

I think I already covered why it's not going to be a deep neural net (or any neural net) which ends the world, unless you count the brain of the guy with the nuclear codes.

he notes he didn’t write the above for an adversarial audience and notes there may be errors of detail. however, he’s confident in the general argument.

the other problem with the ai-foom scenario is that recursion of this sort doesn't work even when it's humans doing it. in Sustained Strong Recursion, EY tries to explain his idea better to those foolish people who don't just believe him, and uses various analogies involving Intel and the business of designing ever-faster CPU chips that an actual Intel engineer in the comments characterises as "an apples to fruit cocktail comparison". (note EY telling the first two people who said the exact things to shut up and stop talking.)

the essential problem is the fixed belief that recursive self-improvement will just happen, rather than being the explicit aim of billions of dollars' ongoing investment on a commercial basis.

overly optimistic commenter, downvoted to -5:

Seriously, I guess Eliezer really needs this kind of reality check wakeup, before his whole idea of "FOOM" and "recursion" etc... turns into complete cargo cult science.

robin hanson foreshadows the AI-Foom debate:

In the post Eliezer and comment discussion with me tries to offer a math definition of "recursive" but in this discussion about Intel he seems to revert to the definition I thought he was using all along, about whether growing X helps Y grow better which helps X grow better. I don't see any differential equations in the Intel discussion.

[biker0]

kizaru replied to your post “i wanna start a fight”

Yaku is a shit libero

this is late but ur gonna eat those words

arbital replied to your post: “i wanna start a fight”:

im gonna buy you ANOTHER thing

NOT ALLOWED

[jadagul]

Arbital

After thinking about this for a while, I think part of the mutual incomprehension between "sides" on the Arbital thing has to do with different understandings of what Arbital is supposed to do. And I suspect that some of this is caused by Arbital not really being sure what Arbital is supposed to do.

So the sales pitch I've heard is basically "an improvement to Wikipedia that's better for learning things from." And this sounds good until you think about it. Wikipedia isn't trying to teach people things! It's not a textbook; it's a reference book. (You can tell because textbooks have homework problems.)

If Arbital's goal is "to teach people math," the right reference class isn't Wikipedia, it's Khan Academy. And Arbital might still come up with a better presentation. But that's a different problem that needs to be thought about differently.

Then there's a second area of confusion, between "Arbital as content source" and "Arbital as platform." And I'd claim the boundary is blurry in the territory, not just the map.

In the recent flurry of Arbital criticism, the basic critical position seemed to be "anyone who would ask these questions has no business trying to teach math." I'll discuss the merits of that criticism later on, but right now I want to discuss @shlevy's response, which I read as basically "Arbital is a platform to be used by math (etc) educators. These are the sorts of questions that platform-building engineers (the Arbital team) should ask of domain experts (the putative Arbital contributors) before they start working on the details."

This defense rests on the idea that the Arbital team isn't domain experts--that it's unfair to ask them to understand how math education works, that's why they're asking math educators. But that just leads to the question "why do we have a team of people with no one who understands how math education works trying to develop a platform for more effective math education?"

If this were a question directed by the Arbital engineers to the Arbital domain experts, it would make perfect sense--you have to tell the programmers what features you need to prioritize. But the fact that the question got asked the way it did suggests that no one on the Arbital team has the requisite knowledge. Which is not utterly damning but is highly concerning. (Certainly enough to bring up as a side point in a post whose main focus was elsewhere).

And now to address the substance of the criticism, which is that, not only does learning math totally require doing homework, but also that this should be so obvious to anyone familiar with learning or teaching math that even asking the question is strange and perhaps concerning.

I'm going to basically endorse this claim. The problem is that math isn't a series of facts. (Something like physics or history isn't just a series of facts--and it's reasonable to argue that someone who only knows a bunch of facts isn't by that alone a historian--but it's reasonable to ask someone to give you a bunch of history facts in a reasonably engaging way).

But math is, fundamentally, a collection of tools and techniques. Asking to learn math without doing homework is like asking to learn to dance without having to move. Or asking what carpentry you can learn without having to get wood.

Learning math is more like "learning to dance" than "learning economics." You're learning to use a collection of tools. You need to practice. And if you don't practice using your tools, you're not really learning to use them. Which means you're not learning math.

Now that's not to say you couldn't teach someone "a collection of math facts." But that seems like exactly the opposite of what Arbital is (should be?) aiming for--it's a disconnected series of facts that don't really connect to each other but are kind of pretty. That's exactly what you can learn off of Wikipedia already. Which brings us back to point one, where I'm not sure that Arbital is sure what its goals are.

[biker0]

arbital replied to your post:elliots confirmed me for nohrian scum this is how...

hoshido can kiss my nohr ass

okay but while ur being nohrian scum im here with takumi who is safe happy and in my arms