#worldlypositions

[worldlypositions]

Women should be able to open things

*** (Crossposted from world spirit sock puppet)

[worldlypositions]

AI risk was not invested by AI CEOs to hype their companies

I hear that many people believe that the idea of advanced AI threatening human existence was invented by AI CEOs to hype their products. I’ve even been condescendingly informed of this, as if I am the one at risk of naively accepting AI companies’ preferred narratives.

If you are reading this, you are probably familiar enough with the decades-old AI safety community to know this isn’t true. But I don’t have a good direct way to reach the people who could use this information, and still I hate to leave such a falsehood uncontested. So if this is obvious, I hope the post is still perhaps useful to point more distant and confused people toward.

~

I personally know that AI risk was not invented by the tech CEOs because I have been near the middle of it since at least 2009—before any of the prominent AI companies existed, let alone had CEOs who might be trying to hype their products.

Here’s are some miscellaneous events over the years to give you a sense of the implausibility of this:

2008 - I attempt to contact Eliezer Yudkowsky to inform him that I am ‘trying to figure out the optimal way to use my life’ and would like to hear a better account of why his plan (of worrying about AI risk) is good. I have read about it online, but would like a clearer account. Traveling the world shortly after undergrad later, I meet a handful of people in person in the Bay Area who care about this, and one argues strongly that I should prioritize AI risk over my previously preferred causes e.g. climate change. I decide to think about this.

2009 - I am still not very convinced that AI is the most important thing to work on, but go to stay with the people who are worried about it for a few months. I argue about it a lot with a handful of them. There seem to be about twenty of them locally in the South Bay, though many more who comment on the relevant blogs. My photography collection from this era is quite sparse.

I go to The Singularity Summit for my first time (and its fourth), which is very lively and full of people who are thinking seriously about the future of AI.

2010 - Deepmind is founded. (I am back at school.)

2011 - I start a philosophy PhD at CMU, hoping to be eligible to work at somewhere like the Future of Humanity Institute one day, which is a happening hub of discussion about existential risk, AI and other important issues, that I like to visit.

2012 - I visit the Bay more and hang out with the growing AI risk community there. I visit the UK and do the same. I go to the AGI 2012 Winter Intelligence Conference.

2013 - I move to Berkeley and work at MIRI for a semester during grad school. I measure algorithmic progress over time across various computer science domains, as input to expectations for artificial intelligence in future. I visit the UK and attend the Center for Effective Altruism’s ‘weekend away’ where we have a debate on which cause is best, between global poverty, animal welfare and extinction risk. Extinction risk wins—the crowd leaves having changed their mind in that direction on net. The three advocates just before or after:

2014 - I join MIRI properly. I research The Asilomar Conference and Leó Szilárd as evidence about whether it is worth people trying to deal with risks early, because people around mostly believe that the risks from AI are at least a decade away, and there is disagreement about whether that makes it futile. I run an online reading group about Superintelligence, a new book about AI risk. I co-found AI Impacts, a project to answer questions about the future of AI, because AI risk seems at least fairly plausibly the most important thing to work on, and I want to investigate more and share my thinking with others.

2015 - I attend the first FLI conference—it seems that more people and more prominent people are interested in AI safety! OpenAI is founded.

2016 - I lead a team to run the first Expert Survey on Progress in AI. The median probability given to an outcome of advanced AI that is “Extremely Bad (e.g., human extinction)” is already 5%.

2017 - Some people around me are getting very worried, and saying AGI will happen within several years. My survey gets a shocking amount of media attention, becoming the ‘16th most discussed paper’ in 2017 according to Altmetric. Apparently there is interest in this topic..

2018 - I go to a big workshop for people working on AI risk in the English countryside, and a Chilean summit where I talk on TV and the radio about AI risk. It feels like interest is still picking up, and I feel optimistic about talking to the public.

2019 - GPT-2 comes out. Someone tries to get it to name our house. My favorite names include things like “World peace: tigers and humans” and “rooftop hillside: the highest place in the world”. It is hilarious and useless, but also magical and wild. The things we have worried about for years are feeling more tangible, and people’s ‘AI timelines’ are shrinking.

2020 - The world is reminded that really crazy things can happen. AI Impacts becomes remote. I spend the year with my household, who are almost all working on AI risk. We enjoy whiteboards a lot and run at least one good house conference in this period.

2021 - Anthropic is founded

*** (Crossposted from world spirit sock puppet)

[worldlypositions]

A game of mattering

*** (Crossposted from world spirit sock puppet)

[worldlypositions]

A game of mattering

When I have an overwhelming number of things to do, and insufficient native urge to do them, I often arrange them into a kind of game for myself. The nature and appeal of this game has been relatively stable for about a year, after many years of evolution, so this seems like a reasonable time to share it. I also play it when I just want to structure my day and am in the mood for it. I currently play something like two or three times a week.

The game

The basic idea is to lay out the tasks in time a bit like obstacles in a platformer or steps in Dance Dance Revolution, then race through the obstacle course grabbing them under consistently high-but-doable time pressure.

Here’s how to play:

Draw a grid with as many rows as there are remaining hours in your hoped for productive day, and ~3 columns. Each box stands for a particular ~20 minute period (I sometimes play with 15m or 30m periods.)

Lay out the gameboard: break the stuff you want to do into appropriate units, henceforth ‘items’. An item should fit comfortably in the length of a box, and it should be easy enough to verify completion. (This can be achieved through house rules such as ‘do x a tiny bit = do it until I have a sense that an appropriate tiny bit has been done’ as long as you are happy applying them). Space items out a decent amount so that the whole course is clearly feasible. Include everything you want to do in the day, including nice or relaxing things, or break activities. Drinks, snacks, tiny bouts of exercise, looking at news sites for 5 minutes, etc. Design the track thoughtfully, with hard bouts followed by relief before the next hard bout.

To play, start in the first box, then move through the boxes according to the time of day. The goal in playing is to collect as many items as you can, as you are forced along the track by the passage of time. You can collect an item by doing the task in or before you get to the box it is in. If it isn’t done by the end of the box, it gets left behind. However if you clear any box entirely, you get to move one item anywhere on the gameboard. So you can rescue something from the past, or rearrange the future to make it more feasible, or if everything is perfect, you can add an entirely new item somewhere.

I used to play this with tiny post-it stickers, which I would gather in a large moving pile, acting as a counter:

Now I just draw the whole thing. Crossed out = collected; [] = rescued from the past, now implicitly in the final box; dot in the lower right = box cleared; dot next to item = task done but item stuck in the past (can be collected immediately if rescued).

Why is this good?

I think a basic problem with working on a big pile of things in a big expanse of time is that if you work or not during any particular minute, it feels like it makes nearly no difference to the expectation of success. I’m not quite sure why this is—in fact if I don’t work this minute, I’m going to get one minute less work done. But it feels like if I don’t work this minute, I only need to work a smidgen faster on average to get any particular amount of work done, so what does it matter if I work now or later? And if i had some particular goal (e.g. finishing writing some massive text today), it’s unlikely that my other efforts will get me exactly to the line where this minute pushed me over—probably I will either succeed with hours to spare (haha) or fail hours from my goals.

I picture what’s going on as vaguely something like this—there is often some amount of work that is going to make your success likely, and if you know that you are on a locally steep part of the curve, it is more motivating than if you are either far away from the steep part or don’t know where you are:

Yet on the other hand, the appeal of various non-work activities this specific minute might be the most distinct and tangible things in the world. So when there is a lot to be done in a long time, not working often looks more exciting than working, even if a more rational accounting would disagree.

Having a single specific thing to do within minutes is much more compelling: the task and the time are lined up so that my action right now matters. Slacking this minute is the difference between success and failure.

It feels very different to have one email to deal with in three minutes and to have a thousand to deal with in next fifty hours.

One might naively respond to this issue by breaking up one’s tasks into tiny chunks, then laying them out in a day of tiny time boxes, then aiming for each to happen by the end of its allotment. But this will be terrible. A few boxes in, either you’ll be ahead or behind. And either way, your immediate actions have drifted away from feeling like they matter. If you are ahead, the pressure is off: you’ll probably succeed at the next increment whether or not you work hard now. If behind, you are definitely going to fail at doing the next box on time, and probably some others, and your present work is for an increased chance of catching up at some vague future box, much like before you had these boxes. (Plus your activities are no longer in line with what your plan was, which for me makes it tempting to scrap the whole thing and do something else.)

A big innovation of this game is to instead ensure that you keep meeting tasks one at a time where each one matters in its moment, as in a game like Beat Saber or Dance Dance Revolution. The game achieves this by adjusting the slack to keep the next ten minutes’ action near the actually-mattering-to-success region all day. If you get behind you have to give up on items and move forward, so you aren’t left struggling for a low probability of catching up. If you get ahead, you add more items and thus tighten the slack.

A thing I like about this is that it actually makes the activity more genuinely fun and compelling, and doesn’t involve trying to trick or uncomfortably binding oneself. It is superficially a lot like a ‘productivity hack’, but I associate these with somehow manipulating or forcing yourself to do something that you at some level have real reason to dislike. I expect such tricks to fail, and I don’t think I want them to succeed.

This seems different: I think humans are just genuinely better at being in an enjoyable flow state when their activities have certain structures that are genuinely compatible with a variety of tasks. Beat saber wouldn’t be fun if all the boxes were just sitting in a giant pile and you had to beat your way through as many as you could over an hour. But with the boxes approaching one at a time, at a manageable rate, where what you do in each moment matters, it really is fun (for many people, I hear—I actually don’t love it, but I do appreciate this particular aspect). The same thing that makes Beat Saber more fun than Saber-a-bunch-of-boxes-on-your-own-schedule can genuinely also be applied to giant piles of tasks.

The fact that this game has lasted a year in my life and I come back to it with verve points to it not being an enemy to any major part of myself.

Another promising way of seeing this game is that this structure lets you see more clearly the true importance of each spent minute, when you were by default in error. Whereas for instance playing Civ IV for five minutes every time you do work (another sometimes way-of-being of mine) is less like causing yourself to perceive reality truly and more like trying to build an alternate incentive structure out of your mistaken perception, that adds up to rational behavior in the real world.

If anyone else tries this, I’m curious to hear how it goes. My above explanation of its merit suggests it might be of broad value. But I also know that perhaps nobody in the world likes organizing things into little boxes as much as I do, so that could also be the main thing going on.

*** (Crossposted from world spirit sock puppet)

[worldlypositions]

Podcasts on surveys, slower AI, AI arguments

I recently talked to Michael Trazzi for his podcast, The Inside View. It just came out, so if that’s a conversation you want to sit in on, do so here:

The main topics were the survey of ML folk I recently ran, and my thoughts on moving more slowly on potentially world-threatening AI research (which is to say, AI research in general, according to the median surveyed ML researcher…). I also bet him a thousand dollars to his hundred that AI would not make blogging way more efficient in two years, if I recall. (I forget the exact terms, and there’s no way I’m listening to myself talk for that long to find out. If anyone else learns, I’m curious what I agreed to.)

For completeness of podcast reporting: I forgot to mention that I also talked to Daniel Filan on AXRP, like a year ago. In other old news, I am opposed to the vibe of time-sensitivity often implicit in the public conversation.

(Crossposted from worldspiritsockpuppet.com)

[worldlypositions]

Bernal Heights: acquisition of a bicycle

The measure of a good bicycle, according to me, is that you can't ride it without opening your mouth in joy and occasionally exclaiming things like 'fuck yeah bicycle’. This is an idiosyncratic spec, and I had no reason to think that it might be fulfilled by any electric bicycle—a genre I was new to—so while I intended to maybe search through every electric bicycle in The New Wheel for one that produced irrepressible, physically manifest joy, I also expected this to likely fail, and to be meanwhile embarrassingly inexplicable and irritating to bike shop employees—people who often expect one’s regard for bikes to map fairly well to facts about their price, frame shape, and whether the gears are Shimano. But several bikes in, when I uncomfortably explained to a guy there that, while the ones I had tried so far were nice, bicycles had been known to make me, like, very happy, he said of course we should find the bicycle that I loved. So I at least felt somewhat supported in my ongoing disruption of his colleague’s afternoon, with requests that bicycle after bicycle be brought out for me to pedal around the streets of Bernal Heights. The guy would maneuver each bike out of the crowded shop and to the sidewalk, and adjust it to fit me, and we would chat, often about his suggestion that I maybe ride up to the hill on the other side of the main road. Which I would agree might be a good idea, before riding off, deciding that turning left was too hard, and heading in the other direction, through back streets and around a swooping circle park with a big ring road, where I would loop a few times if the mood took me. Some bicycles were heavy, and rode like refrigerators. Most bicycles were unsteady, and urged even my cycling-seasoned bottom to the seat while pedaling. Most bicycles added considerable assistance to going up hills. Many bicycles seemed fine. Bernal Heights, on the other hand, seemed awesome. As I paused before my habitual turn-in-the-wrong-direction one time, the house kitty-corner to me was playing music louder than I recall ever hearing anything play music that wasn’t a large-concert speaker. It was truly not considerate. And a middle-aged guy on my corner was having a great time, laughing, and was like, ‘Welcome to Cortland Avenue’. I pulled up and said that I hadn’t been here before actually, and didn’t know what he was talking about. He explained that Cortland Avenue was some kind of peaceful and placid place, and that they could use more of whatever this was. The whole street felt old-fashioned-okay somehow, and not really like my sense of modern America. I wanted to say it was a bit like the 80s in Hobart (the capital city of Tasmania, where I grew up) but since I’ve barely experienced the 80s in Hobart, I probably shouldn’t say that. Nothing shiny, nothing preying. Yellow things, sincere things, people who care about rock music, people who bought some vegetables, people talking to friends in streets and outdoor restaurants. So many electric bicycles—do I just not notice all the electric bicycles when I’m not on one? On one outing from the shop, a woman called out to me to say that she had also tried the bicycle that I was now riding, the other day, and wasn’t it good? I pulled up to tell her that I was actually struggling to find the ‘on’ switch, and she showed it to me. (To be clear, I converse with strangers in streets quite rarely in my normal life about three miles away.) I got a headache from all the bike-trying, and requested a lunch pause. Then I explored further down the street, and found a dineresque crêperie. I was practicing making imperfect choices fast, so in an uncharacteristic snap of decision I went in to get a lemon sugar crepe (which is only a potentially imperfect choice on axes other than deliciousness). The place was some kind of institution, and the man behind the counter seemed to be savoring the motions of crepe-provision. I had fun ordering, and sat outside. It was so nice there that I repeatedly tried to photograph it, but it wasn’t a kind of niceness that my phone could capture it seemed. Perhaps the fact that I was sitting in the street and didn’t look mildly distressed would convey something to an experienced viewer. Back at the bike shop, I had a bike in mind for if no amazing bike materialized, and continued working through the tail of the bike options. Then there was an amazing bike. There was not much visual foreshadowing of this: it was an unsleek thing, painted in an impure grey with questionable red highlights. But it felt like freedom. I could stand up on it. It moved as an extension of my body. An extension full of energy and exhilaration. My smile became round with delight and I swore gleefully. I rocketed up steep streets and to the circle park. I flew around it, elated, bumping over speed bumps, pedaling passionately around the upward side and flying down around the down. Then after quite a relatively long investigation into a bike for which there was no actual open question, I made my way back to the shop. I said I’d buy it. They looked at their records, and their back storage, and their records and determined that they didn’t have one to sell. This was the floor bike, for trying, and not to be sold. Though they had a slightly bigger red one to sell. They carefully measured me, with a platform and a springed thing between my thighs and such, and determined that the red one was actually the right size for me, and the one I had ridden was too small. I wasn’t meant to ride the red one because it was a new bike for selling, not a floor model. But they would let me take it out a little anyway. It was nice. Was it as nice as the other one? I didn’t know—it seemed maybe less nice, but also now that I was obsessively paying attention to signs of ineffable goodness, and worried, I was probably just having less fun, no fault of the bike. It was basically the same as my perfect bike, but the right size, and possible to buy, and more beautiful, and not obviously less awesome, so probably I should get it and stop engaging in such fun-dampening neurosis. I went back. Then it occurred to me that I could still try that grey bike one more time. I did. It was awesome. It seemed obviously better than the red bike. It didn’t matter if I was caught in some tangle of neuroses: such joy would not be smothered. I stopped by the road and relayed my problem by text to my boyfriend, who wisely started googling for other stores that might have such a bike. Then I took photographs of the bike from all sides in the sun by the park. Then my phone with connectivity died. (For reasons to do with my own forgetfulness re phone charging and complications of phone plans, I had brought two phones: one with power, and one with connectivity.) I rode around and mentally rehearsed purchasing the floor bike. Did they need a floor bike for which they have no actual corresponding salable bikes? I’d pay as much as for a new bike. I’d pay more. They would be astonished and grateful. I’d talk to the manager, who would be free to disregard floor bike protocol, for such an exceptional case. I went back to the store. No, they would not sell me the floor bike. It didn’t belong to them. I could buy it in months, when floor bikes get replaced or something. I was also told: don’t do that—I hadn’t seen what the customers did to floor bikes. (What could customers possibly be doing to floor bikes to warrant such fear?) ‘Months’ was also about how long it would take them to order in a new bike. They let me use their wifi, and I reached my boyfriend again with my charged phone, and he had actually phoned a bunch of bike shops, like some kind of hero (or some kind of superhero with the ability to just talk to people in shops on the phone—if I phoned a bike shop, they might say something like “phh shu anganga mph ghe?” and I’d say I couldn’t hear them, but they wouldn’t hear me, and we’d go back and forth like that a few times, until it became too embarrassing to be borne). He had located a couple of very similar bikes, possibly one of them identical, at other bike shops in San Francisco and nearby Berkeley. I decided to go home and charge my phone, and so ended my and The New Wheel’s long afternoon together. At home, I charged my phone and acknowledged my failings re phone charging, and bravely acted on my boyfriend’s claim that it would be reasonable to just phone the most promising bike shop back to check it really was the same bike they had, before spending over an hour driving to Berkeley. It also became apparent that my other boyfriend would not hate taking me on a long bicycle-pursuing excursion in his car that evening. So we set out, me feeling kind of defensive and silly, because I could have got a bike that was better on every front except for ‘ineffable greatness’ hours earlier and with a lot less bothering other people. I vaguely attempted to defend myself as we went over the bridge, but it didn’t seem very necessary, and we got on to more interesting conversation. The bike shop, it turned out, was a few doors away from a house I used to live in, between a coffee shop with romantic memories, and a bench with different romantic memories, from multiple ancient times. Stepping into the thick past, I left my boyfriend to park the car and walked up to the new shop. It was a big warmly lit warehouse room, which I didn’t remember seeing when I lived here. Friendly: a place of children in baskets and wholesome rolled-up-trouser types. I read out the string identifying my desires to a plump, friendly man: “Gazelle ultimate T10+ 46inch”. He went looking for the corresponding item amidst the central sea of handlebars and frames. He couldn’t find it. Strange. He consulted his records, and the back storage, and his records, and another bike shop man. At last, it was right there—the problem had been that the record said that the bike was ‘dust’ colored, and he understandably hadn’t considered that someone would come up with that name for my beloved bike’s reddish-grey tone. Relieved but further paranoid for the preservation of the hard-to-measure magic, I got out my photographs from earlier, and asked him if there was anything different between the photograph and this bike. There was! The other bike had had some kind of fancy suspension seat post installed. They had the same for sale, so I asked for it. I rode the bike around half the block and back, and couldn’t tell if it was amazing, but it really was a short and constrained ride, and what more could I reasonably do? I bought it. We could barely push its giant, heavy body into the back of the car, and it made an alarming cracking sound, which hopefully was just the light changing position. We took it home. Another day, I took it out for a little ride, and it was great to power up the hills of San Francisco, and shoot along the flats. It’s the kind of bike that only adds power when you pedal, so it seemed that riding was still a lot of exercise, but your exertions got you all around the city, instead of half way up the nearest hill. And while I just meant to have a little ride, I went further around the city than I may have ever been on a single outing. What had been an intractable country of mountainous slopes and distances and intersections like war zones was shrunk to my scale. And I felt safer, though faster, because usually getting out of the way of things requires my own feeble strength, which might be completely overcome by starting on a little bit of a bump or something. Now I could more move when I wanted. And I could go fast enough to feel no guilt riding in the middle of the car lane, rather than the door-prone bicycle lane. I was as about as fast as cars, and nimble. I had to admit though that it wasn’t the joy I had sought so hard. It was merely mundanely good. And I was tense, and San Francisco was frightening, and cars were everywhere, and it was all exhausting. Maybe now I was just too stressed, and it would be good later? Or maybe the bike was somehow adjusted slightly wrong, and the potential for that same joy could never be found among the myriad possible positional combinations? About twenty minutes from home, I realized that I actually couldn’t leave the bike and walk without my back hurting a lot. So, questionably, I got back on it, which was bearable, and rode home. I spent the next day or so in bed. Shorter rides favored the hypothesis that it was fun but not extraordinary. I returned to the first bike shop, and asked a man there to adjust my new bike to be exactly like their floor bike, ignoring the very likely possibility that they had adjusted the floor bike since my visit. He didn’t seem to obviously understand either the situation or my request, but was willing to make some changes, and phone me when done. He also sold me an expensive lock and some neat (and expensive) panniers. I went and tried to buy crepes, but the store was closed. But then I found a cafe and a restaurant next door to each other with back patios, a recent passion of mine, and took a nice iced coffee from one to the other, where I ate fries and read about words for an hour, in the company of a cat, someone else reading a book, and some kind of raw European music. It was pretty good. My phone died, because my life is too complicated and/or I’m an idiot. I went back to the bike store. He had changed some things, such as—promisingly—the angle of the handlebars. I rode home via the circle park, detouring to fly around it, iced coffee lodged in my front gear cables. It seemed amazing. But I’m uncertain, and doesn’t that mean it wasn’t? Well I smiled a lot on the way home, anyway. *** (Crossposted from world spirit sock puppet)