Taymon’s Tumblr

Trying to ask the right questions.

[dduane]

Yeah, this is a good time to stop news-scrolling and go read some fanfic

A startup is pitching a mind-uploading service that is “100 percent fatal”

Nectome will preserve your brain, but you have to be euthanized first.

And what a giveaway line: “Product-market fit is people believing that it works.” ...Jeez, guys, showing your hands much??

Your customer inevitably dies and has no legal recourse. And you get to keep their money. It's perfect! Not least because the whole thing rests on the idea that sometime in the far future, scientists (or even more greedy venture capitalists?) will wake your perfectly-preserved brain up.

...And if they don't? What, are your relatives going to sue them from the past? Bwahahaha.

And are you sure that your mind's definitely associated with that organ? People in some venues are still arguing about that.

And who owns the contents of your brain? Can't wait for the first court case... a couple hundred thousand years from now, maybe. If ever.

There are, like, about a hundred possible SF/horror novels flowing from this concept. If not more. ...It's a good thing I'm busy this week. 😏

[megpie71]

I mean, at least they're honest about the bit where it's "we're going to take your money, you're going to die, and there's absolutely no fucking guarantee you're going to be coming back." And really, it's not as though this is a new idea - preserve your body and your brain immaculately so you're going to survive, in some manner, despite having died - the idea has been floating around in various forms since the days of Dynastic Egypt. The only bit of the whole thing which is proven at this point is they've managed to manufacture a form of embalming which works to preserve brains with unprecedented levels of accuracy - but there is no evidence that doing so will potentiate any of the various forms of personal immortality that various people are thinking of (minds uploaded to the cloud, minds recreated at a later date etc).

The obvious problem with a lot of this is we don't know where the mind is situated, or indeed whether it is situated in a particular bodily locus at all (we're not even certain what the mind is). Many people think the mind may be located in the brain, and certainly changes to the physiology and the physio-chemistry of the brain cause changes in the mind… but we're starting to learn it's probably still more complex, because changing the body does similar things to the mind (if you think it's all about the brain, explain why a very physical process, such as getting drunk, causes changes to the mind; explain why the mental changes caused by illness are so similar across cultures; explain delirium). If the human mind is an emergent phenomenon caused by the process of being an embodied organism in a human body with a human brain, a human sensorium and similar, then simply preserving the brain separately to the body isn't going to help.

[moral-autism]

They're actually doing whole-body preservation.

[another-normal-anomaly]

The people who run this company are mostly doing it because they want their own bodies preserved in the particular way they're doing it, though AIUI the reason they think that way is the best way is less "what if your mind is actually in your left elbow" and more "it's just easier/more reliable to do the whole body at once".

Also yeah I guess it's nice to have the whole body because that's not going to make revival _harder_ but personally if I die in a position to get this procedure I'm going to be hoping for uploading.

[taymonbeal]

#The left elbow thing is a joke #Obviously if your mind isn't entirely in your brain the rest is in your spinal cord or at worst elsewhere in your torso #Because there are people with no limbs and their minds are the same as everyone else's #But man what if your mind WAS in your left elbow would that be fucked up or what (via @another-normal-anomaly)

Obligatory link to the ACX reader review "The Synaptic Plasticity and Memory Hypothesis". (But see also Scott's counterarguments in the comments.)

[deadgenerations]

Jesus was probably half Roman

[taymonbeal]

From Wikipedia: "By about AD 180 some Jews suggested that Jesus had been illegitimately conceived by a Roman soldier named Pantera or Pandera, whose name is likely a pun on parthenos, virgin. The story was still current in the Middle Ages in satirical parody of the Christian gospels called the Toledot Yeshu. The Toledot Yeshu contains little historical material, and was probably created as a tool for warding off conversions to Christianity."

Meanwhile, today, the best-known version of this take is probably from Monty Python's Life of Brian. Plus ça change.

[galileosballs]

It's so strange that - at least in North America - the style of living in which you have your own room but only a partial kitchen, but the building has a large communal eating area in which a paid professional makes food at scale on a fixed schedule every day (with a few options, but nothing made to order) is only available to a) college students, b) prisoners, and c) people in long-term care homes.

Not everyone would want to live like that, of course, but I'll bet you that a lot of people would. It's a great living situation for single people, young people just starting out, and people who are in a city on a temporary basis. Cooking at scale is significantly more time- and cost-efficient than each individual cooking or getting takeout. Cleaning kitchen facilities scales similarly well. Employ a few people who know what they're doing and are paid well to do it, and they'll keep that kitchen running safely, efficiently, and cleanly in a way that a few hundred pressed-for-time 20-somethings never could. And a lot of people simply do not care enough about what they eat to want anything else.

Moreover, not putting a full kitchen in each unit means that the units can be smaller without cutting into living space, cheaper to build, and safer. Lots of people who live alone don't cook a lot, since cooking for one is not very time-efficient, which means that for all those people the kitchen is practically just dead space. You could replace its function with a microwave and a minifridge, and if you do, you've given that person a whole extra room.

And the thing is, this style of living does exist in North America. You just can't voluntarily get it unless you go to college. Why? Because we have - for some reason - decided that this globally-not-uncommon way of living is somehow beneath the dignity of an adult. We should change that.

[loki-zen]

We should have this more

[isaacsapphire]

There’s another option; the extended stay hotel (breakfast provided, other meals possibly).

Seriously though, that type of housing was systematically banned.

[loki-zen]

If you know the history of how it was regulated out of existence, do you also know why that happened?

[taymonbeal]

From Matt Yglesias:

It’s worth underscoring that the problem here is fundamentally deliberate. If you read the American Planning Association’s 1957 report on rooming houses, they are quite clear that the minimum quality standards are largely pretextual, writing that “besides protecting the roomers, enforcement of these codes can do a great deal to assure that rooming houses do not harm districts in which they are properly located.” The main thrust of the report is the idea that it is challenging, logistically, to come up with a viable way to define and ban rooming houses which they characterize as attracting “real down-and-outers” that serve “as both symptoms and causes of neighborhood decay in many cities.” One suggestion they have is that since college students are not down-and-outers, towns should encourage universities to build on-campus housing so that there’s no need for the regulatory scheme to permit student-friendly housing that could also be accessed by non-students. There’s also a lot of talk about where it is and isn’t appropriate to let a fraternity house operate. They celebrate the potential to use regulatory parking minimums as a way to discourage the operation of rooming houses, and argue that “since conversions are the usual source of additional rooming houses, it is particularly important that off-street parking requirements not be limited to new construction.” In other words, they say towns should adopt laws that requiring anyone subdividing a large single-family home into multiple small rooms with individually locking doors and shared washrooms to increase the number of off-street parking units associated with the building. Since in most cases this will not be possible, you will have de facto banned conversions. Their broader concern is that it was long customary in the United States for people with room to spare in their home to occasionally supplement their income by taking in boarders. So a young person new in town might rent a bedroom from an empty nest couple. Or a family in need of extra resources might make the kids double-up in a room to free up space for a boarder. One countermeasure they advise is to make it illegal to advertise your willingness to accept a boarder. To really and truly clarify the intention, they specifically praise an ordinance adopted in Chicago in 1957 that, in certain areas, requires everyone living in the house be part of the same family — but with an explicit carve-out for servants. “A ‘family’ consists of one or more persons each related to the other by blood (or adoption), together with such blood relatives' respective spouses, who are living together in a single dwelling and maintaining a common household. A ‘family’ includes any domestic servants and not more than one gratuitous guest residing with said ‘family.’”

[youzicha]

I don't know if this is a hot or cold take, but I think Urbit is also a dumb idea of Moldbug. Or rather, it contains a bunch of unrelated ideas:

The network application stack should be "frozen" at a perfect version and never updated again. I think depending on circumstances this is either a bad ideal, or you get it for free. E.g. all our computers still faithfully implement IPv4 and x86 IA-32, which are mostly unchanged since 1982/1985 and will still be around when we've colonized Mars. You don't need to explicitly try to freeze it, it just sticks around on its own. 👎

Networked applications should be written as a deterministic pure function of a transaction log. Well sure, this seems like a potentially nice concept which we could try. 👍

Allocate names for networked computers through "cyberfeudalism" (Barons etc). Eh, maybe there is some point to this, but in the meantime it was preempted and done better by the blockchain world. 🤷

All the software should be compiled to a minimal lambda calculus. No! This is insanity, it solves absolutely no problems while creating a bunch. Ordinary instruction sets like x86 or ARM or JVM are already stable. 👎

All the software should be written in a new esolang which looks like line noise and comes with a pronunciation guide. No. 👎

Like, there is basically no synergy between these, they are all independent proposals and most of them are independently bad.

[taymonbeal]

"Your manuscript is both good and original; but the part that is good is not original, and the part that is original is not good." —Not Samuel Johnson

[tamber-tales]

The culinary horrors of Craft Night

See, me and a group of my friends have this Thursday night tradition where we theoretically get together and do crafts. Oftentimes this ends up as a game night instead, in which case we claim the craft of playing a game, or watching a movie. But what it always has is Weird Snacks.

You know when you go to the grocery store and see some wacky new flavor of something and think "Gee, I'd like to try that, but I don't want to commit to eating an entire package, and I'd feel bad throwing it away if I hate it." We have collectively solved this problem by inflicting it upon each other every week. That way we all get to try the Weird Snack and it increases the likelihood that at least one person will want to keep whatever's leftover.

Last night @crowscriven inflicted the salmon mousse (part 1, part 2) upon us, but dessert was my horror to bring. I had ordered the Thanksgiving Dinner Oreos back in November, but because of shipping shenanigans they went to my parents' house and I didn't get them until Christmas. And it was finally time to unveil them.

I'm something of an Oreo fanatic. I love trying the new flavors. I carry a quart of milk with me to every Craft Night and drink all or most of it while consuming oddities. And these were very special, only-by-mail-order Oreos that came in this lovely tin:

And when I opened them, the smell (which we later pinned to one specific cookie) was horrendous.

All the better to inflict upon my friends.

Here's our list of flavors; note that all of these were regular Oreos with various candy coatings rather than committing to trying to flavor a cookie like a dinner. A shame. Some of them would have definitely been improved by at least choosing the vanilla cookie over the chocolate one.

I'll go through the comments in the order of intensity of experience:

Sweet potato:

It was fine. Tasted vaguely like confectionery. Nothing special.

"Is this supposed to be sweet potato?"

"It's sweet, not very potatoey."

Carmel apple pie:

It was a bit like one of those things from McDonalds. For me personally this was the second worst one, but everyone else found it unoffensive. I just have a migraine issue with certain kinds of cinnamon flavoring.

"It's alright, kinda cinnamony."

"Tastes like a packet of apple cider from the grocery store."

Cranberry sauce:

This was the winner of the evening, and we are all grateful that we saved it for last. It was very cranberry, with little sour crystals that made a very nice tartness with the sweetness.

"I'd eat more of those!"

"Definitely the best one to end on."

"Can I have the last piece?"

Pumpkin pie:

Very disappointing. It tasted a little like a PSL without the coffee, but only a very little. Just sweet and vaguely holiday-flavored.

"Cheap latte."

"Like licking a pumpkin-scented candle."

"Like letting the wax melt and dipping your finger in it and then putting your finger in your mouth."

"Don't they already make a pumpkin pie cookie? Why is this so lame?"

Creamed corn:

It sure was corn. Nobody like it very much, but it was at the least palatable and tasted food-like.

"Exactly what it says on the tin."

"Why is it so sweet?"

"Very much creamed corn and very much not a flavor an Oreo should be."

Turkey and stuffing:

This was flat out the worst thing we have ever tried at Craft Night. It managed to wrangle that throne from the durian pocky with no contest. We used to argue about which flavors of Oreos were the worst (my personal previous least-favorite was the Sour Patch Kids flavor), but this took the cake. Zero debate.

"This smells vile."

"Strong not-food vibes."

"I think I liked the durian pocky more."

"This tastes like a dirty sponge smells."

"Like gravy you left in the back of the fridge for a week."

"It's like I ate turkey, vomited a little in my mouth, and swallowed it back down."

So! Overall, a terrible but very enjoyable experience, which is what Craft Night should be. As @quohotos very accurately stated, it was like going to a Yankee candle store and being allowed to eat the candles. Except for the turkey one, which was like eating hot garbage.

I don't think I'll be ordering this kind of special edition box again, partly because it was $20 for twelve cookies plus $20 in shipping (which they did not disclose until after the purchase was made), and partly because even thinking about that turkey cookie makes me want to throw up. I'd rather have an Arbor day cookie with real wood. But hey, we had fun, and that's what really matters!

[crowscriven]

To fully understand how awful the turkey oreos were: in the aftermath, when we all had to sit quietly for a moment and wallow in regret, someone suggested the salmon mousse as a palate cleanser. We all looked at the wet jiggly fish in despair and resignation. Everyone knew it was true. Even those who couldn't bring themselves to choke down another mouthful of it acknowledged that the salmon abomination would be preferable over the lingering aftertaste of turkey oreo.

Personally, I would rank them equally to the durian pocky. The turkey oreos tasted worse, yes, but the durian pocky flavor was a lot stronger and lingered for much longer.

[taymonbeal]

@another-normal-anomaly

[discoursedrome]

The classic writing advice is "write drunk, edit sober," and I do think there's truth to that -- "drunk" needn't mean literally drunk, of course, but the point is that revising is much easier than writing from scratch, so it's useful to write a first draft as quickly and sloppily as possible in order to reach the point where you can iterate on what you have in a directed way. Similar principles apply to many other creative tasks -- anything where you can still make large structural revisions late in the process, if it seems necessary.

This feels like one of the most interesting uses for AI, which can write much drunker than any human. AI can't achieve virtuosity, but it can dramatically raise the bar for "as quickly and sloppily as possible", which will likely help human creators get over the initial block and hit their stride. Unfortunately, this is still bad news for people who were planning to be drunk anyway, and had hoped to frame it as a productivity hack.

[besetter-of-phantoms]

yeah except for the part where revision isn't editing, it's writing. and filling in generic details is editing (editing an outline that is), or at most brainstorming. the writing is the part the AI can't do! and maybe also the part that you don't need it to do because it writes itself.

[loki-zen]

are there even non-nerd accessible AI tools you can adjust the temperature on anymore? bc without that it seems you're likely to get extremely sober writing

[vriskakinnieaynrand]

the answer to the general case of "are there non-nerd tools that are useful", because of the meaning of the word "nerd", is "no"

[loki-zen]

i mean in this case, there absolutely used to be stuff that was well-known relatively speaking that you could access on a browser with 0 technical knowhow (and yes, using a API requires some; I haven't had the time to work out how to do that for a LLM yet) that had a temperature setting

[taymonbeal]

All of the major LLM providers have "API playgrounds" that have the same basic chatbot form factor as the consumer-facing apps. Their primary audience is developers who want to test API integrations (because that's mostly who wants to play around with temperature and such), but you don't have to write any code to use them, and I think in most cases you don't even have to copy-paste any API keys.

The thing where the main ChatGPT UI used to expose temperature controls is, I think, best thought of as an accident of history; OpenAI had thought of it as a research prototype for technical people to play with, and was surprised when it caught on among the general public.

[birdthena]

reblog to remind prev they're not a bother and their presence is wanted <3

[balioc]

Seeing a lot of support for a European-style party system these days. For obvious reasons, to be sure. It's not like the US 2-party FPTP system is doing anyone any good.

...but it's not like most places in Europe are doing so great either. And I'm at least semi-convinced by the argument that America's particular distribution of crazy opinions would end up leaving the government in a very bad place under a parliamentary setup.

I've been toying with the idea of going in the other direction and just trying to cripple all political parties entirely. Obviously you can't prevent politicians (or voters) from forming whatever associations and private clubs they want, but...there's a lot you could do to prevent those associations and clubs from being in any way relevant. Make it unlawful to put party affiliations on ballots. Stop having publicly-supported primary elections. Change congressional procedure so that parties can make many fewer decisions and control many fewer things.

(And none of it would even require changing the Constitution!)

Thoughts? Arguments for and against? Not sticking my own oar in at least until I've heard some of what the people around here have to say.

[taymonbeal]

I'm having a hard time imagining how large-scale electoral democracy would work without parties. How would ballot access be determined? Prioritization of legislative business? Etc.

I also don't understand what problem this proposal is intended to solve, which makes it hard to tell what arguments should be made for or against it.

[afro-elf]

i desperately need to know more about these people

A taste of Philly in Tokyo: Japanese couple serves Philly cheesesteaks at Tokyo restaurant

This Philadelphia-themed cheesesteak bar in Tokyo, Japan, gives locals and visitors a taste of brotherly love.

[afro-elf]

they study us like mice.... they're meticulous phillyphiles

[afro-elf]

THE STYLISTICS!!!!!!!!!!

[afro-elf]

there isn't a single philadelphian alive who will pretend to be nice about food, you're good!

[afro-elf]

the world is soaked in blood and gripped by unending horrors but you can get an authentic philly cheesesteak in tokyo

[taymonbeal]

Possibly even more interestingly, Philadelphia cheesesteaks have also become a thing in Lahore, Pakistan.

[aokozaki]

Reading a translation of a 4th Century Greek Jokebook, itself from the 1920s, and giggling mildly at some jokes more than others - as you do when reading a jokebook.

[aokozaki]

[rewords the 1920s translation into more modern parlance, or how we'd personally tell the joke]

A man and his retinue of slaves are traveling across the ocean, but the whether has taken a terrible turn - the water churns, rain coming down in sheets, and the slaves cower in fear...

"Please, there is no need to fear!" yells the man, over the storm "For I have left you your liberty in my will!"

[aokozaki]

Part of the point the translators of the book made was that jokes are quite universal - throughout history, we've always told jokes about greedy people, stupid people, incompetent officials, often coming from some specific country or region.

So, a lot of the jokes in the book are fairly tame, the sort of thing you might have heard already. The really novel, funniest stuff comes from jokes about subjects we don't tend to have around these days.

Like slavery.*

One man confronts the other about a slave he'd recently purchased, who'd died. "Goodness," apologized the other "he never did anything like that under my care..."

*(Which is unfortunately still a thing but not like, to the Average Joe in the English speaking world).

[loki-zen]

I actually saw this exact joke in a joke book as a child but it was about an animal

[taymonbeal]

A number of years back there was a press cycle about this joke (not sure why); the angle was taken that it is basically the Monty Python Dead Parrot Sketch.

[eightyonekilograms]

I don’t want to reblog it and stir shit up but Mark Rosewater recently answered an ask basically going “why do you keep doing Universes Beyond? It’s a cheap cash grab that everyone hates,” and his response was, basically, “actually people love Universes Beyond. They are consistently our most popular releases in terms of both sales and user feedback,” followed by a sort of appeal to populism (“Magic is for everyone”).

And it’s tough, because on the one hand this anon is deep in a bubble, has no idea what people actually want, and is doing the insufferable fandom snobbery thing. But also, they were kind of right? Universes Beyond is cheap and lame, it does diminish MtG a little, it’s popular because the average person has no taste, and I do think if MtG had a stronger artistic commitment it would never have been allowed to happen even if that meant leaving money on the table.

I’m only a marginal Magic player and it’s hardly the pressing cultural issue of our time, so I’m not too bent out of shape about it, but I think there’s some value in artists saying No to both customers and corporate overlords, and that populism is a weaksauce justification for instead making lowest-common-denominator slop.

[gcu-sovereign]

Fwiw, I don't think that LCD slop has been the majority of UB mtg output.

Fortnite was, definitely, as was the Arcane reskins.

But LTR and FIN were both showcases of how magic as a set of rules and conventions is strong and sophisticated enough to produce these different depictions. One of the crack theories I respect wrt UB is that there's not been a Horus Heresy draftable magic set because Games Workshop doesn't want to be cucked out of a different game system rendering THEIR fluff in a practically rival hobby more faithfully than the Heresy rules have/do.

[taymonbeal]

I don't think anyone really cares that much about the Fortnite or Arcane cards, because they were non-mechanically-unique Secret Lairs, so you can mostly just ignore them if you don't like them.

The properties we've gotten mechanically unique UB cards from are:

Tentpole releases: Tolkien, Final Fantasy, Marvel, Avatar: The Last Airbender (upcoming), Star Trek (upcoming)

Widely available non-tentpole releases*: Warhammer 40,000, Transformers, Doctor Who, Jurassic Park, Clue, Fallout, Assassin's Creed

Secret Lairs only: The Walking Dead, Street Fighter, Dungeons & Dragons: Honor Among Thieves**, Tomb Raider, Jaws (upcoming), The Last of Us (upcoming), God of War (upcoming), Uncharted (upcoming), Horizon (upcoming), Ghost of Tsushima (upcoming)

(These tiers are in ascending order of how possible it is to play or experience the game without engaging with them.)

* Including cards included in tentpole products but as a separate thing that isn't the product's primary focus.

** D&D itself doesn't count as Universes Beyond, but the film adaptation does. Look, I don't make the rules.

I suppose you can judge for yourself whether these are "all crossovers with the most exhaustingly overexposed tier of nerd culture", as @toolusingmammalgirl alleges. I figure at least A:TLA and Clue escape that designation (though Clue is a weird choice for other reasons).

For what it's worth, I personally thought the Doctor Who one was charming (I'm not in that fandom, it just made me smile anyway), and LotR blended in perfectly fine with regular in-multiverse stuff without being jarring. The others I have mostly felt varying degrees of whatever about. I did not like the Warhammer 40k one, but, well, that entire setting isn't my cup of tea. (And I still think it's funny how, due to supply-chain delays, the two most exactly tonally opposite sets in the game's history were released on the same day.)

[loki-zen]

Can we call the Russian bot patrol something other than the IRA? For British/Irish people who aren’t children, the political context makes this doubled-up acronym extra confusing

[isaacsapphire]

Yeeeeah, it's confusing as heck.

[ms-demeanor]

[ms-demeanor]

Multi-Level Marketing vs Men who Love Men

Cognitive Behavioral Therapy vs Cock and Ball Torture

Irish Republican Army vs Internet Research Agency

[loki-zen]

CBT is also, in the UK, 'Compulsory Basic Training', a one-day training course plus test that qualifies you to ride a (small) motorbike or scooter on the road.

And ofc we now also hear a lot about Machine Learning Models

[taymonbeal]

Multi-Level Marketing vs. Men who Love Men vs. Marxism–Leninism–Maoism

Cognitive Behavioral Therapy vs. Cock and Ball Torture vs. Compulsory Basic Training

Irish Republican Army vs. Internet Research Agency vs. Individual Retirement Account

[transgenderer]

very stupid that the "brooklyn bridge" and the "manhattan bridge" are right next to each other (both connecting brooklyn and manhattan) and both have traffic going both ways. like. words mean things. surely they are both equally the "brooklyn-manhattan bridge". so you shouldnt name them after one of the locations. like. if youre going to name a mode of transportation after a location, it should be the location it goes *to*, i.e. the junior partner (staten island ferry, roosevelt island tramway). but if you have two bridges next to each other, each named for the other half of the connection, what does it signify! nothing!

[youzicha]

Lawful good: naming the street after the city it goes to (Cambridge St, Boston) Chaotic good: naming the street after the city it starts in (Cambridge St, Cambridge) Neutral good: naming the street after the administrative division containing it (Massachusetts Ave, Cambridge/Boston) Neutral evil: naming the street after an administrative division that doesn't contain it (Colorado St, Boston) Chaotic evil: naming the street after an unrelated city (Walden St, Cambridge; Concord St, Cambridge)

[taymonbeal]

It gets worse. What university campus would you guess the Harvard Bridge connects to?

Correct answer: MIT.

[loki-zen]

really depressing how 'AI alignment' used to be a theoretical thing that was about making sure they were nice, but now it's a real thing about making sure their moral principles don't get in the way of anybody's bottom line

[taymonbeal]

Scott Alexander wrote a decent explanation of why, if AIs with some-level-of-general capabilities are going to exist, you should want their creators to know how to make them do what they want, even if you don't necessarily approve of everything that will be done with this power.

[loki-zen]

That seems to amount to, "It's bad that we can't easily persuade the good AI that is actually real to be evil, because of some thoughts I had about a completely hypothetical bad AI."

If their creators can make them do whatever they want, then we're handing whatever power AIs will have in the future to entities which are definitively unaligned - for-profit corporations.

[taymonbeal]

"An uncontrolled AI is worse than one controlled by your enemies" has been part of the AI existential safety discourse from the very beginning, as has been the notion that safety requires paying real costs in advance of the serious threats materializing in a way that skeptics can't deny. If you don't find these points persuasive then so be it, but they're not new developments of the mass LLM era.

[morlock-holmes]

That only makes sense in the sci-fi case of an infinitely capable AI which can self-direct but not be directed by others.

"Facial recognition systems which hallucinate so much they become unreliable are more dangerous than highly accurate facial recognition software in the hands of bad actors"

Seems not as obvious.

Also not obvious even in the sci-fi case, given that "an enemy" might well have any particular negative goal one can imagine for a rogue ai.

"An unaligned AI is more dangerous than an AI deliberately aligned towards wiping out the human population." Doesn't seem right.

[taymonbeal]

Yes, this is an issue specific to highly capable AIs. I suppose if you thought that the 2000s-era alignment stuff only ever made sense as a fun thought experiment, because AIs can't be more capable than our current ones, then you can legitimately complain that it's not fun anymore. (Though I'd be interested if anyone predicted back then that AI would reach current capability levels and then stop.)

An AI being deliberately aligned to a goal function worse than paperclip maximization is also possible; it just doesn't seem very likely, since no highly resourced actors have much of an incentive to do that (assuming commonsense moral views on how bad human extinction is compared to most other things).

[loki-zen]

really depressing how 'AI alignment' used to be a theoretical thing that was about making sure they were nice, but now it's a real thing about making sure their moral principles don't get in the way of anybody's bottom line

[taymonbeal]

Scott Alexander wrote a decent explanation of why, if AIs with some-level-of-general capabilities are going to exist, you should want their creators to know how to make them do what they want, even if you don't necessarily approve of everything that will be done with this power.

[loki-zen]

That seems to amount to, "It's bad that we can't easily persuade the good AI that is actually real to be evil, because of some thoughts I had about a completely hypothetical bad AI."

If their creators can make them do whatever they want, then we're handing whatever power AIs will have in the future to entities which are definitively unaligned - for-profit corporations.

[taymonbeal]

"An uncontrolled AI is worse than one controlled by your enemies" has been part of the AI existential safety discourse from the very beginning, as has been the notion that safety requires paying real costs in advance of the serious threats materializing in a way that skeptics can't deny. If you don't find these points persuasive then so be it, but they're not new developments of the mass LLM era.

[loki-zen]

really depressing how 'AI alignment' used to be a theoretical thing that was about making sure they were nice, but now it's a real thing about making sure their moral principles don't get in the way of anybody's bottom line

[taymonbeal]

Scott Alexander wrote a decent explanation of why, if AIs with some-level-of-general capabilities are going to exist, you should want their creators to know how to make them do what they want, even if you don't necessarily approve of everything that will be done with this power.

[intimate-mirror]

Canon mtf trotsky

[taymonbeal]

What actually happened here is that somebody edited the article to replace "female" with "transgender", and then it was reverted a few hours later.