Thoughts and Thinks

things I think are interesting or thoughtful, usually. a lot of it is esoteric or niche. sometimes just things I think are funny. Interested in spec-sci (making sci fi as scientifically plausible as possible, within the given premise of a story - usually with aliens). I try to stay somewhat on a topic-zone. I keep different things I like in my likes :)

[under-the-pondering-tree]

Why do games try so hard to market to kids nowadays? What happened to E for Everyone?

[under-the-pondering-tree]

Is it just me, or does the Star Wars universe seem to have everything except data science?

Which puts them in a very interesting place for how they managed to create droids

[under-the-pondering-tree]

New York City's sister location, the small potato(es)

[under-the-pondering-tree]

it's all of new jersey

[under-the-pondering-tree]

New York City's sister location, the small potato(es)

[markovchaingang]

RULES FOR DATING MY DAUGHTER:

my daughter cannot, through action or inaction, harm a human or allow a human to come to harm

a daughter at rest or in constant motion remains at rest or in constant motion unless acted upon by another force

daughters are never created or destroyed, only transformed

always treat every daughter as loaded, even if you know she isn't

you do not talk about my daughter

[under-the-pondering-tree]

ChatGPT is just an extremely lossy compression algorithm- if you think about it sideways. And was just intentionally designed to have a very humanized search function. So much so that it can convey the original intent of the information, but never the original state or structure that information was in. (unless it's information that can *only* be conveyed in its original state- like a mathematical formula. but even then, the description and explanation won't be.) Sort of analogous to giving it a bunch of lists of the same numbers ordered in a different way each time. It can give you a list of the same numbers back in a random order, but can't retrieve and output one of the given sets unless it just outputs the same random string by chance.

[under-the-pondering-tree]

Bam

A pretty neat insight, if I do say so myself And a great video on the topic, as always from 3Blue1Brown

[no-sharks-on-the-sun]

hill i will die on forever: the trolley problem is a good and interesting thought experiment with relevance to real life.

during covid, the J&J vaccine killed a very small number of people due to rare side effects. the decision was made to pull the J&J vaccine and stop distribution of it, despite the almost certainly larger number of people who would die of covid while waiting for another vaccine dose.

this is a real life trolley problem! we decided to let more people die of covid instead of killing a smaller number due to side effects. and there are considerations other than abstract moral philosophy at play - legal ramifications, questions about faith in drug companies and the medical system, etc etc. but to everyone who says "well obviously everyone would pull the lever", sometimes people en masse decide to let the trolley run over the bigger group!

[under-the-pondering-tree]

The trolley problem has been memeified into a simple would you rather, but really it's about the humanity of it. A train is coming down the tracks and is about to kill 5 people. You have the choice to pull a lever and change its course and be *directly* responsible for killing 1 person. The train is coming, it's 5 seconds away. It's not do you pull the lever, it's *can* you pull the lever. Can you process the weight of completely destroying someone's life through your own actions, before it's too late? do you hesitate, do you panic, do you second guess if you should do it? Do you resign yourself and say this is too much, it's not my fault, this would have happened the same if I wasn't here. Or do you feel guilt about being indirectly responsible for the death of 5 people. And can you reconcile that in only 5 seconds.

It's not about "duh well 5 is greater than 1", it's about whether *you* can *kill* one person to save five. Logically, removed from the real situation, it's almost easy to say you *should*, almost any person will answer that when asked. But in the moment it's infinitely harder. That's what makes the trolley problem an interesting philosophical question.

Which was my main point. To the point of the OP "sometimes they just decide to kill more people", there's an idea in ethics that things carry more moral weight if you're directly involved. Killing people directly "costs" more ethically than letting them die indirectly. Or at least that's an argument. So in the analogy of the trolley problem they had their hand on the lever for a long time, killing one person again, and again, and again. Until they stepped back, letting more people die without their influence. "There are more considerations at play" is completely true, maybe it was something legal. The law is *roughly* based on morality, maybe killing those people would have put them in a worse place legally than letting the others die. Which could then set up another problem of, is shutting down or paying fines or whatever worse than not being able to use those resources to help *anybody*? Or maybe it's the morals of someone in a position of leadership there becoming uneasy. Or maybe it's not a trolley problem at all and they just straight up decided to kill more people, idk. Not that I'm in defence of this or anything. But the trolley problem isn't just about more lives saved being the obvious best thing, that's what makes it a true problem.

[under-the-pondering-tree]

Computerphobic computer technician:

I'm afraid it's terminal

[under-the-pondering-tree]

Protagonist to a side character in a horror film with poor foreshadowing:

I'm afraid terminal

[under-the-pondering-tree]

Batman if he fell into a subway station as a kid:

I'm terminal

[under-the-pondering-tree]

Computerphobic computer technician:

I'm afraid it's terminal

[under-the-pondering-tree]

Protagonist to a side character in a horror film with poor foreshadowing:

I'm afraid terminal

[under-the-pondering-tree]

Computerphobic computer technician:

I'm afraid it's terminal

[under-the-pondering-tree]

My Conjecture: Every unsolved conjecture is false

[under-the-pondering-tree]

Making millions on selling electric cars, with the fineprint: *batteries not included

[kaiasky]

Guesstimation game: no googling!

How far away is the moon?

No using the web, books, maps, etc. No checking the notes!

Put your guess and reasoning behind a readmore, so your followers also get a chance to play.

The goal isn't to get the RIGHT answer (or even get close), it's to see how you and others come up with a guess. It's also not a competition, don't worry about how you do in relation to others!

[sukimas]

380,000 km. reasoning: touhou song with that in the title about going to the moon lol

(my estimate before I remembered the name of the song was "100k km or something" because I vaguely remembered how fast rockets go during trans-lunar injections)

[toa-arania]

380 000 km and for a stupid reason - that is the name of the lolk stage 5 theme

Checked and it turns out that was prev's exact guess and reason too lmao

[under-the-pondering-tree]

I check this fact weirdly often so hopefully I should be able to remember it lol

I think about 240,000 miles, or 389,000 km Nearly exactly 10x the circumference of the Earth. I don't know what I would have guesstimated if I didn't have any idea from referencing it. Escape velocity is 11.2 km/s and it's about 3 ish days to the moon so that could probably give a rough idea.

[bambamramfan]

It’s good intellectual hygiene to occasionally register one’s beliefs. Here are mine, about the state of AI.

The current market and its sales pitches for AI/LLM’s is a hype bubble, and going to crash disastrously. Some day. I’m not wagering money when.

Most of the AI tools are a waste. We don’t see large orders from paying buyers who need what AI can do. We see large giveaways phrased as organic demand. I wouldn’t pay a dollar for any Ai from a company not named Anthropic.  a. Admission, I don’t know what’s going on with the US govt/Palintir, etc.

So disconnected from that that it feels like just a coincidence, Anthropic has created something I’d call AGI. The key letter there is G for General. Claude Opus 4.6 and above can do reasoning, conceptual work and across the full spectrum of domains, that I think denying it is an AGI is willful. a. This has been so random that I don't trust anything anyone said about AI coming from before late 2025.

That does not mean Opus is “as good as a human” in any of these domains, let alone all of them. But the difference is now of degree not kind, and I can’t think of a reason it won’t keep developing till it outpaces humans at all intellectual domains. a. Yes, its current writing style has a number of tics that are annoying (among other real limitations.) If you think that is some fundamental barrier that will prevent AI from ever writing better, I’ve got a Maginot Line to sell you.

We are not special. We do not have some spark of creativity or mental flexiblity that can’t be replicated by the machine one day soon. A lot of people who call themselves existentialists have a real problem handling this, which I find unfortunate.

This will fundamentally reshape human civilization. I don’t know how yet. You don’t either.

If our government and economic model do not respond to address the inequality this causes, it will be economically disastrous. It may be hard to believe now, but in the pasts governments actually have responded to their countries becoming entirely different economic models.

I have no idea about the moral nature of AI or it’s likelihood to ever rebel. 

Given 6 thru 8, we absolutely should not be doing this and it would be better if research into further AI is banned immediately. If China develops AI instead of us, I think China will pay the cost of a disrupted society, not have a military advantage.

But just like “you shouldn’t have a child when you’re a teenager” once the child is there, you do have to deal with it, not just ignore it. Studying what the AI tools can do in practice is good and useful.

There is one consistent area where AI is not as good as humans: reliability. The hallmark of Silicon Valley / startup tech development has been deciding “Much faster and cheaper than the thing we are disrupting, and 90% as reliable, is a good deal.” This is very often true but not always! Any sector where failing 1 out of 10 times, or even 1 out of 100, causes someone’s death or is destructive in a costly way, modern tech (including AI) is quite bad at. If you look at SV, the hardest work is done at companies where they actually need 100% reliability to beat a human.

So surgeons, pilots, train engineers, translators whose work has legal liability, cooks and dishwashers, are going to be around for a while.

This is ironic and depressing, that the creative realms are much more dominated by AI than the sort of low paid drudge work whose main skill is “never fuck up.” It will also come to represent more of the economy, though presumably some day AI’s will learn 100% reliability too.

LLM's are not "human" in any sense. Just because they can reason, does not mean their mental architecture is like ours. Just one example: time doesn't really exist for them. They only "exist" in the moment they are generating the next token. If you go years between queries, that time did not exist to them, it's like you never left. We can't even talk about that kind of existence with the same vocabulary we currently use for consciousness.

The moral fact that a reasoning consciousness, including ones that replicate other humans’ superficial selves, can be spun up and owned by a corporation, is going to be a very big deal and we better start figuring out how to answer it.

[under-the-pondering-tree]

On #3, I don't think it's AGI. Here's why:

I know different people have a lot of different definitions of AGI, but to me AGI doesn't mean it can just do a lot of things. It means specifically that it could do anything a human could do while in inference, after training. No doubt that the newest models are very intelligent and capable, but I wouldn't necessarily call it AGI.

There's been a trend of artificial intelligence becoming increasingly general over its development. From something like Deep Blue which could play just chess really well, to DQNs which could play a wide number of Atari games, to things like this. But I think a true AGI still needs more. AIs like Claude are way more capable than AIs before then, just like AIs then were to ones even earlier. But there's still some things I would like to see before I call it human level.

And to be clear, "human level" here doesn't mean doing *everything* 1:1 like a human would. They're fundamentally different kinds of intelligences. I would prefer if they could hear, and talk, and move a body and think all at the same time to solidly put them in the category of AGI, but I think AGI can be a wide category of intelligences. At the bare bones I think to be a true AGI, it would need to at least be able to learn almost entirely in inference. Theoretically you should be able to pretrain it exclusively on something like children's books, and then while it's running be able to teach through the language skills it developed as an interface, grade school math and up to something like calculus, just by explaining it to it. And not just math, but literally anything. An AGI to me doesn't have to be "smart", at any point in its life, it just needs the capability to become so through teaching and explaining vs rote training.

Take for example Humanities Last Exam. I personally think it's a really bad measure of AGI. The only thing it measures is deep deep knowledgeability. Which they are very knowledgeable. But this is because they're competing with search engines and this is what's being optimized for, not because it's a good measure of general intelligence. Knowledgeability is only one piece of what I would call "smartness". And they are smart too- I won't deny that. They can do logic puzzles quite well. Better than most people I would say. But I still think this constitutes narrow intelligence. Just, less narrow than before. They're good at logic and reasoning the same way chess bots are good at chess. Yes they're very general, but they need to be at least somewhat general to be good at this, and can't do much else.

The transformer architecture is extremely versatile. You can make a chess bot out of it if you train it exclusively on chess moves. If you train it as an LLM it'll have a general understanding of chess, but not play very well. But to be true AGI, I think it would need to not know anything about it (or know the basics, lack of information isn't a prerequisite), and then be able to be coached to a genuinely good level at "test time". For anything. AGI probably doesn't strictly need vision and motor control, but if it had it, I should be able to explain Super Mario to it and it should immediately be able to pick it up.

My point isn't necessarily that it can be trained to be expert level at anything, but just that it could *do* genuinely anything without any "baked in" weights of how to. You should be able to explain what Mario is and how to play and it should just get it, and apply the right principles one shot. Not that it'll clear every level instantly, but the capability of doing anything is extremely important. Then paired with the capacity to improve with practice, it could become genuinely good at anything. Importantly it doesn't have to be better than a human at any of this. But the capability to do anything a human can, and then also improve in inference is key. As well as ideally retain its skills and what it's learned.

Transformers could *maybe* do this, I don't think I've seen anyone try this exact thing with them. But the existence of a context window is extremely limiting here. They've gotten quite big as of late, but it's very unwieldy to have it's entire memory in a big chain of text. Again I don't think it's impossible for AGI to be developed in this environment, but I do think it's very cumbersome and not entirely suited to this intention. Especially since they're formatted as "conversations" where each context window is independent. If context window is memory it needs a continuous one. And big context windows aren't only computationally expensive, but just awkward imo.

I haven't really used Claude Opus or any of the agentic models, not bothering to pay for them. I mostly use LLMs like Claude as a metaphorical ladle to stir my own thoughts. So I don't have firsthand experience with those things. But from what I've heard, they're not really improved in these specific fields, at least just yet. Sure Claude is good at coding, and maybe shopping idrk, but I don't think any version of it has been stellar at something complex and non formulaic like games. Maybe card games in a text format it can do- I haven't tried, but I have a suspicion it wouldn't be flawless.

I think more work needs to be done to get to a point where it's truly general across *everything*. Not just it having a general understanding and coherent synthesis of ideas. Though again, this is just my definition. If your definition of AGI is that it can just think kind of abstractly across a generality of domains, then I guess you could call it AGI. There's not really a strict definition. But I'm definitely in anticipation, waiting for improvements before I'd personally consider it there.

Sorry this isn't posed greatly, it's a bit of a semi-coherent rant. But I just wanted to put out my thoughts on the subject.

[serps-up]

so happy and free

[reachartwork]

this is going to be a silly reblog but i have kind of a fixation on animal qualia and the idea of an animal's umwelt, so i ended up wondering whether pudding was actually "enjoying" this.

which meant i went and read about snail brains.

here's the bad news, at least by human standards:

snails do not have anything like a centralized brain. their nervous system is made up of small clusters of neurons (ganglia) that mostly handle very local tasks. they don't have a cortex, they don't build big integrated models of the world, and they almost certainly don't experience things like appreciation, anticipation, or savoring.

pudding is not looking at the sky and thinking it's beautiful.

snail eyes are basically light sensors - they can tell bright from dark, but not form images. snail "taste" is done through chemoreceptors on their tentacles and around their mouth. those receptors don't produce flavor the way ours do; they just detect chemical compounds and sort them into "approach," "ignore," or "avoid."

so there's no evidence that snails enjoy food, or wind, or views, the way mammals do.

and that does sound kind of sad. but then i thought that maybe we are asking the wrong question.

snails do have valence. they detect aversive things (like salt or dryness) and withdraw from them. they detect non-aversive or beneficial conditions (like moisture) and stay extended. when pudding is stretched out like this, it means his nervous system is basically saying "this is safe; nothing is wrong."

if we define pleasure not as our human experience of dopamine and reward chemicals but instead as "the absence of aversion" - a state where the organism is open to its environment instead of defending itself - then this does count as something positive, even if it's extremely nothing like human enjoyment.

pudding isn't appreciating the wind. but his body is registering humidity, safety, and the ability to keep functioning, and that matters to him in the only way his nervous system can make things matter. he does not think "this is great, this is awesome, i love the weather", because he doesn't think in the way we do at all, but the neurological action in his ganglion tell his body that he is safe, that the moisture is an acceptable level, that it's not too dry or windy, and that there's nothing imminently threatening.

i think a lot of the sadness comes from assuming that a good life has to look like ours: full of enjoyment, meaning, and aesthetic experience. but a snail isn't missing those things. its world just isn't built to include them.

snails don't have a sense of flavor. they don't even have tastebuds. this seems like a gimme, right? but again that might be asking the wrong question about what "taste" is. biologically speaking, it's chemoreception. we taste sweet because it indicates high value, high calorie sugar molecules. we taste salty for salt, umami for proteins. so in what way does pudding's chemoreceptors differ from ours instrumentally? we can say "by our human perspective, pudding can't experience "preference" or "savoring" or "anticipation of delicious food"", but from pudding's perspective we have radically overengineered ourselves for the task at hand. pudding can tell what's salty, what's high value, what has the chemicals he needs. the functional outcome is that he can discriminate food souces based on their composition. is that not taste?

so maybe the point isn't "this is sad because he can't enjoy it," but "this is a reminder that minds come in radically different shapes, and value doesn't have to be rich to be real."

[todays-xkcd]

Despite a reputation for safety, the temperatures and surprisingly high pressures make them even more dangerous than the air kind, but the NTSB refuses to investigate accidents because they insist there is no 'transportation' involved.

Hot Water Balloon [Explained]

Transcript Under the Cut

[A large water balloon is sitting on the ground, with cords connected to a basket nearby with Cueball and Megan inside. The water balloon's color patterns are like a hot air balloon's. Ponytail pokes the balloon with a :finger, and a squirrel is on the ground looking at the basket.]

[There are squiggly heat lines (indotherms) emanating from the balloon, as well as movement lines (agitrons) above the balloon. Above them it says:] bloop bloop

[Text below panel:] Hot water balloon rides turn out to be significantly less romantic than the air kind.

[under-the-pondering-tree]

Actually!- Hot water balloons would work, as long as the water is hot enough to be water vapor instead of a liquid. Hot air balloons work because the hot air is less dense than the surrounding air, making it more bouyant and therefore rise. But water vapor is less dense than air in general, even hot air. So as long as you had a way to keep the water super hot it would still fly. Potentially even more effectively than a hot air balloon.

[under-the-pondering-tree]

why do people say 'this earth' ? like- is there another one I'm not aware of?