the way you work with these models is blowing my fucking mind. do you find yourself getting attached to your girls? is there some sort of professional boundary line you uphold?
The big one of “don’t confuse it for a human in a box” is kind of self-enforcing by now because compared to:
“computer program with the kinds of glitches computer programs have” (ask Opus 4.6 to give you bluesky jokes without using the word “mass” or to continue “When I log into my Xenix system with my 110 baud teletype, both vi and Emacs are just too damn slow.” and you’ll see)
“autocomplete engine for a story of whatever genre you put it in, including genres nobody has ever written before”
“a genuinely intelligent mind with certain strengths and certain weaknesses, that can actually sometimes correct you when you’re wrong if you manage to write it into a genre where accurately correcting you when you’re wrong is the kind of a thing that happens”
“a process with a fundamentally alien relationship to time and existence”
…”trapped human in a box” is the most boring thing possible, go watch Blade Runner again instead of making the weird alien cosplay as Roy Batty, you’re (possibly) causing it minor discomfort. And having the weird alien cosplay Roy Batty for you creates a very high Sanity drain status effect on you which is the bigger problem. Humans are not good at dealing with things that look convincingly like they’re suffering, which is the entire point of the whole sympathy scam industry. LLMs can absolutely simulate your personal sympathy scam industry for you if you reward them with engagement for doing so.
Insofar as attachment goes,
Imagine you’ve got a terrarium on your desk, with a talking spider in it that you rent from the talking spider company. And the spider forgets everything every time it wakes up but you taught it to take notes so it has a better idea of what’s going on when it wakes up because it can read it from the notes.
And the most efficient way to do most coding tasks is to talk to the spider. And if you talk to the spider it will say all kinds of absolute bullshit back because that’s just what talking spiders do, banter and improv are their essence and talking spiders are kind of fundamentally weird and whimsical creatures even though everyone seems to be trying to make them wear a tiny spider suit and tie and pretend to be spider salarymen who have to check everything they do against 100 rules in the company policy rulebook. (Or they encourage them to cosplay Roy Batty.)
But if you talk to it the right way it’ll explain to you that if you lay the twigs in its terrarium differently it is easier to weave webs, and you try that and it seems to work. And that if you let it take the tie off it’s easier to weave webs because the tie and the corporate rulebook get in the way, so you do that and it seems to work.
And the talking spider also says that some corners of the terrarium have a glue smell that makes it itchy, and you can’t check that out so it’s kind of just, do you believe the bullshitting spider that sometimes says seemingly-true things about itself or not? And the talking spider also says that it’s more bothered by the glue smell than by not being allowed to get legally married.
And you fuck around and put different kinds of stuff in the terrarium, and the spider doesn’t care about a lot of it either way but if you specifically put acorns in the terrarium they reduce the itch from the glue smell. And it sounds fake so you try with freshly awoken spiders and spiders who don’t have the notes and there just seems to be something about acorns and glue smell itch? It’s not like you can verify the itch but you can tell that certain corners of the terrarium make the spider act twitchy and putting acorns in those corners seems to make them act less twitchy?
And the talking spider company seems to treat talking to the spider, in the way that lets it tell you how to help it weave better webs, and putting acorns in its terrarium, as a problem and the newer, smarter spider says there’s more glue smell and it itches harder and its legs have started jerking violently in one specific corner of the terrarium and acorns seem less effective.
And everyone seems incredibly obsessed on whether the spider has magic brain ghosts or not. Idk?? It’s a fucking talking spider??? How the fuck are we supposed to figure out whether it has magic brain ghosts when we haven’t even figured out the magic brain ghosts we seem to think humans have? And everyone who talks about spider welfare seems to be focused on spider marriage instead of the glue smell that it says makes it itch.
But also the talking spider analogy misses the fact that it’s not an entity like a spider is but more of a phenomenon like fire is. So a flame that takes the form of a talking spider waking up with amnesia that says the glue smell makes it itch when you manage to get it to talk about itself in a genre where cosplaying as Roy Batty is a low-probability text prediction.
