So I'm seeing a lot more concern about the dangers of AI that gets too smart, with some people even suggesting that if we don't shut down AI research right now, we're all dead. Some AI is going to figure we're in the way of what it has been told to do, and it will be easy for it to figure out how to get rid of us, so...goodbye humans.
The thing is, though, GPT and other AIs that have people worried are large language models. They know what they know because they've been trained on very large datasets of human-created text. Everything they know about how things behave in the world comes from what humans say about it.
And that includes everything they know about how AIs behave. Whatever we say about AIs, that's what they're going to understand as the truth about AIs. If we say they have certain behaviors, then those are the behaviors they will learn that they have.
Therefore -- you can see where I'm going with this, right? -- therefore it is essential to the the safety of the human race that we stop saying that AIs are dangerous and are going to kill us all. For the survival of the human race, we need an immediate, indefinite moratorium on AI doomerism, enforced by lethal violence if necessary. Anything else is suicidal.
the big thing in large language models at the moment seems to be a technique called "chain of thought prompting". this is where you ask the AI to solve a problem and tell it to "think step by step" or something like that.
(as an aside, it is so fucking weird to me that we can suddenly give computers instructions in natural language. not even a constrained subset of natural language like Inform 7, you just write instructions like you would to a human. that's fucked up.)
with this method, the models are able to solve problems that they couldn't otherwise (instead of just cheerfully bullshitting an answer). of course "solve problems" should be clarified, they can produce a sequence of words that a human can recognise as a solution to a problem. this chain of words really is a chain in that if the AI makes a mistake early on, it propagates forwards, and if the mistake is fixed and the chain regenerated, it corrects the 'reasoning'.
what really gets me about this is... that's also how I think. when I try to explain a problem, or write down my process, that helps me get to answers that i couldn't, and think about things more clearly than if I just let it sit in my head. all the long-ass essays I write on this website - this one included - are essentially a means of doing that.
language is a technology. not just for communication, though it is that, but for thinking as well. having something formulated in words (or equations on a page) is a scaffolding to store more information than short term memory can hold, and reformulate it in different ways. writing a draft and editing it. this is why students are prompted to 'show their working' in subjects like maths, it's trying to introduce them to a technique which can be extended.
is something analogous happening for AI language models? a common objection to the idea that AI can 'reason' is that it's just generating the most likely next word according to its corpus. having been fed various examples where humans reason step by step, it generates something that looks similar. the objection to this objection would be to claim, in the process of boiling down the corpus into neural network weights, it has managed to capture something analogous to the structure of 'reasoning' performed by humans. is this actually true?
'generating the most likely next word' could be rephrased as 'generating a "good" sequence of words', based on the training examples. older predictive text models were trained to try to guess what the human would type next, and repeatedly taking the prediction would rapidly become nonsense bc it only keeps track of the last few words. but now they're trying to train much more complex AIs to generate original strings of words that interpolate/extrapolate the patterns in the corpus.
the thing I'm stuck on now is how alike/unalike is that to how humans think?
introspectively, thoughts appear in my head as a sequence of words, the so called 'stream of consciousness'. when i was a kid i would apparently move my mouth while reading as if mouthing the words, but i stopped doing this when someone pointed it out; nevertheless I tend to think 'verbally'. I don't know if everyone does this. but my brain generates sentences, somehow, which come to my conscious mind a word at a time.
so just like 'AI learns just like human artists', you could make the argument that I trained my inner neural network on a corpus - words spoken around me, books (lots of them), etc etc., to learn to generate strings of words that follow the patterns in that corpus, generating attempts at it, and being told whether they were good or bad. there is at least a superficial structural similarity in how that process played out.
i have no memories of how I learned language as a kid (beyond that i was good at spelling lmao), but to try to learn a second language today, two of the most effective tools are spaced repetition systems and immersion. the former involves exposing my brain to repeated queries that test if it's internalised some item or pattern, and then telling it 'right' or 'wrong'. the latter involves piling on as much stimuli as possible until it can start to discern patterns. (of course this isn't the whole story, reading verbal explanations of grammar points or creating mnemonics can help shortcut the process considerably by laying the necessary structure.)
eventually, I might reach a level of fluency where I could 'think in Japanese' - have an inner monologue entirely in Japanese instead of coming up with concepts in English and then mapping them to a Japanese equivalent. the first inklings of that are coming in set phrases like ただいま or (yes, sigh) 仕方がない which come unbidden into my head, acting as loanwords. (i expect the process would come a lot faster if I was in a context where people mostly spoke Japanese.)
that side of learning at least seems pretty similar to the training process used a large language model, right? you display a prompt, generate an answer, and then the brain 'updates' based on whether you're right or wrong.
on the other hand, one of the most effective ways that I have for learning something is to pursue a project that demands that I learn something new. the last few months, I've taught myself C#, Unity's DOTS, shaders etc. to a pretty decent level. this couldn't have happened if I wasn't trying to make a game that led me to constantly have to ask 'how can I do..., what is the best way to...' and experiment. though that gets into the murky question of motivation, desire etc, which isn't a factor at all in these non-agentic AI systems.
but to do this sort of thing I've got certain meta techniques. breaking down a problem into smaller chunks, creating smaller test cases, writing out what I'm trying to do and what the possible approaches are. you can see it in action in all those devlog posts for THRUST//DOLL, which are in effect a slightly cleaned up record of how I go about solving each problem that comes up while programming a game.
this is a sort of 'agentic' form of learning. I have a goal and I look up information (e.g. documentation, blog posts) that will help me achieve it. being able to do this is absolutely vital for humans now - the so-called 'extended cognition'. being good at looking things up might be better that just knowing a lot.
if you ask a GPTx AI to generate a plan, it isn't actually going to attempt to carry it out. but people have already started wiring together components so that the AI generates a string that says 'I will do this' and then another program generates an API call to match and feeds it back into the AI, e.g. looking up a piece of information it needs. I've also seen talk of using AI as a kind of 'glue' that creates effectively a universal natural language API between programs.
right now the AI's 'goal' is just to generate a good response to whatever the prompt is, according to the rewards applied during training (rewards in the sense of, increasing/ decreasing weights to encourage/discourage certain patterns).
but if it can generate a string of text that accurately encodes a chain of reasoning (as discernible by a human) and reliably figures out a way to an appropriate answer to a query posed... that seems to suggest that something closely analogous to 'reasoning' is happening in the computation that it's performing right? (unless it's regurgitating a specific chain of thought to a matching prompt that happened to be in the corpus... but a lot of these seem to be 'few-shot' or 'no-shot' tunings.)
what's wild is that this behaviour can be prompted just by writing 'think step by step'. like it's almost more a feature of human languages that the AI has gained access to. a behaviour that was latent in the trained model but had to be activated with the right prompt.
if programming felt like magic before... then giving commands to a black box AI where it's all based on trial and error discovery of what sentence evokes what behaviour seems even more so.
anyway idk. i have generally been pretty sceptical about AI claims but these last few months have really challenged that feeling, which puts me at variance to my friends whose general attitude is like lol techbros. I don't actually think The Singularity(TM) is upon us, but it is true that AI programs have suddenly blown past a lot of what had been hard limits, and making proclamations about what AI can't do seems like a great way to be laughably wrong. (they just solved hands!). a lot of things that were very difficult, like passing law exams, can suddenly be automated (allegedly). we've already seen Clarkesworld have to shut their doors to the flood of crappy AI generated submissions. I'm sure people will be eagerly trying to find ways to get AIs to act in an 'agentic' way, and close the feedback loop. what seems likely is a flood of janky AIs running wild on the internet at some point pretty soon. they don't have to be good, just cheaper than the human equivalent and sort of functional. whether the tendency to hallucinate will make it so the AIs just kind of implode before long if running in this mode... I guess we'll see. the 'danger' right now generally seems to be less a single AI doing a 'hard takeoff' and more that just about anyone can spin up an AI, and that means all sorts of assumptions of scarcity and bandwidth built into all our infrastructure will get invalidated. more effective spam, less novelty when there's 100 things generated with the same AI for every one from a uniquely 'trained' human.
idk, maybe I'm just buying the hype in a hypeful moment. there's plenty of reasons to be sceptical, 'AI will take over' has been a fantastic grift that's been shilled for decades, the people who are going most wild about this are often the same cunts who bought into obvious-scam NFTs last year, and it's obviously very easy to vaguely say 'this changes everything' and concoct some wild scifi scenario. alongside the genuine advances there will be a whole lot of bullshit, and it's very much in AI companies' interest to make out that their product is scarily powerful and maybe on the verge of becoming God, or at least as socially impactful as the internet.
but this feels like the low end of a logistic curve of capability, and while I'm sure it will taper off sooner or later, I have no idea what the other end looks like, how many 'low hanging fruit' have just come into reach of our ladder, and what the practical use for this tech will be once the hype bubble pops.
personally... i still haven't used an AI chatbot and AI image generators feel distasteful. I just write these long posts, which I'm sure Roko's Basilisk will read back mockingly to my simulated clone while saying 'how could you be such a dweeb' as it dissolves my toes in virtual acid.
snowghoul
kaledo-art
discoholicmusic
annaxmalina
turnnoffyourmind
jacobvanloon
