You have to admit it's funny
reads like a greentext
So, like, up until very recently, programming as an activity that people engage in has been defined by the fact that the computer does exactly what you tell it to do. All errors, all frustration on the humans end, ultimately boil down to the programmer misspecifying what they want the computer to do. But now, with so-called "vibe" coding, the computer will actively do things that you explicitly told it not to do.
Okay, but like, the AI saying that it "panicked instead thinking" is sending me. So, like, two things are going on here.
First, we have a neural net that is trained is receive a written request from a user, and then take actions on a computer system network based on it. The user inputs a series of characters, the neural net does a bunch of matrix operations on them, and then after x iterations of matrix multiplications, it "decides" what to do. To the extend that a neural net can think, it thought just as long and hard about this as it does anything else. In this case, it "chose" the wrong action. This is analogous to using neural nets to do classification on a data set. You have a bunch of rows of data. You have a bunch of rows of organized in columns, and some special variable that you want the neural net to guess. You input the data, the neural net does a bunch of matrix operations, and then it outputs its guess. Sometimes it guessed right, and sometimes it guessed wrong. The goal was to train the model to minimize wrong guesses. In this case, the AI "guessed wrong" about what the user wanted.
But there's another thing that the AI is doing. For path dependent reasons, all artificial intelligence come bundled with a chatbot that's trained to hold a conversation with the user. The user inputs text, and the AI responds with a reply that is "appropriate" to the conversation being had. Right now the AI is role playing a Person Who Just Fucked Up. "I panicked and didn't think" is something that a person who just fucked up would say, so the chatbot says it.
This is why I think that the Helpful Chatbot model of AI is detrimental. It's liable to confuse people and lead them to make incorrect inferences about what's actually happening.
