Saw a comment that reminded me most people are not close to AI in their discipline--which was someone wondering why anyone would 'program' an AI to do a certain ridiculous thing.
So I'm here to say AI isn't programmed--not in the traditional sense of "a set of instructions written by a person that the computer executes." And in fact, Software Programming and AI Development are genuinely different fields. I am a software developer. I am not capable nor trained in AI development.
I have enough career exposure to both to understand core differences. But to be entirely clear, my statements about AI development are from a college-class level of understanding. But my statements about software engineering and AI in the tech industry are industry knowledge.
A key thing about 99.9% of software engineering--and a detail I very strongly like--is that code execution is deterministic. And deterministic means that, when knowing all the inputs, you can with absolute certainty know the output. This will not change unless one of the inputs changes.
And programs are--simplistically--a highly detailed instruction list. If something is going wrong with the code, you can walk through it at a granular level. You can see every individual step it is taking and identify and isolate what is wrong. There are a million ways this can be massively complicated, but deterministic execution pretty much guarantees this is possible.
AI is not deterministic. AI is not code written by someone. AI is a plinko machine 1 million feet high by 1 million feet long that folds over itself 1 million times and eventually spits the balls out somewhere. It is doing math that cannot be solved by a million people in a million days, and is the domain of GPU chips alone. It is fundamentally unknowable on the granular level.
Humans train the model by giving it known inputs with known desirable outputs and tweaking parameter weights as to reward the machines whose plinko peg configuration most leans toward the desired output. And then they iterate on generations and generations of that plinko machine until they get one that was trained to be tailored to that known data, and also yields good outputs when testing it on a new set of data.
And the human trainers may be trying to configure the rows and columns of the plinko pegs as mapping to qualifiable concepts, but by and large the pegs don't map to anything material. By and large the process becomes math weights. And even if you gave someone 1 million years to memorize every peg in the machine, the individual pegs don't mean something like the pieces of a written program mean something.
If you're using a product I worked on and the user icon rendering fails, there will be a definitive step of the program's process which is going wrong, and which I can pinpoint and fix. It is entirely possible for me to understand this whole program and know exactly why it does or does not do things. And as such, a human is capable of having complete knowledge and complete certainty about what this program is doing.
If you're using an AI product at work and it keeps making up a guy named 'Bob', well the 1-million-high by 1-million-wide plinko machine that folds itself over 1 million times spit its balls out in such a way that talks about Bob. Indirectly, the training this model went through accidentally rewarded false information in the trained pursuit of providing information. Data scientists can try to train a new model that more strongly punishes false information. They might struggle if this ends up degrading other behaviors. Absolutely no one is physically capable of understanding the exact inner details of the 1-million by 1-million plinko machine.
And this, honestly, is a hugely important detail to me in my dislike of AI rampancy. We are incapable of precisely knowing why it does things. We can tweak it, and adjust parameter weights, and try to reward different behavior. But we do not have domain knowledge over it. It is a technology capable of doing what the fuck ever.
