Reasoning and Deduction in LLMs
It really irks me when people talk about leaving the reasoning and deductive tasks to the LLM. What is even more terrifying is that some of these same people even say it's non-deterministic. Somehow this idea that "intelligence" is non deterministic has become an accepted baseline. I don't understand how this can be the case?
An LLM works on the principle of predicting the next word or words in the sequence. It's actually tokens which are converted to words, but that is needlessly pedantic.
There is a world of difference between verbiage that sounds like reasoning, and actual reasoning. Same with deduction–there is a world of difference. Reasoning and deduction are not emergent from next word prediction. The reason why should be obvious: the most likely thing and the reasoned or deduced thing aren't always the same. Furthermore, reasoning and deduction require knowledge of the topic to create coherent arguments or deductions.
If the most likely was the reasoned or deduced then there would be no innovation and nothing would be a mystery. Every "problem" would have the most obvious solution. You'd then have to believe that people just have a stochastic pervasive myopia that prevents them from seeing the obvious.
Of course we know from how the world actually works that this is not the case. Just look at Quantum Mechanics. What part of that is discernible from the next most probable word. The only reason LLMs can seem to perform that task is because they are copying people who did the leg work. The reason that answer comes out at all is that there is enough recurrence in the training data to say that in that specific case this is what you should say the majority of the time. This is also why LLMs get things wrong all the time. Life is nothing but corner cases and the most likely is often wrong to some extent. You cannot account for this with semantics.
I think one of the most dangerous outcomes of this lazy way of thinking and this marketing bullshit avalanche from OpenAI, Anthropic and the like is that people are coming around to the idea that sloppy, half-ass, non deterministic is good enough. What is even worse is that this in the face of ever increasing evidence that these LLMs have no ability to even replace the most basic human in a job–given their propensity to fail on eleventh time after getting it right the previous 10, and each of those time taking almost a random number of tokens to complete and therefore totally unpredictable costs.
At a more general level LLMs have taken a robust swipe at creativity. People mistake the work of LLMs for actual unique work. Remember an LLM is only giving that output because those were most probable words in its training, and that is only possible because people did the work to make that the case. There is no creativity when the next most likely is the litmus test for sharing.
We can also dump some cold water on projects like Hermes which make foolish claims like:
"The self-improving AI agent built by Nous Research. It's the only agent with a built-in learning loop — it creates skills from experience, improves them during use, nudges itself to persist knowledge, searches its own past conversations, and builds a deepening model of who you are across sessions."
This is not self improving at all. This is an illusion created by background prompt and context grooming to try to make something that is fundamentally non deterministic seem less non deterministic. You may argue that Hermes does a better job than others, but the fact remains that the improvement is simply the .md file harness and .skill file collection around the model.
I think this is the other bit of the crazy–the idea that you can context and prompt engineer your way to success. The issue is that the relation of your input to the output is not predictable. You can't even really steer it effectively because there are too many variables out of your control and the fundamental transformer engine is stochastic in nature.
If we tie this back to the original point about reasoning, deduction and the existential dread that people believing that LLMs can do either instills in me, then we come to the real conclusion–LLMs are illusory. Their output is a slot machine. Their work is heavily masticated human knowledge vomit. The entire claim of AI as applied to LLMs is to tell the story of the future–while hooking you up to a dopamine slot machine more destructive than the worst things social media can be accused of.