What is artificial intelligence?
Over the years, it’s been defined in many ways. Alan Turing, considered to be the “father” of AI, introduced the Turing Test, and claimed that a machine was artificially intelligent if it could convincingly mimic the speech patterns of a human. If you were talking to someone over the phone, could a computer convincingly speak to you as a human would, to the point that you wouldn’t even question its humanhood?
More recently, there has been an insurgence of folks who consider AI to be a program that learns, or has learned its capabilities from data or experience.
These are both good attempts, but they are also both wrong. Yes, it sounds weird to call out the father of artificial intelligence as being wrong about artificial intelligence. But, to quote Isaac Newton, “If I have seen that Alan Turing is wrong, it is by standing upon the shoulders of Alan Turing.” Or something like that. With 100 years of additional hindsight, he would probably think he’s wrong too.
The problem with the first, i.e. Alan Turing’s definition, is that there are many things that sound intelligent to the average person but are not: parrots, tape recorders, my ex-partner. And there are many things that do not sound intelligent or even comprehensible to the average person but at least on some level are: Computer bytecode, secret messages, Dan Quayle. In fact, computers have even “won” Turing tests by simply appearing convincingly, humanly confused.
The problem with the second definition is that it precludes prescribed knowledge. If you encode a very clever reasoning capability into software, is it not smart? Is it not intelligent? Even throughout nature, many organisms are born with knowledge imprinted in their neural circuitry, typically survival skills for themselves or their species.
So, what is artificial intelligence?
Classically, the accepted definition in the field has been that AI is search. No, this does not mean robots that can go find buried treasure; rather, this means agents that intellectually can logically consider possibilities in order to find a solution to a problem. They are searching for an answer, and they do that by inductively and/or deductively exploring candidate solutions through trial and error. When they find one that works, they have simulated its result, and thus can prove to you that they have found the answer.
This definition worked for a long time. You use this type of technology, maybe even every day. A GPS navigation app searches for a path from your starting point to your requested destination, and can show you a path from point A to point B that you can see is correct, maybe even optimal. When you do a text search for a file on your computer, when it finds you the file with the name you’re looking for, you can see that it’s the one you wanted.
This version of AI racked up all sorts of wins for decades. But, it also pushed up against the limits of tractability and observability.
A perfect example of this is in chess. Chess is perhaps the most classic application of AI, ever since a man in 1770 dressed up as a robot and pretended to be a chess-playing computer, and if you don’t believe me look up “The Mecahnical Turk.” For the longest time, humans summarily defeated their silicon counterparts in chess and similar complexity board games (such as Shogi and Go). The problem was that AI players just couldn’t cut by using search. These co-called chess engines worked by imagining a move, then imagining all of the moves that their opponent could play in response, then imagining all of the moves that they could play in response to all of those moves…and so on. So, if a computer wanted to consider one move, it might then have to consider four countermoves by its opponent, and then the 4 x 4 countermoves it would play in response to those, then the 4 x 4 x 4 countermoves its opponent could play…even if we consider that a player only has up to four moves at any given time (and they usually have more), by looking ahead to just move 20 it has to consider over a trillion moves. Even being clever about pruning obviously bad moves, a computer can’t consider all reasonable strategies and prove to you it’s making the best moves. Its opponent would die of old age before it finished that search.
So, chess AI started relying on heuristics. It would look ahead only three or four moves, then decide “is this a good position to be in?” It did so by applying a heuristic function — based on what it knew about the future of the game, it would take a guess as to how good the game position was, based on a mathematical score it would assign.
And by picking good heuristic functions and searching just deep enough at any given time, IBM’s Deep Blue Supercomputer was able to finally beat the world champion and top 3 chess player of all time, Garry Kasparov, in 1996.
These same heuristic guesses are also used in cases of partial observability, by virtue of necessity. In instances where a computer can’t exhaustively search all possible outcomes because it doesn’t have all information, it has to get by on some amount of guesswork. A self-driving car, for example, can’t use an exhaustive search in choosing its next maneuver. It might know all the roads, but it has to take guesses about the position of other cars outside of the range of its cameras.
But coming up with these heuristic functions is difficult, and arguably worse, it’s boooring.
Luckily, a subdiscipline of AI, called “machine learning” or ML, had a potential solution to automate away all of the boring (read: difficult) aspects. Rather than prescribe mathematical guesses for things computers can’t see, what if computers could learn to make good guesses, from data or experience? After all, this is what humans do.
Machine learning allows computers to look at datasets, use some data to try to improve its heuristics, use some data to test the efficacy of that heuristic, and iteratively try to continue to improve it some more. That “testing” step is key, by the way — machine learning fundamentally is only useful if it produces heuristics that are predictive on some “test” data that is not used for training. That predictiveness, versus pure datafitting, is the fundamental difference between statistics and machine learning. Machine learning is trying to be useful in the future, but it’s not necessarily trying to explain what it’s seen in the past.
From the advent of machine learning in the 1940s to the early 2010s, machine learning worked…to a point. Like, not really super well in most domains. Like, 80-90% well. Computers were B+ students in many domains. Yeah, you could use them to reasonably guess property values. But they had difficulty with some domains. Images. Words. Predicting the behavior of humans. In the 1950s, Marvin Minsky famously thought that learning models of human vision was a problem that could be solved in two months, but by 2010 computers still couldn’t solve CAPTCHAs (thankfully) or tell you how many cats were in a given picture (The answer is always: not enough). Until the late 00s, computers weren’t competitive at trivia games such as Jeopardy!, not because they didn’t know a lot of facts, but because they couldn’t understand the language. And even when they eventually triumphed, they did so through statistical guesswork, matching key phrases with corresponding knowledge in their databanks. In the mid 00s, Netflix finally was able to predict a movie you might like to watch next — to a roughly 90% success rate.
All of these attempts at creating human-like intelligence were largely very statistically, mathematically based. But humans probably aren’t constantly running numerical computations in their brains for every decision they make. What if we tried a different, more bioinspired approach, such as mimicking the human brain? Neural networks, perhaps one of the most famous types of machine learning model, were invented to do just that.
Now, it’s important to note that these “neural networks,” really kind of sort of look nothing like neural networks of the brain. They’re, uhh…made of something we call “neurons.” But these “neurons” are more like simple mathematical switches, like digital logic gates on a computer. But computers can do a lot, ad so could these artificial neural networks! And without going too far into the details, each individual neuron can predict more types of phenomena than the most classical statistical models. So, you could imagine the potential power these neurons would provide when tons of them were linked up together into a massive circuit!
That’s right, they provided no power whatsoever, because designing that circuit — what types of switches there are and how they are hooked up to one another — was itself a massive search problem that computers couldn’t solve. Neural networks sucked.
In 2012, certain parts of computer hardware (specifically the GPU, which was traditionally used for creating graphics) had gotten more powerful, we learned to create better search algorithms for neural networks, and we had invented new types of neural networks that were easier to search. And suddenly, these new hardware and new software approaches were able to be combined to design very large and very capable neural networks in a matter of days. With the colloquially titled “AlexNet,” Deep learning was proven to be both practical.
And how did we harness this deep learning power in order to solve humanity’s most pressing problems?
First, we used deep learning in order to be able to learn to identify cats in pictures. That was called supervised learning. Then, we were able to determine if cats in pictures were different enough from one another to be put into separate “classes”; this is commonly known as unsupervised learning. Then, we were able to teach robot cats how to walk, from their own experience, in what’s referred to as semi-supervised or “reinforcement learning.” (This same type of machine learning has also led computers to finally reign victorious in Shogi and Go, and extend their dominance in chess.)
And now, in 2023, just ten years later, we have reached the apex of computer intelligence. We are able to use deep learning to generate images of cats, in what’s known as “generative machine learning,” or “generative AI.”
Clearly, this book is AI’s magnum opus.
All machine learning is on some level guesswork, but generative machine learning, which fundamentally draws outputs based on probabilities, is the “guessiest” of them all. It provides plausible possibilities, such as images, sentences, or music, based on what a person wants, without any guarantees of correctness.
Oh dear. I fear we have lost the plot, in this essay, in the field of AI, and maybe in society.
I claimed that AI is search. Machine learning-powered AI is search that uses some guesswork, relying on heuristics learned from data. And generative AI is also search, but it’s a quick search, one in which the computer is so confident about what it has learned that it assumes that the very first guess it makes is correct and declares victory.
Is this intelligent, if there is no verification?
Is this artificial, if it’s so based in real-world data?
At the beginning of this chapter, I was arrogant. I said, pompously, that Alan Turing, as well as many of my living colleagues, were wrong with their two definitions of AI, and that my third definition was correct. But maybe I’m no better than a generative AI algorithm, immediately confident that my learned instincts would necessarily produce me the right answer.
Generative AI is the “new hotness” in AI, but it is but one flavor. Every iteration listed above has come with its own strengths and weaknesses, and with those weaknesses, hidden repercussions.
