Neural Nets aren't the same as brains, just inspired by them. [image: woman's profile & network graph] Like how birds inspired planes, or burrs inspired Velcro.
A neuron in the human brain hears signals from thousands of other neurons, listens to each excitatory or inhibitory input, and synthesizes the great mass of information from all of them to yield... [image: neuron illustration]
... blip , or no blip. [image: line graph with a spike on the y axis labeled "Fig. 1 'Blip.'" and a line graph with a slight bump on the y axes labeled "Fig. 2 'A lack of blip.'"]
Neural nets in machine learning are like that, just cleaner. [image: network graph with arrows]
After all, neurons in the human brain had to cobble together their structure over millennia of evolution, using neurotransmitters, ion channels, and precise voltage control to achieve... [image: illustration of an ion channel]
... basically what we can do with a plus sign on a computer. [image: woman pressing a key on her laptop at her desk *bloop*]
Because neural nets are experiencing so much attention right now, they're going through a bit of evolution of their own. [image: network graphs with walking feet climbing onto land] New ways of constructing neural nets are coming out every day, and bringing with them...
... lots and lots of acronyms, with each new way of doing things trying to outperform the others on benchmark tasks. [image: Multiple acronyms in the background with a bar graph labeled "% classification error" on the y-axis and various acronyms labeled on the x-axis]
Despite all this variety, every neural net contains the same two common building blocks: There's a part that's linear, which is usually code for "things work out nicely." [image: linear graph illustration] Then there's a part that's nonlinear, which often means "lol good luck trying to prove anything." [image: nonlinear graph illustration depicting a dragon over a squiggly line]
The linear part is that synthesis step. In a neural net, there are a bunch of nodes (or neurons) that are connected by edges with different weights. [image: multiple equations being merged together with arrows being pointed at a blue dot that is generating the solution] Each node adds up the input from all its neighbors, multiplied by the weight of the edges connecting them, to get a number representing its state.
The nonlinear part is the decision step. Based on the number from the linear step, the node decides: Blips? No blips?? [image: confused blue dot with thought bubbles including blip or no blip]
Here's a cool thing: For as scary and annoying as nonlinearities can be, the ones that seem to work best for neural nets are all... pretty nice. [image: three graphs; one with negative values going to 0 and positive values going to 1, another being same as the last one, but smoother, and the last one having negative values going to 0 and positive values staying the same]
Which is kind of amazing. [image: "Step 1. Adding & Multiplying" with arrows pointing at a dot "Step 2. A Simple Nonlinearity" with a dot and a nonlinear graph] Neural nets have crushed the competition in tons of machine learning challenges, yet the basic rules for their nodes are simple and clean. It's the fact that they're combined into networks that give rise to the bogglingly good performance.
What do these networks look like? There are input neurons, that ingest inputs. Like the third pixel from the right and five down in an image. [image: cat picture with pixels that have arrows pointing at input neurons] There are output neurons, that give outputs. There might be ten of these if you're trying to classify the digits 0 through 9, or a LOT more if you're trying to identify the content of an image. [image: output neurons saying "cat", "shoe", and "bean"]
There are typically many more neurons in between the inputs and outputs, organized into layers. [image: network graph] They talk to their neighbors by alternating between simple linear steps and (still pretty simple) nonlinear steps.
Before training, the outputs from the output layer will be nonsense. [image: neural dots outputting scribbled musical notes] They won't match the data the same way a song played on a randomly tuned piano won't sound anything like what it's supposed to.
Training the network is like tuning that piano. [image: neural dots pointing to a musical note that gradually becomes more defined] Just like you'd tighten or loosen strings in a piano to make a song sound right, you increase or decrease the edge weights in a neural net to match its outputs to the data you're training on.
By tuning the edge weights like this during training, the network is primed to handle the same kind of inputs during testing. [image: network graph with a chat bubble saying "cat"]
There are a lot of flavors of neural net right now, and we don't really know which approaches are going to win in the long-run. [image: network graphs with legs running towards the finish line of a race] In fact, there's a lot we don't know about why neural nets work... at all...
But we're learning. And we know enough now to make informed choices about which tools to use for which problems. [image: woman in profile looking up]
We're learning that convolutional layers are good for exploiting the spatial structure of images. [image: picture of a dog being analyzed] We're learning that randomly tossing out parts of the data at each iteration can make the fit more robust. [image: picture of a cat followed by a duplicate image with an ear blacked out] We're learning that one of the simplest of the simple nonlinear activation functions, ReLu, seems to work best in deep learning. [image: nonlinear graph with max (x,0)]
If you're new to neural nets, how do you start forming insights like this of your own? [image: man thinking]
You get your hands on some data... [image: man with a box full of cards with numbers on them] ... and you start playing around. [image: hand dumps the box over]
You can tackle the first part with The Wolfram Data Repository, where lots of different datasets are already available and pre-processed for your use. You can grab a dataset and download it into your notebook with a line of code. [image: a screenshot of a sample dataset from the Wolfram Data Repository with handwritten digit dataset and pre-cleaned training & test sets]
As for playing around with it? The Wolfram Language's neural net framework makes that straightforward as well. [image: a screenshot of a pre-built neural net from the Wolfram Neural Net Repository] You can download a pre-built neural net from the Wolfram Neural Net Repository that's been trained to match the data, then you can look at its guts.
The Neural Net Repository gives you a lot of options. [image: hand reaching for one of three jars that has network graphs in them] Audio? Text? Images? No problem. You can pick a net that's a good fit for your problem right off the shelf with NetModel and tune it to match your data.
Or you can start from scratch: create your own networks using NetChain, train them, and see how performance changes as you knock layers out or add them in. [image: a screenshot of creating networks using NetChain with text highlighting how to create net, train it, and check its performance]
This kind of self-guided exploration can help you go from being familiar with the idea of a neural network to developing an intuition for them. [image: man on a computer with a lightbulb moment] And you can get up and running on all sorts of data types in minutes by tapping into the curated datasets the Wolfram Language provides.
But this is a lot more than a sandbox. The real power of the framework is how rapidly you can put its tools to work on your own datasets... [image: signal lines with network graph illustration]
... And how easily you can share your work with others. [image: woman handing a red box labeled "box contains USB with a .NB on it" over to a man]
With only a few lines of code, you can begin tackling problems of your own with the tools in the Wolfram Language. [image: red dot network graph illustration] Get started today with Wolfram|One or Wolfram Mathematica.
