The goal of a neural network is to achieve a certain output or outputs from a given set of inputs. It is relatively straightforward to create a software program that can provide a specified output based on some inputs. A more challenging task is to produce a meaningful result. Let’s say I feed a program an input, like a picture of a dog, and I want the computer to tell me whether the photo is of a dog or not. This is a very simple task for a human to complete, even a one-year-old human child could probably differentiate between a dog and something that is not a dog, like a cat. However, until recently, this was not a trivial task for computers.

Until programmers realized that learning, at least for computers, is a lot like finding the most efficient way down a mountain, there was little hope of developing software that could make these distinctions. So how is Machine Learning like Mountaineering? Essentially, what we’re trying to do is teach the computer to arrive at the correct answer to a question we already know the answer to. In order to do that, we have to train the program. Training, in this context, means feeding the computer thousands or millions of images of dogs and images of not dogs and having the software output results and check those results with the label on the images. We’re comparing the answers on the images we already know, to what the program thinks the answer should be.

If we start out with a program that outputs random answers based on the inputs, we would expect that those random answers would not correctly identify a large percentage of the dog images. This means the error is large and we’re high up on the mountain. Our goal is to descend ‘Mount Errorest‘, reducing the difference between the actual answer (dog or not dog) and the output of the program or algorithm.

So what’s the quickest way down a mountain? Well, off the steepest slope, of course, a cliff! That’s basically what we’re doing here. Figuring out the fastest way to reduce the output errors to a minimum so that we can make the output of the function equal to the expected answer that a human would give.

Think back to your high school or undergraduate math courses. How do we find the slope of something? By taking its derivative, of course! As it turns out, there is a formula for finding the steepest slope of a multivariable function, it’s called the gradient or the symbol for short. Since we want to find the quickest way down, we actually want the negative gradient or .

If you need a refresher on how to calculate the gradient of a function, or what it is. Here’s a great video from Khan Academy on the subject:

Once we figure out the steepest way down, it’s just a matter of iterating our program through small steps over and over again until the program’s output matches the expected output. Once the training is complete, we can start to feed in the images we want to classify and see how well our model does. We can even use the results from these images to further test and improve the model. Pretty cool, huh?