Tip:
Highlight text to annotate it
X
Hello world, welcome to Sirajology! Today we're gonna be building a neural net in 4 minutes. Lets get started!
There are like a million and one machine learning models out there but neural nets in particular have
gotten really popular recently because of two things. Faster computers and more data. There have been
some amazing breakthroughs in everything from image recognition to generating rap songs. There are really
just 3 steps involved in machine learning. Build it, train it and test it. Once we build our model, we can train it
against our input and output data to make it better and better at pattern recognition. So lets build our model
a 3 layer neural network in python. We'll want to start off by importing numpy which is my go to library for
scientific computing in python. Then we'll want to create a function that will map any value to a value between
0 and 1. This is called a sigmoid. This function will be run on every neuron in our network when data hits it. its
useful for creating probabilities out of numbers. Once we've created that, let's initialize our dataset as a matrix.
Each row is a different training example. Each column represents a different neuron. So we have 3 training
example with 3 input neurons each. Then we'll create our output dataset. 4 examples, 1 output neuron each.
Since we'll be generating random numbers in a second, lets seed them to make them deterministic.
this just means, give random numbers that are generated the same starting point or seed so that
we'll get the same sequence of generated numbers every time we run our program. This is useful for
debugging. Next, we'll create our synapse matrices. Synapses are the connections between each neuron
in one layer to every neuron in the next layer. Since we'll have 3 layers in our network we'll need 2 synapse
matrices. Each synapse has a random weight assigned to it. After that, we'll begin the training code.
We'll create a for loop that iterates over the training code and optimizes the network for the given dataset.
We'll start off by creating our first layer, its just our input data. Now comes the prediction step, we'll perform
matrix multiplication between each layer and the synapse. Then we'll run our sigmoid function on all the
values in the matrix to create the next layer. The next layer contains a prediction of the output data.
Then we do the same thing on that layer to get our next layer. Which is a more refined prediction.
so now that we have a prediction of the output value in layer 2, lets compare it to the expected
output data using subtraction to get the error rate. We'll also want to print out the average error rate at a set
interval to make sure it goes down every time. Next we'll multiply our error rate by the result of our sigmoid
function. The function is used to get the derivative of our output prediction from layer 2. This will give us a delta
which we'll use to reduce the error rate of the predictions when we update our synpases every iteration.
then we'll want to see how much layer 1 contributed to the error in layer 2. This is called backpropagation .
we'll get this error by multiplying layer 2's delta by synapse 1's transpose. Then we'll get layer 1's delta
by multiplying its error by the result of our sigmoid function. The function is used to get the derivative of
layer 1. Now that we have deltas for each of our layers, we can use them to update our synapse weights
to reduce the error rate more and more every iteration. This is an algorithm called gradient descent.
to do this, we'll just multiply each error by a delta. Finally lets print the predicted output. And there you have it!
Let's run this in terminal and see what we get. Awesome! We can see that our error rate decreases every
iteration and the predicted output is very very close to the actual output
there is so much we can do to improve our neural network. For more info, check out the links in the
description below. and please subscribe for more technology videos. Thanks for watching!