[MUSIC] In this video, we'll begin our discussion of neural network and we'll talk about a neuron model and how we can learn artificial neural network model. We'll see the differences between how a computer and our brain works. So, in terms of arithmetic calculations, even a small pocket calculator is 10 times more powerful or faster than our brain. Whereas in terms of vision and knowing the details while looking, our brain is much more powerful, even maybe 1000 times of a supercomputer. So, in terms of memory of arbitrary detail, computer wins. Whereas, for real world facts, brain is a clear winner. So, a computer must be programmed explicitly. It should be told how to make decisions, whereas our brain can learn by experiencing the world. So, in terms of different items, we can see the difference between computer versus human brain. So, in terms of complexity, a computer has an ordered structure and it process everything in order serial wise. Whereas our brain has 10 billion neurons and 10 to the power 14 connections. In terms of processor speed, computer is much faster. It can do as much as 10 million operations per second, whereas our brain can only up to 100 operations. In terms of computational power, a computer do one operation at a time which have one or two inputs. Whereas, our brain can do millions of operations at a time with thousands of inputs. So, inherent advantage of the brain is distributed processing and representation. So, we have parallel processing speed, fault tolerance, graceful degradation and ability to generalize the results. So, artificial neural networks are algorithms which aim to imitate brain in the computers. So, neural networks are a new method of programming computers. So, they are inspired by human brain's information processing mechanisms, that is our nervous system. In which the network of highly interconnected processing elements, neurons, learn by example. So, they are exceptionally good at performing pattern recognition, especially those that are difficult to program using conventional techniques. And are capable of learning on their own and adapting to the changing conditions. So if we look at how the human brain learns, so as an input, we need a neuron collect signals from others through a host of fine structures called dendrites. Then the neuron outputs or sends out a spike of electrical activity through a long thin stand known as axon. So, then axon output is converted to input for other neurons. So, this is a typical way in which our brain neuron cell works. So, we have a cell in which lots of input comes and then it generates an output. So, when a neuron receives input that is sufficiently large compared to the threshold, it sends a spike of electrical activity down its axon, that is, it sends an output. Here, if we look at the learning parameters, these are neuron threshold. That when a neuron sense output and the weight given to each input. So, they can be represented in the figure as follows. So, the cell body has lots of input coming through dendrites. Then based on all these inputs, it generates an output which is passed through axons. So, the output is based on a threshold. To imitate a neuron in a computer, an artificial neuron is called as a perceptron. So, it's a learning element with many inputs and one output, and has two modes of operation. A training mode and a use mode. In training mode, a neuron is trained to fire or not fire for different input patterns. Whereas in use mode, whenever a taught pattern is detected at the input, its associated output become a current output. However, if the input pattern does not belong to any taught list, the firing rule is used to determine whether to fire or not, and depends on all the input patterns. An Artificial Neural Network (ANN) is a collection of perceptron grouped by layers. So, there are three classes of layer, one is an input layer. Then we have an intermediate or hidden layer which can be one or many and we have an output layer. The behaviour of an ANN to a particular input depends on several factors, such as structure of each node. That is how a node works, which is given by its activation function, structure of the network, that is, architecture. How many hidden layers are there? How many neurons in each layer and so on? What are the connections? And then weight on each of these connections. So, to show using a simple example, this is how a simple neuron learns. So, let's say we have a full meal deal problem in which we have two inputs fries and burger, and the output Y represent whether it is a full meal or not? So, it is a full meal only when Fries and Burger both are taken. That is, they're both values are one. Any of them is zero, the output would be zero. So, a simple neuron learns this weighted function A is W1 X1 plus W2X2 plus W0 where W0 Is a DC value. W1 and W2 are the weights given to inputs X1. And X2 which are fries and burger. The output f(a) is a step function whose value is one, if the value of A is greater than zero. And its value is zero if a is less than zero. The perceptron learning algorithm in the step one initialize weights WI. Then it presents a pattern to the neuron. After that, it compute output using current value of the weights. Then it updates weight based on the difference between computed and desired output for the given pattern. The steps two to four are repeated until acceptable level of error is achieved. So, how do we update the weight? For this, we use delta rule for weight modification, in which at step T plus one, the weight WI is given as WIT, which is the weight at step T plus delta W. Where delta W is theta times XIT times T minus Y. Where T is the desired output, and Y is the current output. Theta which is between zero and one is the learning rate. To train a perceptron in Python from scratch, we define few classes. The first one is a neuron class, which calculate total net input. And then based on the threshold function, it calculate the output. A Neuron, Layer class, generates a layer of neurons and it gets outputs and then feed forward. The Neural Network, first initialize weight from input to the hidden layer network and then weight from hidden layer to the output. It then train the output neuron delta, hidden neuron delta, update output neuron weights. And based on that, update hidden neurons. Let's see an example in which our aim is to train a logical OR function, which is, Y is X1 or X 2. So, the value of Y is zero if and only if X1 and X2 both are zero. Otherwise it is one. So, as we can see, a simple neuron network can do this. So, here we have four inputs 00, 01, 10 and 11. So, the decision boundary can be drawn anywhere below the diagonal. So, it is able to form only linear discrimination function. That is, classes which can be divided by a line or a hyper plane. In this example, it can be easily done because only for values of 00 for X1, X2 or output is zero. Let's take a slightly complex problem in which our aim is to model the XOR function. Which is one only when X1 and X2 are different. That is, either X1 equals zero, X2 equals one or vice versa. In this, we will need a Multi Layer Neural Network consisting of hidden layers. So, two layers of neurons are needed because these functions cannot be discriminated by a single line. However, their combined result can produce good classification. In this video, we introduced the Neural Network through a perceptron and Artificial Neural Network. Thank you. [MUSIC]
