Neural Networks

Features and targets: Example

It is helpful to view the shape y as a decision function of x

--> non-linearly separable dataset

Use one dimension for simplicity

In the context of neural networks, its helpful to think of the classification problem as a decision function

We can represent the box, this box function is an example of a decision function

A neural network will approximate the function using learnable parameters

+ view the problem as trying to approximate the box function using logistic regression

anything in blue region y = one i.e dog

anything in red region y = zero i.e cat

If this was our cat-dog dataset, we cannot use a straight line to separate the data

Neural Networks

This line can be used to linearly separate same of the data, but some of the data is on the wrong side of the line

--> use the node to represent the line and the edges to represent the input x and output z

If we apply the logistic function, in the context of neural networks this is called the activation function

Represent the sigmoid function with the following node taking the input z from the linear function and producing an output

technically "A" is a function of z and x

Function "A" the activation function and the output of "A" is called the activation

this line looks like it can be use to separate the data

What happens when we apply the sigmoid function?

- after applying the sigmoid or activation function, we get an incorrect result for some of the samples

Consider the following sigmoid functions, we call them "A sub script one" and "A sub script two"

if we subtract the second, sigmoid function from the first sigmoid function we get something similar to the decision function

+ also apply the following operations with a linear function

i.e) just subtract the second activations from the first activation function

----> values will be learnable parameters

If we apply a threshold setting every value less than 0.5 to zero and greater than 0.5 to one

We get the exact function; we are trying to approximate

Then, classify the data, we obtain the parameters via gradient descent

--> use the graph to represent the process, we apply two linear functions to x and we get two outputs

--> to each linear function we apply a sigmoid

Then, we apply a second linear function to the outputs of the sigmoid

Usually, apply another function to the output of this linear function then apply a threshold

This diagram is used to represent a two-layer neural network, we have a hidden layer

Each linear function and activation is known as an artificial neuron

--> the hidden layer has two artificial neurons

The output layer has one artificial neuron, as it has two inputs; the input dimension for this neuron is two

The outputs of the activation function is a 2D plane that looks like this

--> it turns out that we can split the point using the following plane

In the same way, we can add more dimensions to the input, notice that there are a lot more weights between the input layer and hidden layer

These type of Neural Networks are called FeedForward Neural Networks or fully connected networks

More dimensions, more Neurals are required !!

Copyright Coursera All rights reserved