Linear classifiers

What is linear classifiers?

It is widely used in classification and they are the building blocks of more advanced classification methods

Let's set y=0 to cat and y=1 dog

How a simple function can take an image as an input and output the image class using algebra?

How a simple function will take in images and input, and output a probability of how likely that image belongs to a class?

Learnable parameter

Z = WX + b

W = weight term

b = bias term

For arbitrary dimensions, this equation generalizes to a hyper-plane

Z = W1X1 + ..... WdXd + b

We can also represent this equation as a dot product of row vector w and image x

Z = WX + b

(called decision plane)

*Just a compact way to express the equation of a line in dots of dimensions*

Two dimensions for visualization

*when the letter x are not bold, it's just simple algebra, not a sample*

Set the weight W_1 to one W_2 to minus one and the bias to one

Z = 1X1 - 1X2 + 1

Plane when z equals zero

0 = 1X1 - 1X2 + 1

The line is where the decision plane intersects with the planes z equals zero

=> Decision boundary

Overlay the samples on the plane

Red and Blue circles = samples

Anything on the left side of the line = dog

Anything on the right side of the line = cat

How we can use the value of Z to determine if it's a dog or a cat?

Plug this value in the equation Z = 1X1 - 1X2 + 1

Z = 1(0) -- 1(-1) + 1

Z = 2

The value of Z is positive

Let's see the other sample's Z value

Every point of the left side of the line = positive value

Every point of the right side of the line = negative value

But, we need a class between zero and one

How do we convert these numbers?

--> Threshold function

Therefore, Z > 0 return to one and Z < 0 return to zero

Every sample on the right side of the line will be classified a cat

Every sample on the left side of the line will be classified a dog

*However, a plane can't always separate the data

The X7 sample is misclassified; the data is not linearly separable

Logistic Regression

Logistic function resembles the threshold function

= sigmoid function

This gives us a probability of how likely our estimate is

Plus, better performance than the threshold function

If the value of Z is very large negative number, the expression is approximately zero

If the value of Z is very large positive number, the expression is approximately one

Everything in the middle, the value is between zero and one

If the output of the logistic function is larger than 0.5, we set the prediction value y hat to one

If it's less, we set y hat to zero

*Also can represent the logistic function as a probability*

Find the probability of the image of being dog that is y hat = 1

Your app also use the linear classifier to make a prediction and output the class as a string