*To see this with sample code, check out my Kaggle notebook: https://www.kaggle.com/thaddeussegura/absolute-basics-simple-linear-regression

Linear Regression is “supervised”  “regression” algorithm.

Supervised meaning we use labeled data to train the model.

Regression meaning we predict a numerical value, instead of a “class”.

Implementing this algorithm, we fit a line using “least squares”, where we try to minimize the squared distances between the data-points and our line.

In its simplest form, we start by drawing any line through our data.

We measure the distances between the points and the line, square them, then sum them.  We’ll call this our “loss”.

Next, we rotate the line and calculate the loss. 

Repeating this step, we see how the loss changes based on the rotation.

We want the point at which the loss is minimized, which we find by taking the “derivative” of our “loss function”. Simply, this tells us the slope at any given point, so we calculate the value where the slope = 0.

We do this for both the “slope” and the “intercept”, which ultimately gives us our best fitting line.

Using the formula for our line, we can predict the output values for any input.

Overall, Linear Regression fast, intuitive, and a great starting point for simple regression problems, but its application is limited to 2-dimensional datasets.