Linear regression: your first model

Time to build your first machine learning model. The task: a friend is selling a house of 1,500 square feet and asks what it might sell for. You have a list of recent sales in the neighbourhood. Can the data give a sensible answer?

The training set and its notation

The data a model learns from is its training set. Every machine learning text uses the same short names, so it’s worth learning them once:

• $x$ is the input (also called a feature): here, the size of a house in thousands of square feet.
• $y$ is the output or target: the price it sold for, in hundreds of thousands of dollars.
• $m$ is the number of training examples.
• $(x^{(i)}, y^{(i)})$ is the $i$-th training example. The $(i)$ is a label, not a power: $x^{(2)}$ just means “the size of the second house”.

So if the third house is 1,500 sq ft and sold for $310k, then $x^{(3)} = 1.5$ and $y^{(3)} = 3.1$.

The model: a straight line

The learning algorithm’s job is to produce a function $f$, the model, that takes a new $x$ and outputs a prediction. We write the prediction as $\hat{y}$ (“y hat”): our estimate of the true $y$.

The simplest model that can capture “bigger houses cost more” is a straight line:

$$f_{w,b}(x) = wx + b$$

It has two numbers we get to choose, called the model’s parameters:

• $w$, the weight or slope: how much the predicted price rises for each extra 1,000 sq ft.
• $b$, the bias or intercept: where the line crosses the vertical axis, the price it predicts for a size of zero (a baseline, not a real house).

Because there is a single input feature, this is called linear regression with one variable (or univariate linear regression).

Drag the line yourself

The dots are 24 real-looking sales. Move $w$ and $b$ with the sliders, or grab the line itself: drag its middle to shift it up and down, drag the round handle to tilt it. Then use the House size probe to read off the model’s prediction $\hat{y}$ for your friend’s house.

Loading the interactive graphic…

Play walkthroughStep 1 of 10

Each dot is a house that sold recently. Size runs along the bottom, and price runs up the side.

Some things worth noticing:

• Different values of $w$ and $b$ give different models and different predictions for the same house. Training means picking good ones.
• Even the best line doesn’t pass through every dot. Real prices depend on more than size, so a good model is one whose misses are small overall.
• With $w \approx 1.9$ and $b \approx 0.5$ the line hugs the data, and for 1,500 sq ft it predicts $\hat{y} \approx 3.3$, about $330k.

You found good values by eye. But “looks about right” is not something a computer can do. In the next lesson we turn “how good is this line?” into a single number the computer can make smaller.

Check yourself

Self-check · 4 questions

Linear regression

1. In the notation of this course, what does $x^{(3)}$ mean?

x cubedThe input of the third training exampleThree times xThe third feature of every example

2. In the model $f_{w,b}(x) = wx + b$, what does $w$ control?

The number of training examplesThe size of the errorsWhere the line crosses the vertical axisThe slope: how much the prediction changes when x increases by 1

3. With $w = 2$ and $b = 0.5$ (size in thousands of square feet, price in hundreds of thousands of dollars), what does the model predict for a 1,500 sq ft house?

3.0, about 300k dollars3.5, about 350k dollars4.0, about 400k dollars2.5, about 250k dollars

4. What does $m$ stand for?

The number of training examplesThe mean priceThe modelThe slope of the line

Check answers