The cost landscape

When both w and b can change, the cost becomes a bowl-shaped surface. Learn to read it in 3D and as a contour map.

Lesson · 14 min

In this lesson you will

Picture J(w,b) as a surface over every possible pair of parameters
Read a contour plot as a map of that surface
Connect a single point on the map to a single line through the data

Last lesson you saw the cost as a simple curve, $J$ against $w$ , with $b$ held still. But a real model gets to change both parameters. Every pair $(w, b)$ is a different line, and each line has its own cost. Plot all of them and you get a landscape.

One point, one line, one cost

Here’s the key mental model for this lesson:

Every point $(w, b)$ in the parameter space is one line through the data.
The height of the landscape at that point is that line’s cost $J(w, b)$ .
The lowest point of the landscape is the best line.

For linear regression with squared error, the landscape is always a smooth bowl: it has a single bottom and no other dips to get stuck in. (That’s not true for the neural networks you’ll meet later, whose landscapes are much bumpier.)

Two ways to draw a landscape

A 3D surface is intuitive, but it hides things behind its own hills. So we usually draw it the way hikers do: as a contour map, looking straight down from above. Each ring connects points of equal cost, just as a contour line on a hiking map connects points of equal height. The middle of the smallest ring is the bottom of the bowl.

The three panels below are linked. Click anywhere on the contour map (or focus it and use the arrow keys) to pick a $(w, b)$ ; the left panel shows that line, and the ball on the 3D surface shows its cost. Drag the surface to turn it around.

Notice how the rings are long, stretched ellipses rather than circles. Changing $w$ changes the cost much faster than changing $b$ , because $w$ is multiplied by the house size in every prediction. That stretch will matter when we start walking downhill, and it’s the reason for the feature scaling trick you’ll learn in Section 2.