Vectorisation: doing it all at once

The model from the last lesson is a dot product: $\vec{w} \cdot \vec{x} + b$. There are two ways to compute one. You can walk through the features one by one with a loop, or you can hand the two whole arrays to NumPy and let it do the multiplying and adding in one call. The second way is called vectorisation, and it’s one of the quiet reasons machine learning became practical.

The loop way and the vector way

Here is the dot product written as a loop, exactly as the sum $\sum_{j=1}^{n} w_j x_j$ reads:

import numpy as np

def predict_loop(x, w, b):
    total = 0.0
    for j in range(len(x)):       # one feature at a time
        total = total + w[j] * x[j]
    return total + b

def predict_vec(x, w, b):
    return np.dot(w, x) + b       # all features at once

x = np.array([1200, 2, 1, 40])
w = np.array([0.2, 10, 5, -1])
print(predict_loop(x, w, 30), predict_vec(x, w, 30))

Same answer. The vectorised version is one line instead of four, and there’s no loop index to get wrong. But the bigger difference only shows up when the vectors get long.

Race them

Modern models have thousands or millions of features and parameters. Let’s make two vectors with 200,000 entries each and time both versions. (You timed loops against arrays in Python for Data, section 1; this is the same experiment with an ML flavour.)

import time

rng = np.random.default_rng(0)
n = 200_000
x_big = rng.normal(size=n)
w_big = rng.normal(size=n)

start = time.perf_counter()
slow = predict_loop(x_big, w_big, 0.0)
loop_time = time.perf_counter() - start

start = time.perf_counter()
fast = predict_vec(x_big, w_big, 0.0)
vec_time = time.perf_counter() - start

print(f"loop:   {loop_time * 1000:8.2f} ms")
print(f"np.dot: {vec_time * 1000:8.2f} ms")
print(f"np.dot was about {loop_time / vec_time:,.0f} times faster")
print("same answer?", np.isclose(slow, fast))

On most machines np.dot wins by a factor of a hundred or more. (Your exact numbers depend on your computer, and the browser adds its own overhead, so run it a couple of times.)

Why is it so much faster?

Three reasons stack up:

1. No Python per step. In the loop, Python has to look up w[j], check its type, create a new number object for the product, and so on, two hundred thousand times. np.dot makes one call into compiled code that runs the whole sum at machine speed.
2. Numbers packed together. A NumPy array stores its numbers side by side in memory, all of the same type, so the processor can stream through them without detours.
3. Parallel hardware. Modern processors can multiply several pairs of numbers in a single instruction, and GPUs can do thousands at once. Vectorised code lets the library use that; a Python loop can’t.

That third point is why deep learning runs on GPUs. A neural network’s training is mostly huge vectorised multiplications, which is exactly what graphics chips were built for.

All examples at once: X @ w + b

Vectorisation doesn’t stop at one example. Stack the $m$ examples as rows of a matrix $X$ with shape $(m, n)$. Multiplying $X$ by the weight vector computes one dot product per row, which is one prediction per house:

In NumPy the @ operator is matrix multiplication (Math for ML, section 2), and adding the single number b to the result adds it to every entry, thanks to broadcasting.

X = np.array([
    [1850, 3, 2, 12],
    [1200, 2, 1, 40],
    [2400, 4, 2, 8],
    [950, 2, 1, 55],
    [1600, 3, 1, 25],
    [3100, 5, 2, 3],
])
y = np.array([412, 236, 538, 178, 330, 690])
b = 30

y_hat = X @ w + b
print("shapes:", X.shape, "@", w.shape, "->", y_hat.shape)
print("predictions:", y_hat)
print("errors:     ", y_hat - y)

Read the shapes as a sentence: $(6, 4)$ times $(4,)$ gives $(6,)$, one number per house. The inner sizes (4 features, 4 weights) must match. If you accidentally build w with 3 entries, NumPy refuses with an error that mentions mismatched dimensions. That error is your friend: it catches a whole class of bugs that a loop would silently get wrong.

Your turn

Compute every prediction and the error for each house without writing a single loop.

import numpy as np

X = np.array([
    [1850, 3, 2, 12],
    [1200, 2, 1, 40],
    [2400, 4, 2, 8],
    [950, 2, 1, 55],
    [1600, 3, 1, 25],
    [3100, 5, 2, 3],
], dtype=float)
y = np.array([412, 236, 538, 178, 330, 690], dtype=float)
w = np.array([0.2, 10, 5, -1])
b = 30

y_hat = None
errors = None
worst = None
print(y_hat, errors, worst)

Check yourself