Run the three building blocks of machine learning and watch each turn out to be matrix arithmetic — first fit a line from scratch and read the slope the normal equations recover, then factor a matrix with the SVD and read how much energy its top few components carry, then push an input through a tiny neural network and read the forward-pass output. Predict each number before the screen shows it, and record your findings at the end.
Machine learning looks like three different subjects — regression, dimensionality reduction, neural networks — but underneath they are one language. A least-squares fit is the normal equations x̂ = (AᵀA)⁻¹Aᵀb, pure matrix algebra that recovers the exact line when the data lies on one. Matrix factorization is the SVD A = UΣVᵀ, and keeping the top few singular values is the low-rank model behind PCA and recommender systems. And a neural network's forward pass is just matrix-vector products through a nonlinearity, ŷ = σ(W₂ σ(W₁x)). This project is that unity: three stages, one engine.
Open the three sandboxes in tabs. Each step below names the one sandbox to read and the single choice to make in it; leave every other knob at its default so your numbers match these. Two sandboxes are in this course; the forward-pass stepper is borrowed from Machine Learning 101.
1 Least squares (LA 101) -> xs = 0 1 2 3 4, ys = 1 3 5 7 9 (points on y = 1 + 2x)
2 SVD compression (LA 101) -> rank-5 image, 64x64, seed 1, k = 3 3 Backprop (ML 101) -> 2-2-1 sigmoid, x = [1,0]
Read three numbers across the three tabs: the fitted slope in the least-squares panel, the “energy kept” readout in the SVD explorer, and the forward-pass output in the backprop stepper.
In the least-squares sandbox pick the exactly collinear dataset xs = 0 1 2 3 4, ys = 1 3 5 7 9 — the points lie on y = 1 + 2x. The fit solves the normal equations AᵀA x̂ = Aᵀb with no calculus at all. Predict the fitted slope c₁, then read it.
The slope is c₁ = 2 (with intercept c₀ = 1 and RMS residual 0): the normal equations recover the exact line when the points already lie on one. Linear regression is nothing but matrix algebra — no training loop needed. This is the demo's own lineFit() output, not a typed number.
Switch to the SVD explorer, choose the “rank-5” image (64×64, seed 1), set k = 3 and read the “energy kept” readout — the fraction (Σi≤3 σi2) / (Σi σi2) of the factorization A = UΣVᵀ. Predict whether three components already capture most of the energy.
The top 3 singular values already capture 0.9939 of the energy — over 99%. That is the whole idea of low-rank ML: a few factors of A = UΣVᵀ stand in for the whole matrix, which is how PCA compresses features and recommenders fill in missing entries. This is the demo's own svd() / explained-energy arithmetic.
Finally, open the backprop stepper (a Machine Learning 101 sandbox) and build the tiny 2-2-1 sigmoid network: x = [1, 0], W[1] = [[1,0],[0,0]], W[2] = [[1,1]]. The forward pass is matrix-vector products through σ: a[1] = [σ(1), σ(0)] = [0.7311, 0.5], then z[2] = 1.2311. Predict ŷ = σ(1.2311), then read it.
The output reads ŷ = 0.7740 — one number in (0, 1). That is the whole forward pass: a matrix times a vector, a nonlinearity, again, and one probability out. The same matrix arithmetic as stages one and two, now stacked into a network. This is the demo's own forward() output, not a typed number.
Open the three sandboxes and run each block yourself: pull a point off the line and watch the least-squares RMS climb, raise k and watch the SVD reconstruction sharpen, and step a value through the network and watch each layer's matrix multiply. Three demos, and the same matrix arithmetic underneath every one.
Each block produced one number that is really matrix arithmetic. Write them down — fill in each blank from the sandboxes, then add one sentence of reasoning. This is the deliverable; there is no number to read off the screen here.
Stage 1 Regression ...... xs/ys on y = 1 + 2x slope c1 ....... ______
RMS residual ... ______
Stage 2 Factorization ... rank-5 image, k = 3 energy kept .... ______
Stage 3 Forward pass .... 2-2-1 sigmoid, x = [1,0] output y-hat ... ______
Unity note ............... why is each stage just matrix arithmetic? ______
Design note .............. which matrix would you change first, and why? ______
Then change one thing — pull a point off the line, drop k, or edit a weight — and note which of the three numbers moved and by how much. That every stage responds to the same kind of edit, because every stage is the same matrix arithmetic, is the whole lesson of the project.