Least Squares as Physics — Why the Best Fit Is Just Equilibrium

You can't make this shelf fair. Seven nails disagree about where it belongs; lift it and the left side complains, tilt it and the right side does. Every fix breaks somebody — so stop fixing it. Let go. It swings, settles, and stops at the one pose where nobody is happy and everybody is least unhappy. No one solved anything. The shelf did. That is least squares — not as a formula you compute, but as something the physical world falls into on its own. This is the companion to "The Shadow It's Allowed to Cast": where that film showed what the best fit is (a projection), this one shows why the world keeps finding it — because squared error is elastic energy, and a system sheds energy until the forces balance. What we build, from springs up • Hooke's law: a stretched band stores ½ k (stretch)² — nature doesn't choose to square the error, physics squares it for you • The energy landscape E(a,b): one convex valley, and "letting go" is literally gradient descent (slow, damped rolling downhill) • The normal equations as statics: at rest the vertical pulls cancel (∑r = 0, force) and the twists cancel (∑xᵢrᵢ = 0, torque) — written down, those two balance conditions ARE XᵀXβ̂ = Xᵀy • Stiffness = trust: stiffer bands (k = 1/variance) give weighted least squares; a constant-tension cord gives L1 / the median • The soft trough: near-collinear parameters make the valley a long flat groove — the coefficients wander and settle last (that ratio is the condition number κ), and a spring on the dial is ridge regression • The crown — take the guide wires off: let the bands slide freely and they snap perpendicular, and the same shelf now minimizes perpendicular error — total least squares, the errors-in-variables fit, the line PCA would draw • Warmth: heat the shelf and the poses it visits trace a Gaussian — the physical reason least squares equals maximum likelihood — and the honest limit, that the bands only ever feel stretch, never whether a nail was in the right place Chapters 0:00 — A shelf you can't make fair 0:39 — The workshop: nails, rod, rubber bands 1:21 — Hooke's law — why nature squares the error 1:57 — Let go 2:17 — The valley — settling is gradient descent 3:00 — Force + torque = the normal equations 3:48 — Stiff bands and slack bands 4:09 — Stiffness is trust — weighting & the median 5:04 — The soft trough — collinearity, κ, and ridge 5:56 — The guide wires come off — total least squares 7:06 — Warmth — why the bowl is Gaussian 7:55 — Recap: every thread, one shelf 8:40 — Let go, and the errors square themselves away Topics: least squares, ordinary least squares (OLS), the normal equations, linear regression, elastic energy, Hooke's law, energy minimization, equilibrium, gradient descent, force and torque balance, weighted least squares, the median / L1, condition number, collinearity, ridge regression / Tikhonov regularization, total least squares, errors-in-variables, Deming regression, PCA, regression dilution, the Boltzmann distribution, maximum likelihood, Gauss–Markov, curve fitting, pharmacometrics, quantitative systems pharmacology. New here? QSPplus makes cinematic explainers on pharmacometrics, statistics, and machine learning — one hard idea, one honest metaphor at a time. This is the physics companion to our least-squares film "The Shadow It's Allowed to Cast." Subscribe for more. #LeastSquares #Physics #MachineLearning