You Never Fit the Data. You Fit Its Shadow.

Your phone is running least squares on you right now. A handful of satellites each swear you're a certain distance away — and they disagree. No point sits on every circle at once, so your phone doesn't find where you truly are; it finds the place that's *least wrong*. That's the whole idea. Least squares is the quiet workhorse under linear regression, curve fitting, and the linear layers of neural networks — and it has one of the most beautiful geometric pictures in all of mathematics. With a single controlling metaphor — a shadow cast onto the ground — we build it from intuition up: you never really fit the data, you fit the closest thing your model is allowed to become, and the error always points straight home. What we cover • The objective: why we minimize the sum of squared residuals — and that squaring is a choice • The geometry: the fit ŷ = Xβ̂ is the orthogonal projection of the data onto the column space, and the residual is perpendicular to everything the model can reach (Xᵀr = 0) • The exact address: the normal equations XᵀXβ̂ = Xᵀy, the closed form (the pseudoinverse), and why least squares can jump to the answer while gradient descent gropes • The hat matrix H = X(XᵀX)⁻¹Xᵀ and why a shadow of a shadow doesn't move (H² = H) • The ruler is a choice — straight-line distance gives L2 (the mean), taxicab gives L1 (the median), and one outlier can bend least squares while L1 barely flinches • Weighted / generalized least squares — trusting reliable points more (w = 1/variance) • Collinearity: when a perfect fit still lies — ŷ stays pinned while the coefficients explode (κ(XᵀX) ≫ 1) — and ridge as the gentle leash • The probabilistic view: under round Gaussian noise least squares is maximum likelihood, it's the Best Linear Unbiased Estimator (Gauss–Markov), and when the truth isn't in the model the leftover is bias, not noise — the best answer the model can give, not the right answer Bilingual throughout: every idea is named the way statisticians and the machine-learning / pharmacometrics worlds say it. Chapters 0:00 — Your phone is running least squares on you 0:32 — The field: you, the path, the data 1:12 — The objective — the sum of squared residuals 1:47 — Noon on the clearing: the shadow 2:14 — The projection & the right angle — Xᵀr = 0 2:56 — The exact address — the normal equations & the hat matrix 3:50 — Two rulers: how do you measure “far”? 4:14 — The ruler is a choice — L2 vs L1 5:01 — Trust unequally — weighted least squares 5:33 — The creased plane — collinearity & ridge 6:22 — Is the shadow the truth? — MLE, BLUE, and the limit 7:20 — Recap: every thread, one shadow 8:06 — The best fit is a shadow Topics: least squares, ordinary least squares (OLS), linear regression, the normal equations, orthogonal projection, column space, the hat matrix, leverage, pseudoinverse, R², weighted and generalized least squares (WLS/GLS), L1 vs L2 / least absolute deviations, robustness and outliers, collinearity and the condition number, ridge regression / Tikhonov regularization, maximum likelihood, the Gauss–Markov theorem (BLUE), model bias, curve fitting, pharmacometrics, quantitative systems pharmacology. New here? QSPplus makes cinematic explainers on statistics and machine learning — one hard idea, one honest metaphor at a time. Subscribe for more. #LeastSquares #LinearRegression #MachineLearning