Set Up the Data Before Calculating
Least-squares errors often begin before any equation is solved: a response vector is placed in the wrong orientation, the intercept column is omitted, or a residual is defined with the opposite sign from the one used in the calculation. These choices matter because matrix dimensions encode what each entry represents. A reliable setup starts by counting observations and coefficients, assigning one row to each observation, and stating the residual convention.
Suppose there are \(n\) observations and \(p\) coefficients. The design matrix \(\mathbf{A}\in\mathbb{R}^{n\times p}\) has one row per observation and one column per coefficient. The response vector \(\mathbf{y}\in\mathbb{R}^n\) and coefficient vector \(\mathbf{x}\in\mathbb{R}^p\) are columns. Thus \(\mathbf{A}\mathbf{x}\in\mathbb{R}^n\) is the vector of fitted responses. For a model with an intercept, one column of \(\mathbf{A}\) consists of \(n\) ones; the corresponding entry of \(\mathbf{x}\) is the intercept.
The order of the columns determines the order of the coefficients. For example, if the columns are the intercept, \(x_1\), and \(x_2\), then the coefficient vector must be \((a,b_1,b_2)^\top\), not \((b_1,a,b_2)^\top\). Matrix multiplication applies each coefficient to its corresponding column. Writing down that correspondence prevents a dimensionally valid calculation from nevertheless describing the wrong model.
Dimensions and the Transpose in the Normal Equations
For an \(n\times p\) design matrix, \(\mathbf{A}\mathbf{x}=\mathbf{y}\) has matching dimensions: both sides are \(n\times1\). The least-squares normal equations are \(\mathbf{A}^{\top}\mathbf{A}\widehat{\mathbf{x}}=\mathbf{A}^{\top}\mathbf{y}\). Here \(\mathbf{A}^{\top}\mathbf{A}\) is \(p\times p\), and \(\mathbf{A}^{\top}\mathbf{y}\) is \(p\times1\). In contrast, \(\mathbf{A}\mathbf{y}\) is usually not defined, because an \(n\times p\) matrix can multiply an \(n\times1\) vector only when \(p=n\). Even if \(p=n\), that product does not take the place of \(\mathbf{A}^{\top}\mathbf{y}\).
Proof. Write \(\mathbf{r}_0=\mathbf{y}-\mathbf{A}\mathbf{x}_0\). First suppose \(\mathbf{A}^{\top}\mathbf{r}_0=\mathbf{0}\). For any other coefficient vector \(\mathbf{x}_0+\mathbf{h}\), its residual is \(\mathbf{r}_0-\mathbf{A}\mathbf{h}\). Expanding its squared norm gives
Since \(\mathbf{r}_0^\top\mathbf{A}\mathbf{h}=(\mathbf{A}^{\top}\mathbf{r}_0)^\top\mathbf{h}=0\), this equals \(\|\mathbf{r}_0\|^2+\|\mathbf{A}\mathbf{h}\|^2\), which is at least \(\|\mathbf{r}_0\|^2\). So \(\mathbf{x}_0\) minimizes the residual norm. Conversely, suppose \(\mathbf{x}_0\) minimizes it. Fix any \(\mathbf{h}\in\mathbb{R}^p\) and put \(\mathbf{v}=\mathbf{A}\mathbf{h}\). For every real \(t\), minimality gives \(\|\mathbf{r}_0-t\mathbf{v}\|^2\geq\|\mathbf{r}_0\|^2\). If \(\mathbf{v}\ne\mathbf{0}\), the left side minus the right side is \(-2t\mathbf{r}_0^\top\mathbf{v}+t^2\|\mathbf{v}\|^2\). This cannot be nonnegative for every \(t\) unless \(\mathbf{r}_0^\top\mathbf{v}=0\): if that inner product were nonzero, a sufficiently small \(t\) with the same sign would make the expression negative. If \(\mathbf{v}=\mathbf{0}\), its inner product with \(\mathbf{r}_0\) is already zero. Therefore \(\mathbf{r}_0^\top\mathbf{A}\mathbf{h}=0\) for every \(\mathbf{h}\). Taking \(\mathbf{h}\) to be each standard basis vector in \(\mathbb{R}^p\) shows that every entry of \(\mathbf{A}^{\top}\mathbf{r}_0\) is zero. This proves the reverse implication. \(\square\)
Worked Example: A Line Fit and a Dimension Check
Fit a line with intercept to the data \((0,1),(1,2),(2,4)\). Let \(a\) be the intercept and \(b\) the slope, so the fitted response is \(a+bx\). The design matrix has three rows and two columns, because there are three observations and two coefficients:
The product \(\mathbf{A}\mathbf{x}\) has size \(3\times1\), as required. The tempting product \(\mathbf{A}\mathbf{y}\) is not defined: its inner dimensions are \(2\) and \(3\). Instead, the normal equations use \(\mathbf{A}^{\top}\mathbf{y}\). Compute both sides’ coefficient matrices and right-hand side:
Thus \(3a+3b=7\) and \(3a+5b=10\). Subtracting the first equation from the second gives \(2b=3\), so \(b=3/2\). Substitution into the first gives \(3a+9/2=7\), hence \(3a=5/2\) and \(a=5/6\). The fitted vector and residual are
Check the normal equations through residual orthogonality: the residual entries sum to \(1/6-1/3+1/6=0\), and their products with the \(x\)-values sum to \(0(1/6)+1(-1/3)+2(1/6)=0\). Therefore \(\mathbf{A}^{\top}\mathbf{r}=\mathbf{0}\), verifying the least-squares condition.
Do Not Omit an Intercept by Accident
For a model that allows an intercept, the column of ones is part of the design matrix. Removing it does not merely shorten the calculation; it changes the model by forcing every fitted value to be zero when all predictors are zero. A no-intercept fit may be appropriate when that restriction is intended, but it is not interchangeable with a fit that includes an intercept.
Worked Example: Comparing Fits With and Without an Intercept
Use the data \((0,2),(1,3),(2,5)\). First fit the model through the origin, \(y=bx\). Its design matrix has one column, \((0,1,2)^\top\), so \(b\) is a scalar. The normal equation is \((\mathbf{A}^{\top}\mathbf{A})b=\mathbf{A}^{\top}\mathbf{y}\), with \(\mathbf{A}^{\top}\mathbf{A}=0^2+1^2+2^2=5\) and \(\mathbf{A}^{\top}\mathbf{y}=0(2)+1(3)+2(5)=13\). Therefore \(b=13/5\). The fitted vector and residual are
The residual is orthogonal to the predictor column because \(0(2)+1(2/5)+2(-1/5)=0\). Its squared norm is \(2^2+(2/5)^2+(-1/5)^2=21/5\). This is the best fit within the restricted through-origin model, not a fit with an intercept.
Now include the intercept, so \(\mathbf{A}=\begin{bmatrix}1&0\\1&1\\1&2\end{bmatrix}\). Then \(\mathbf{A}^{\top}\mathbf{A}=\begin{bmatrix}3&3\\3&5\end{bmatrix}\) and \(\mathbf{A}^{\top}\mathbf{y}=\begin{bmatrix}10\\13\end{bmatrix}\). The equations are \(3a+3b=10\) and \(3a+5b=13\). Their difference gives \(2b=3\), so \(b=3/2\), and then \(3a+9/2=10\), giving \(a=11/6\). The fitted values are \(11/6,10/3,29/6\), so the residual is \((1/6,-1/3,1/6)^\top\). Its entries sum to zero, and their products with \(0,1,2\) sum to \(0-1/3+1/3=0\). Both columns are orthogonal to the residual, as required. The different coefficients arise because including the intercept changes the model.
Residual Signs and Exact Solutions
The convention here is \(\mathbf{r}=\mathbf{y}-\mathbf{A}\mathbf{x}\). Some calculations instead call \(\mathbf{A}\mathbf{x}-\mathbf{y}\) the residual. That alternative is the negative of \(\mathbf{r}\), so it has the same norm and is orthogonal to the columns exactly when \(\mathbf{r}\) is. Problems arise when a sign from one convention is substituted into a formula written for the other. State the convention, subtract in that order, and check by computing \(\mathbf{y}-\mathbf{A}\widehat{\mathbf{x}}\) entry by entry.
Proof. By the residual definition, \(\mathbf{r}=\mathbf{y}-\mathbf{A}\widehat{\mathbf{x}}\). Thus \(\mathbf{r}=\mathbf{0}\) holds exactly when rearranging gives \(\mathbf{A}\widehat{\mathbf{x}}=\mathbf{y}\). If an exact solution exists, its residual norm is zero. Since norms are nonnegative, no residual norm can be smaller, so the least-squares minimum is zero. Conversely, if the minimum residual norm is zero, a least-squares solution has \(\|\mathbf{y}-\mathbf{A}\widehat{\mathbf{x}}\|=0\). A vector has norm zero only when it is the zero vector, so this solution is exact. \(\square\)
Worked Example: A Best Fit That Is Not Exact
Consider the two equations represented by \(\mathbf{A}=\begin{bmatrix}1\\1\end{bmatrix}\) and \(\mathbf{y}=\begin{bmatrix}1\\3\end{bmatrix}\). An exact solution would require both \(x=1\) and \(x=3\), which is impossible. For the least-squares solution, the normal equation is \((\mathbf{A}^{\top}\mathbf{A})x=\mathbf{A}^{\top}\mathbf{y}\). The left coefficient is \(1^2+1^2=2\), and the right side is \(1(1)+1(3)=4\), so \(x=2\).
The fitted vector is \(\mathbf{A}x=(2,2)^\top\), and the residual under our convention is \(\mathbf{r}=(-1,1)^\top\). Its norm is \(\sqrt{(-1)^2+1^2}=\sqrt{2}\), which is nonzero, so the fit is not exact. The orthogonality check gives \(\mathbf{A}^{\top}\mathbf{r}=1(-1)+1(1)=0\), confirming that \(x=2\) is a least-squares solution. It minimizes the discrepancy; it does not make the inconsistent equations true.
Common Mistakes to Catch
Wrong transpose or incompatible dimensions. For the \(3\times2\) matrix in the line-fit example, writing \(\mathbf{A}\mathbf{y}\) attempts to multiply a \(3\times2\) matrix by a \(3\times1\) vector, so the product is undefined. The correction is \(\mathbf{A}^{\top}\mathbf{y}\), which is \(2\times1\), matching the two coefficients. Check the inner dimensions before multiplying, then check that the result has one entry per coefficient.
Missing intercept. If a model is intended to include an intercept, using only the predictor column silently forces the fit through the origin. In the second example that gives \(b=13/5\), whereas including the ones column gives \(a=11/6\) and \(b=3/2\). The correction is to decide whether the model includes an intercept before building \(\mathbf{A}\), then include the ones column when it does.
Sign mismatch in the residual. With \(\mathbf{r}=\mathbf{y}-\mathbf{A}\mathbf{x}\), reversing the subtraction gives the negative residual. For the fit \(\widehat{\mathbf{x}}=(5/6,3/2)^\top\), the first residual is \(1-5/6=1/6\), not \(5/6-1=-1/6\) under this convention. Either convention yields the same norm, but mixing the two reverses the residual entries. The correction is to write the chosen subtraction explicitly before evaluating it.
Claiming the fit must solve every equation exactly. The system \(x=1,\ x=3\) has no exact solution, but its least-squares solution \(x=2\) has residual \((-1,1)^\top\) and satisfies the orthogonality test. A zero residual is necessary for an exact fit; orthogonality alone means only that the residual is minimized.
Check Your Understanding
For each question, state dimensions or show the relevant calculation so the setup can be checked.
- A model has six observations, two predictors, and an intercept. What are the dimensions of \(\mathbf{A}\), \(\mathbf{x}\), and \(\mathbf{y}\)?
- If \(\mathbf{A}\) is \(5\times3\), which of \(\mathbf{A}\mathbf{y}\) and \(\mathbf{A}^{\top}\mathbf{y}\) has compatible dimensions when \(\mathbf{y}\in\mathbb{R}^5\)? State the output size.
- For the convention \(\mathbf{r}=\mathbf{y}-\mathbf{A}\mathbf{x}\), write the fitted and residual vectors for data responses \((2,4)^\top\) and fitted responses \((3,3)^\top\).
- Can a nonzero residual be consistent with a least-squares solution? Explain what condition the residual must satisfy.
- What does including a ones column allow the model to do that a through-origin model cannot?
