Why Audit a Least-Squares Calculation?
A regression report can contain several linked quantities: an intercept and slope, fitted values, residuals, and sums of squares. These quantities are not independent. A small arithmetic error in one fitted value, for example, can make the reported residual list inconsistent with the reported line. The defining identities of ordinary least squares (OLS) provide a systematic way to check such calculations.
This tutorial concerns simple linear regression with an intercept. The identities are algebraic properties of the fitted line; they do not require a probability model, normally distributed errors, or constant error variance. They apply to the observed data when the line is fitted by minimizing the sum of squared residuals and the predictor values are not all identical.
When \(S_{xx}>0\), the OLS coefficients are \(\hat{\beta}_1=S_{xy}/S_{xx}\) and \(\hat{\beta}_0=\bar{y}-\hat{\beta}_1\bar{x}\). If \(S_{xx}=0\), all predictor values are the same and the slope is not identifiable from this calculation. The identities below assume a fitted line with an intercept; they need not hold for a line constrained to pass through the origin.
The Identities That Make an Audit Possible
For an OLS line with an intercept, the following checks must hold, apart from small discrepancies caused by rounding in a report:
- Centroid: \(\hat{\beta}_0+\hat{\beta}_1\bar{x}=\bar{y}\). Equivalently, the mean of the fitted values is \(\bar{y}\).
- Zero-sum residuals: \(\sum_{i=1}^n e_i=0\).
- Predictor orthogonality: \(\sum_{i=1}^n(x_i-\bar{x})e_i=0\), and therefore \(\sum_{i=1}^n x_ie_i=0\).
- Sum-of-squares decomposition: \(\mathrm{SST}=\mathrm{SSR}+\mathrm{SSE}\).
The centroid and zero-sum checks are related: since \(\hat{y}_i=\hat{\beta}_0+\hat{\beta}_1x_i\), the average fitted value is \(\hat{\beta}_0+\hat{\beta}_1\bar{x}\). If it equals \(\bar{y}\), then \(\sum e_i=\sum(y_i-\hat{y}_i)=n\bar{y}-n\bar{y}=0\). They are still useful as separate audit steps: one checks the coefficients against the means, and the other checks the reported residuals.
Why the OLS Checks Hold
The identities follow from the least-squares minimization itself. Write the sum of squared residuals for candidate coefficients \(a\) and \(b\) as \(Q(a,b)=\sum_{i=1}^n(y_i-a-bx_i)^2\). At an OLS minimum, the partial derivatives with respect to both coefficients are zero. These two conditions give the normal equations:
Subtracting \(\bar{x}\sum e_i=0\) from the second equation gives \(\sum(x_i-\bar{x})e_i=0\). The first equation also gives \(\bar{y}=\overline{\hat{y}}=\hat{\beta}_0+\hat{\beta}_1\bar{x}\), proving that the line passes through \((\bar{x},\bar{y})\).
For the sums-of-squares identity, write each deviation from the response mean as \(y_i-\bar{y}=(\hat{y}_i-\bar{y})+e_i\). Squaring and summing produces the two sums of squares plus a cross-product. The cross-product is zero because the fitted values are a linear combination of a constant and \(x_i\), both of which are orthogonal to the residuals:
Thus \(\mathrm{SST}=\mathrm{SSR}+\mathrm{SSE}\). This derivation also explains why the decomposition is tied to OLS with an intercept: the cross-product vanishes because of the residual orthogonality conditions.
A Practical Audit Sequence
When checking a reported calculation, use the identities in a deliberate order. A failed early check can help locate the source of a later discrepancy.
Compute \(\bar{x}\), \(\bar{y}\), \(S_{xx}\), and \(S_{xy}\) from the data. Verify the reported slope and intercept, then check \(\hat{\beta}_0+\hat{\beta}_1\bar{x}=\bar{y}\).
For each observation, calculate \(\hat{y}_i=\hat{\beta}_0+\hat{\beta}_1x_i\), then use \(e_i=y_i-\hat{y}_i\). Keep the observed-minus-fitted sign consistent.
Sum the residuals and calculate \(\sum x_ie_i\) or \(\sum(x_i-\bar{x})e_i\). Both should be zero for an OLS line with an intercept.
Compute \(\mathrm{SSE}\), \(\mathrm{SSR}\), and \(\mathrm{SST}\) using their definitions, then verify \(\mathrm{SSE}+\mathrm{SSR}=\mathrm{SST}\).
Worked Example: Correcting a Reported Line and Sums of Squares
For the observations \((2,5),(4,7),(6,10)\), a draft report gives \(\hat{y}=3+\frac{13}{12}x\), \(\mathrm{SSE}=\frac{7}{18}\), \(\mathrm{SSR}=\frac{173}{18}\), and \(\mathrm{SST}=10\). Audit and correct the report. Method: recompute the OLS coefficients from \(S_{xx}\) and \(S_{xy}\), then check fitted values, residuals, and sums of squares. This avoids accepting numbers merely because two reported components appear to add to a third.
First calculate the means:
The centered predictor values are \(-2,0,2\), so \(S_{xx}=(-2)^2+0^2+2^2=8\). The centered response values are \(-\frac{7}{3},-\frac{1}{3},\frac{8}{3}\), giving
Therefore \(\hat{\beta}_1=10/8=5/4\), and \(\hat{\beta}_0=22/3-(5/4)(4)=22/3-5=7/3\). The corrected line is \(\hat{y}=\frac{7}{3}+\frac{5}{4}x\). Its centroid check is \(\frac{7}{3}+\frac{5}{4}(4)=\frac{22}{3}=\bar{y}\).
The fitted values and residuals are:
The residuals sum to \(\frac{1}{6}-\frac{1}{3}+\frac{1}{6}=0\). Predictor orthogonality also holds: \((-2)(\frac{1}{6})+0(-\frac{1}{3})+2(\frac{1}{6})=0\). Next, \(\mathrm{SSE}=(\frac{1}{6})^2+(-\frac{1}{3})^2+(\frac{1}{6})^2=\frac{1}{36}+\frac{4}{36}+\frac{1}{36}=\frac{1}{6}\).
The fitted deviations from \(\bar{y}=22/3\) are \(-5/2,0,5/2\). Hence \(\mathrm{SSR}=25/4+0+25/4=25/2\). The response deviations give \(\mathrm{SST}=38/3\). Finally, \(\mathrm{SSE}+\mathrm{SSR}=\frac{1}{6}+\frac{25}{2}=\frac{1}{6}+\frac{75}{6}=\frac{76}{6}=\frac{38}{3}\), as required. The draft’s coefficients and sums of squares are incorrect; the corrected values pass all checks.
Worked Example: Finding an Error in a Fitted-Value Table
For observations \((1,3),(2,5),(3,4),(4,8)\), a report lists the line \(\hat{y}=1.5+1.4x\), fitted values \(2.9,4.3,5.7,7.0\), and residuals \(0.1,0.7,-1.7,1.0\). Determine whether the report is internally consistent and give the correct audit quantities. Method: verify the line from the data, then recompute each fitted value and residual. This avoids treating a residual list as correct just because it appears to sum to zero.
The means are \(\bar{x}=(1+2+3+4)/4=2.5\) and \(\bar{y}=(3+5+4+8)/4=5\). The centered predictor values are \(-1.5,-0.5,0.5,1.5\), so \(S_{xx}=2.25+0.25+0.25+2.25=5\). The centered response values are \(-2,0,-1,3\), so
Thus \(\hat{\beta}_1=7/5=1.4\) and \(\hat{\beta}_0=5-(1.4)(2.5)=1.5\); the reported line is correct and passes through the centroid. Applying it gives fitted values \(2.9,4.3,5.7,7.1\). Subtracting those from the observed responses gives residuals \(0.1,0.7,-1.7,0.9\). Only the last fitted value and last residual in the report are wrong.
The corrected residuals sum to \(0.1+0.7-1.7+0.9=0\). Predictor orthogonality using centered \(x\)-values is \((-1.5)(0.1)+(-0.5)(0.7)+(0.5)(-1.7)+(1.5)(0.9)=-0.15-0.35-0.85+1.35=0\). The sums of squares are \(\mathrm{SSE}=0.01+0.49+2.89+0.81=4.2\), \(\mathrm{SST}=(-2)^2+0^2+(-1)^2+3^2=14\), and \(\mathrm{SSR}=(-2.1)^2+(-0.7)^2+(0.7)^2+(2.1)^2=9.8\). The check \(4.2+9.8=14\) confirms the corrected table.
Worked Example: Auditing All Four Identities
A data analyst reports a fitted line for \((0,2),(1,3),(2,7),(3,8)\). Verify the coefficients, residual identities, and sums-of-squares decomposition. Method: calculate each quantity from the observations and use independent OLS identities as cross-checks. This avoids relying on a single check, which may fail to detect more than one arithmetic error.
Here \(\bar{x}=(0+1+2+3)/4=1.5\) and \(\bar{y}=(2+3+7+8)/4=5\). The centered predictor values are \(-1.5,-0.5,0.5,1.5\), so \(S_{xx}=2.25+0.25+0.25+2.25=5\). The centered response values are \(-3,-2,2,3\), giving \(S_{xy}=4.5+1+1+4.5=11\). Therefore the slope is \(11/5=2.2\), and the intercept is \(5-(2.2)(1.5)=1.7\). The centroid check is \(1.7+(2.2)(1.5)=5\).
The fitted values are \(1.7,3.9,6.1,8.3\); the observed-minus-fitted residuals are \(0.3,-0.9,0.9,-0.3\). They sum to \(0.3-0.9+0.9-0.3=0\). Their centered predictor cross-product is \((-1.5)(0.3)+(-0.5)(-0.9)+(0.5)(0.9)+(1.5)(-0.3)=-0.45+0.45+0.45-0.45=0\).
The residual sum of squares is \(\mathrm{SSE}=0.09+0.81+0.81+0.09=1.8\). The total sum of squares is \(\mathrm{SST}=(-3)^2+(-2)^2+2^2+3^2=26\). The fitted deviations from the mean are \(-3.3,-1.1,1.1,3.3\), so \(\mathrm{SSR}=10.89+1.21+1.21+10.89=24.2\). The decomposition check is \(1.8+24.2=26\), and the reported line and associated calculations are consistent.
Common Exam Traps
The checks are simple, but several recurring errors can make an otherwise correct calculation look wrong—or allow an incorrect report to go unnoticed.
- Reversing the residual sign. The convention here is \(e_i=y_i-\hat{y}_i\), observed minus fitted. Using fitted minus observed reverses every residual and changes cross-checks that use residual signs.
- Checking only that residuals sum to zero. A list can sum to zero and still fail predictor orthogonality. Check both conditions, as well as whether the listed values actually equal observed minus fitted.
- Using the wrong definition of a sum of squares. \(\mathrm{SSR}\) is the squared deviation of fitted values from \(\bar{y}\); \(\mathrm{SSE}\) is the squared residual total. Neither is the sum of the fitted values or the sum of the residuals.
- Applying the decomposition to a line through the origin. Without an intercept, residuals need not sum to zero, the line need not pass through the data centroid, and the stated decomposition may not hold in this form.
- Expecting displayed rounded numbers to satisfy exact identities. If coefficients or residuals have been rounded, small discrepancies can arise. Recompute from unrounded values when available; a large discrepancy is not explained by ordinary rounding.
Check Your Understanding
For each problem, audit the line or reported quantities using the OLS identities. Treat all displayed data as exact.
- For \((0,1),(1,4),(2,3)\), find the OLS line, the residuals, and \(\mathrm{SSE}\), \(\mathrm{SSR}\), and \(\mathrm{SST}\). Verify the decomposition. (Answer: \(\hat{y}=\frac{5}{3}+x\); residuals \(-\frac{2}{3},\frac{4}{3},-\frac{2}{3}\); \(\mathrm{SSE}=\frac{8}{3}\), \(\mathrm{SSR}=2\), \(\mathrm{SST}=\frac{14}{3}\), and \(\frac{8}{3}+2=\frac{14}{3}\).)
- For \((1,2),(2,3),(3,7)\), calculate the OLS coefficients and check the residual sum and predictor orthogonality. (Answer: \(\hat{y}=-1+2.5x\); residuals \(0.5,-1,0.5\); their sum and \(\sum(x_i-\bar{x})e_i\) are both \(0\).)
- For \((-1,2),(0,2),(1,5)\), a report gives \(\mathrm{SSE}=2.5\) and \(\mathrm{SSR}=3.5\). Compute the OLS line and the correct sums of squares, and assess the report. (Answer: \(\hat{y}=3+1.5x\); residuals \(0.5,-1,0.5\); \(\mathrm{SSE}=1.5\), \(\mathrm{SSR}=4.5\), \(\mathrm{SST}=6\); the reported components add to 6 but are individually incorrect.)
- For \((1,4),(2,4),(3,7),(4,9)\), calculate the line and check whether residuals \(0.7,-1.1,0.1,0.3\) are consistent with it. Then verify the sums-of-squares decomposition. (Answer: \(\hat{y}=1.5+1.8x\); the listed residuals are correct; \(\mathrm{SSE}=1.8\), \(\mathrm{SSR}=16.2\), \(\mathrm{SST}=18\), and \(1.8+16.2=18\).)