Why Fitted Values and Residuals Matter
A regression model summarizes how a response is expected to change with predictors. Its fitted values describe what the model predicts for the observations in the data; its residuals show how far those observations fall from their fitted values. Comparing the two is a basic way to ask whether the model’s assumptions are plausible. A good fit statistic alone cannot answer that question: a model can explain much of the observed variation and still miss a systematic pattern.
The key distinction is between what can be calculated from the sample and what belongs to the underlying model. Fitted values and residuals are calculated after estimating the regression coefficients. Model errors describe deviations from the true conditional mean, which is generally unknown. Residual diagnostics therefore provide evidence about model adequacy; they do not reveal the errors directly or prove that assumptions hold.
The Model Errors Are Not the Residuals
For multiple linear regression, write the model for observation \(i\) as
where the conditional mean is \(\mu_i = E[Y_i\mid x_{i1},\ldots,x_{ip}]\), the \(\beta_j\) are the unknown coefficients, and \(\varepsilon_i=Y_i-\mu_i\) is the model error. In the usual regression framework, the errors have conditional mean zero and constant variance, and are independent across observations. Normality of the errors is an additional assumption when exact normal-theory inference is required; it is not needed to define a fitted value or residual.
The errors cannot usually be observed because the true conditional mean is unknown. The fitted value instead uses estimated coefficients, \(\hat{\beta}_j\), and the residual uses the observed response:
The distinction can be made explicit. For a fixed set of predictors, \(y_i=\mu_i+\varepsilon_i\), so
Thus a residual equals the model error minus the estimation difference between the fitted value and the true conditional mean. Unless that difference is zero, the two are not equal. Since the coefficients are estimated from the same data being diagnosed, residuals also have restrictions and dependence induced by fitting. It is therefore incorrect to assume that residuals are independent observations drawn directly from the error distribution.
What Residual Evidence Can Tell You
The regression assumptions concern the conditional distribution of the response, given the predictors. Residuals help assess whether the chosen mean structure and error behavior are compatible with the data. Their interpretation depends on the question being considered.
- Mean structure: If the model specifies a straight-line mean relationship, residuals that show a systematic pattern across predictor or fitted-value levels may indicate that the mean relationship is misspecified. Residuals without a clear pattern are more consistent with the proposed mean structure, though they do not establish it.
- Variance: If the model assumes constant conditional variance, residuals whose spread changes across fitted values or predictor levels raise concern about that assumption. A few observations alone are not enough to establish a variance pattern.
- Independence: If observations have a meaningful order, the residual sequence can be examined for structure. Clustering or runs may suggest dependence that a model treating observations as independent has not captured.
- Normality: When normal errors are assumed for exact inference, the residual distribution can provide a limited diagnostic check. Residuals are estimated quantities, so their appearance is not a direct observation of the error distribution.
These are diagnostic clues, not automatic verdicts. A pattern might reflect an omitted predictor, an unsuitable functional form, dependence, measurement issues, or chance variation. Conversely, a small sample can conceal a real departure from an assumption. Diagnostics should be interpreted in context and alongside the model’s purpose.
Worked Example: Calculating Residuals for a Linear Fit
A hypothetical analyst fits a simple linear regression of claim-processing hours \(y\) on weekly case volume \(x\), using four branch-week observations: \((0,2)\), \((1,4)\), \((2,5)\), and \((3,7)\). Calculate the least-squares line and residuals, then state what these residuals can and cannot establish.
Method. Calculate the ordinary least-squares slope from \(b_1=S_{xy}/S_{xx}\), then use \(b_0=\bar y-b_1\bar x\). Here the data are an illustration; no claim is made that they represent actual branches.
The means are \(\bar x=(0+1+2+3)/4=1.5\) and \(\bar y=(2+4+5+7)/4=4.5\). The centered sums are
and
Therefore \(b_1=8/5=1.6\), and \(b_0=4.5-(1.6)(1.5)=2.1\). The fitted line is \(\hat y=2.1+1.6x\). For each observation, subtract its fitted value from its observed response:
| \(x_i\) | \(y_i\) | \(\hat y_i=2.1+1.6x_i\) | \(e_i=y_i-\hat y_i\) |
|---|---|---|---|
| 0 | 2 | 2.1 | -0.1 |
| 1 | 4 | 3.7 | 0.3 |
| 2 | 5 | 5.3 | -0.3 |
| 3 | 7 | 6.9 | 0.1 |
The residuals are small in magnitude for these observations, but that fact alone does not establish that the model assumptions hold. With only four observations, substantial departures could be hard to detect. Common mistake avoided: interpreting each residual as the corresponding true error. Each residual is the difference from the estimated line, not from the unknown true conditional mean.
Why Residual Patterns Are Informative
Residual analysis works because each residual compares an observed response with the model’s fitted conditional mean at that observation’s predictor values. If the mean model is adequate, the residuals should not systematically indicate that the model repeatedly overpredicts or underpredicts in a particular region. If the variance assumption is adequate, the residual spread should not systematically change with the fitted mean or predictors. This gives a practical diagnostic principle: look for structure in the deviations that the model leaves behind.
For ordinary least squares with an intercept, the fitted coefficients are chosen to minimize the sum of squared residuals, \(\sum_i e_i^2\). That optimization explains why residuals are the natural quantities to inspect: they are precisely the observed discrepancies the fit has attempted to make small. Minimizing their total squared size does not ensure that they are patternless. A model might achieve a small total while leaving a coherent curve, a changing spread, or dependence across observations.
Worked Example: Recognizing a Misspecified Mean Pattern
A hypothetical actuary examines a linear model for a response at predictor values \(x=-2,-1,0,1,2\). The observed responses are \(y=4,1,0,1,4\). Calculate the fitted line and residuals. What is the diagnostic concern?
Method. Fit the simple linear least-squares line. The predictor mean is \(\bar x=0\), and the response mean is \(\bar y=(4+1+0+1+4)/5=2\). By symmetry, the centered cross-product sum is
Also \(S_{xx}=(-2)^2+(-1)^2+0^2+1^2+2^2=10\), so the slope is \(b_1=0/10=0\), and the intercept is \(b_0=\bar y-b_1\bar x=2\). The fitted value is 2 for every observation. Subtracting gives residuals \(2,-1,-2,-1,2\), in order of increasing \(x\).
The residuals are positive at both ends and negative in the middle. That organized change is consistent with curvature in the mean relationship, whereas the fitted model is a horizontal straight line. The diagnostic concern is that the linear mean structure may be inadequate. The residual pattern does not, by itself, identify the correct replacement model or prove that curvature is the only explanation. Common mistake avoided: seeing an overall response average of 2 and concluding that a constant fitted mean is adequate without examining how deviations vary across \(x\).
Worked Example: Assessing Evidence of Changing Variance
A hypothetical model relates a portfolio-level response to a score \(x\). There are two observations at each score \(x=0,1,2,3\): the responses are \(1.9,2.1\); \(2.7,3.3\); \(3,5\); and \(3,7\), respectively. Fit the straight line, calculate the residuals, and assess whether the constant-variance assumption is beyond question.
Method. At each score, the two deviations from \(2+x\) sum to zero. Thus the response deviations from that line also sum to zero within every score group. This gives \(\bar x=(0+0+1+1+2+2+3+3)/8=1.5\) and \(\bar y=3.5\). The centered sum \(S_{xx}=2(1.5^2)+2(0.5^2)+2(0.5^2)+2(1.5^2)=10\). Since the deviations within each pair cancel, \(S_{xy}=10\), so \(b_1=1\) and \(b_0=3.5-(1)(1.5)=2\). The fitted line is \(\hat y=2+x\).
The fitted values are 2 for both observations at \(x=0\), 3 at \(x=1\), 4 at \(x=2\), and 5 at \(x=3\). The residual pairs are, respectively, \((-0.1,0.1)\), \((-0.3,0.3)\), \((-1,1)\), and \((-2,2)\). The mean squared residual within each pair is \(0.01\), \(0.09\), \(1\), and \(4\), respectively.
The residual magnitudes increase across the score levels, which raises a concern that the conditional variance may not be constant. However, there are only two observations per level, so this is limited evidence, not a conclusive finding. Common mistake avoided: concluding that a model has constant variance merely because its fitted mean is a straight line. The mean pattern and the spread around that mean are separate features to assess.
Derivation: Why the Residual Equals Error Minus Fitting Difference
This identity makes the distinction between errors and residuals precise. For a fixed predictor setting, let \(\mu_i=E[Y_i\mid x_{i1},\ldots,x_{ip}]\). By definition, the model error is \(\varepsilon_i=Y_i-\mu_i\), so \(Y_i=\mu_i+\varepsilon_i\). The residual is the observed response minus the fitted value. Substituting the expression for \(Y_i\) gives
The final term, \(\hat y_i-\mu_i\), is the fitting difference at that predictor setting. It would be zero only if the estimated fitted value equalled the true conditional mean there. Since the conditional mean is generally unknown and the coefficients are estimated, it is not justified to replace \(e_i\) with \(\varepsilon_i\). This derivation also explains why diagnostics are indirect: residuals contain information about the errors, but also reflect estimation of the regression mean.
Common Diagnostic Traps
- Calling a residual an error. The residual is observable after fitting; the model error is defined relative to an unknown conditional mean.
- Treating a good fit summary as a diagnostic certificate. Measures of overall fit do not rule out systematic residual patterns or assumption violations.
- Reading a pattern as proof. A small sample can produce apparent structure by chance, and a real problem may be hard to see in limited data.
- Confusing mean behavior with variance behavior. A model may have an adequate conditional mean while the response spread changes, or the reverse.
- Demanding residuals behave exactly like independent errors. Residuals are calculated using estimated coefficients, so they are not direct, independent draws from the error distribution.
For an actuarial analysis, the practical consequence is important: diagnostic evidence can affect whether coefficient estimates, intervals, and predictions are trustworthy under the specified model. The appropriate response is to investigate a concern in context, not to treat a visual or numerical diagnostic as a substitute for understanding the data and assumptions.
Check Your Understanding
For each original scenario, distinguish what is calculated from the sample from what is assumed about the underlying model.
- A simple linear regression uses \((x,y)=(1,3),(2,5),(3,4)\). Calculate the least-squares line and all three residuals. (Answer: \(\hat y=3+0.5x\); residuals \(-0.5,1,-0.5\), in order.)
- At one predictor setting, an observation is \(y_i=12\), its fitted value is \(\hat y_i=10\), and the true conditional mean is not known. State the residual and explain whether the model error is known. (Answer: \(e_i=2\); the model error is not known because the true conditional mean is unknown.)
- A fitted model has residuals that are mostly positive at low fitted values and mostly negative at high fitted values. Name a possible concern and state why this observation is not proof of a model failure. (Answer: The conditional mean structure may be misspecified; a residual pattern is evidence, not proof, and may reflect chance or other features of the data.)
- For two observations at each of three predictor levels, the residual pairs are \((-0.2,0.2)\), \((-0.8,0.8)\), and \((-1.5,1.5)\). Calculate the mean squared residual within each pair and identify the possible concern. (Answer: \(0.04, 0.64, 2.25\); the increasing spread raises concern about nonconstant variance.)
- A colleague says, “The errors are independent and normal, so the residuals must be independent and normal observations.” Identify the conceptual error. (Answer: Residuals are calculated using estimated coefficients and are not the unobserved errors; they do not generally behave as independent draws from the error distribution.)