Observed Outcomes, Fitted Values, and Residuals in Output

Three Quantities for Each Record

A regression output can put an observed outcome beside a fitted value and a residual. These quantities refer to the same record, but they are not interchangeable. The observed outcome is what was recorded; the fitted value is what the estimated regression equation predicts for that record’s predictor values; and the residual is the difference between them. Keeping those roles straight makes it possible to read and check an output table rather than treating its numbers as a collection of unrelated columns.

For example, a model might predict a patient’s waiting time from recorded characteristics. The fitted value is the model’s predicted waiting time for that patient. If the recorded wait was longer, the residual is positive; if it was shorter, the residual is negative. Both the observed outcome and the fitted value are expressed in minutes, so their difference—the residual—is also in minutes.

The hats and subscripts matter. The observed value \(y_i\) belongs to one record, and \(\hat{y}_i\) is the model’s fitted outcome for that same record. A residual is not the observed outcome itself, nor is it the difference between two different records. In a table, check that the values being compared come from the same row.

The Model Behind the Fitted Value

In a multiple regression, a model for an outcome \(Y_i\) can be written as an additive equation involving predictors \(x_{i1},\ldots,x_{ip}\). Its estimated equation uses estimated coefficients to calculate fitted values:

Here, \(\hat{\beta}_0,\ldots,\hat{\beta}_p\) are the coefficient estimates used by the model. Substituting record \(i\)'s predictor values into this equation produces \(\hat{y}_i\). Comparing that result with the recorded outcome \(y_i\) produces \(e_i\). The fitted value therefore depends on both the estimated equation and the particular record’s predictors; the residual also depends on that record’s observed outcome.

A statistical model often represents the outcome as a fitted mean structure plus an unobserved error, for example \(Y_i = \beta_0 + \beta_1 x_{i1} + \cdots + \beta_p x_{ip} + \varepsilon_i\). The error \(\varepsilon_i\) is part of the model for the data-generating process and is not directly observed. The residual \(e_i\), by contrast, is calculated from the observed outcome and the fitted value. It can be used as an observable stand-in for examining discrepancies, but it is not identical to the unobserved error.

The usual regression assumptions—such as an appropriate linear mean structure and errors with mean zero conditional on the predictors—matter when interpreting or using the model statistically. Independence and constant error variance are also common assumptions for inference. The arithmetic definition of a residual does not require those assumptions: once an observed outcome and a fitted value are specified, their difference is determined. The examples below focus on reading and checking that arithmetic, not on making an inferential claim about a population.

Reading a Multiple-Regression Output Row

The same definitions apply when several predictors appear in the equation. First use the estimated coefficients and the record’s predictor values to identify its fitted value. Then subtract that fitted value from the observed outcome. A convenient check is to add the residual back to the fitted value: the result should reproduce the observed outcome, subject to rounding.

Verifying Output When Values Are Rounded

Software may calculate a fitted value and residual using more decimal places than it displays. As a result, subtracting the printed fitted value from the printed observed outcome may not reproduce the printed residual exactly. A small discrepancy can be a consequence of rounding rather than a calculation error. To check carefully, use the full-precision values when available; otherwise, compare values at the precision shown and allow for rounding.

A Short Justification of the Residual Identity

The defining equation \(e_i = y_i - \hat{y}_i\) immediately gives a useful reconstruction check. Add \(\hat{y}_i\) to both sides:

The fitted-value terms cancel, leaving the observed outcome. This proves \(y_i = \hat{y}_i + e_i\). It also verifies the sign interpretation: if \(e_i>0\), then \(y_i>\hat{y}_i\); if \(e_i<0\), then \(y_i<\hat{y}_i\); and if \(e_i=0\), the observed and fitted outcomes are equal. Zero residual means an exact match for that record, not that the model is exact for every record.

Common Mistakes and a Reliable Check

A frequent mistake is to calculate fitted minus observed and label the result a residual. That reverses the sign. If an observed outcome is 12 and the fitted value is 15, the residual is \(12-15=-3\), not \(15-12=3\). The correction is to write the definition first—observed minus fitted—and then substitute the values in that order.

Another mistake is to interpret a residual as a model coefficient or as an unobserved error. A coefficient is part of the estimated equation; a residual is a record-specific difference in outcome units. The model error is not observed, while the residual can be calculated from the output and data. Also avoid saying that a positive residual means the model “overpredicted”: if the observation exceeds the fit, the model’s fitted value is below the observed outcome for that record.

When checking a table, use a consistent three-part routine: confirm that the outcome and fitted value belong to the same row, calculate observed minus fitted, and check that fitted plus residual reconstructs the observed outcome. Then confirm that all three quantities are expressed in the outcome’s units. This simple procedure catches many sign, row-matching, and interpretation errors without requiring any additional model calculations.