Why Check the Outcome Model?
An adjusted regression estimate depends on more than which covariates appear in the model. It also depends on how the model represents their relationships with the outcome. A straight-line term for age, for example, may be inadequate if the outcome changes sharply only at older ages. A single exposure coefficient may also be inadequate if the exposure association differs across covariate groups. Either problem can leave residual structure that signals the fitted model has not captured the conditional mean of the outcome.
In an observational analysis, this matters because outcome regression is often used to predict outcomes under each exposure level for the same covariate distribution. If those predictions are systematically wrong, the resulting adjusted estimate can be distorted. Residual checks help identify problems worth investigating; they cannot establish that a model is correct or that the causal assumptions hold.
Model and Assumptions
Let \(A\) be a binary exposure, \(Y\) an outcome, and \(L\) the measured covariates chosen for adjustment. Write \(Y(a)\) for the outcome an individual would have under exposure level \(a\). For the standardized outcome-regression estimate to represent an average causal contrast in the target population, the causal identification assumptions include consistency, conditional exchangeability \(Y(a)\perp A\mid L\), and positivity: each exposure level must be possible at the covariate profiles represented in the target population.
The outcome model represents the conditional mean of \(Y\) given exposure and covariates. For a fitted function \(\hat m(a,L)\), a standardized estimate of the average difference is
This calculation predicts each individual’s outcome first under \(A=1\), then under \(A=0\), and averages the paired prediction differences over the analysis population. It relies on the outcome model providing adequate conditional-mean predictions across the exposure and covariate values used. A regression model can be correctly computed yet still misspecify that mean.
For individual \(i\), the fitted residual is \(e_i=Y_i-\hat m(A_i,L_i)\). If the conditional mean is represented well, residuals should not show a systematic pattern with fitted values, exposure, or covariates after accounting for the model terms. Residual plots and grouped residual summaries are diagnostic aids, not proof: random variation can create patterns, and a misspecified model can escape detection in a small sample.
These checks concern the functional form and included covariates in a specified adjustment analysis. They do not decide which variables are confounders, and they do not detect unmeasured confounding. Nor should every available predictor be added mechanically: the variables used for causal adjustment should be justified by the question and causal structure.
Targeted Checks and Model Comparisons
A useful diagnostic is targeted: plot or summarize residuals against a particular covariate, separately by exposure where appropriate, and ask what omitted feature could produce the pattern. A smooth curve through residuals against a continuous covariate can reveal curvature. Comparing residuals across exposure groups within covariate strata can reveal a missing interaction. Large residual differences across levels of a measured covariate absent from the model can flag a possible omission.
Record its adjustment variables, functional forms, and the standardized estimate it produces.
Check residuals against covariates and fitted values, and compare them across exposure groups or relevant strata.
Add a justified nonlinear term, interaction, or measured adjustment covariate suggested by the analysis question and diagnostics.
Check whether the alternative reduces systematic residual patterns and whether it materially changes standardized predictions or the target contrast.
The comparison is most informative when it changes one feature at a time and keeps the analysis population and target estimand fixed. A lower residual sum of squares alone does not show that an alternative model is preferable; adding terms typically gives a model more flexibility. Consider whether the terms make substantive sense, whether residual patterns improve, and whether the target estimate is stable. A change in the estimate is a warning that modeling choices matter, not by itself proof of which estimate is correct.
Worked Example: A Curved Covariate Relationship
Suppose a hypothetical analysis estimates the difference in mean blood-pressure outcome under a binary treatment, adjusting for age. The first model includes treatment and a straight-line age term. Residuals are summarized across four age bands.
| Age band (years) | Mean residual (mmHg) |
|---|---|
| 20–34 | +6 |
| 35–49 | −6 |
| 50–64 | −6 |
| 65–79 | +6 |
The positive residuals at the youngest and oldest ages mean observed outcomes average above the model predictions in those bands; negative residuals in the middle bands mean the predictions average too high there. This U-shaped pattern is consistent with curvature that a straight line cannot represent. It is a diagnostic clue, not a formal proof of a quadratic relationship.
The analyst fits a plausible alternative that includes age squared and allows the treatment contrast to vary with age. In this hypothetical illustration, the original model’s standardized estimate is 3.2 mmHg. Under the alternative, the predicted treatment contrasts for four equally represented age bands are 2, 4, 6, and 8 mmHg. The standardized estimate is
The estimates differ by \(5.0-3.2=1.8\) mmHg. That difference indicates that the adjusted result is sensitive to how age and treatment-related variation are represented. The analyst should check whether the alternative form is reasonable and whether its residual patterns improve; the calculation alone does not establish that 5.0 mmHg is the true causal effect. For this causal interpretation, the identification assumptions stated above must also be credible.
Worked Example: Checking for an Exposure-by-Covariate Interaction
In a hypothetical study, the covariate is age group and the outcome is a score, in points. The observed group means and sample sizes are:
| Age group | Unexposed: n, mean | Exposed: n, mean | Within-group difference |
|---|---|---|---|
| Younger | 90, 10 | 10, 14 | 4 points |
| Older | 20, 20 | 80, 22 | 2 points |
The exposure contrast is 4 points for younger individuals and 2 points for older individuals. A model with one common exposure effect cannot represent both contrasts. This suggests checking a model that includes an exposure-by-age-group interaction.
If the target population has equal numbers in the two age groups, standardizing the interaction model gives
The common-effect model’s exposure coefficient, fitted to these cell counts and means with age group included, is a weighted average of the two within-group differences. The weights are proportional to \(n_0n_1/(n_0+n_1)\) within each age group, where \(n_0\) and \(n_1\) are the unexposed and exposed sample sizes. Here the younger-group weight is \(90(10)/100=9\), and the older-group weight is \(20(80)/100=16\). Thus the common coefficient is
That coefficient differs from the 3-point standardized contrast for an equally divided target population. The difference arises because the common-effect fit weights the observed within-group contrasts according to the group sample sizes, while the stated target gives the age groups equal weight. The interaction model permits distinct contrasts and makes the standardization weights explicit. In a real analysis, the relevant weights must match the target population, and the model’s predictions should be checked for adequate support.
Worked Example: A Measured Covariate Left Out of the Model
Consider a hypothetical binary outcome recorded as a percentage, with a binary exposure and a measured high-risk indicator \(Z\). The table gives outcome means and exposure counts within each \(Z\) group.
| Risk group | Unexposed: n, mean | Exposed: n, mean |
|---|---|---|
| Lower risk (\(Z=0\)) | 20, 10% | 80, 12% |
| Higher risk (\(Z=1\)) | 80, 30% | 20, 32% |
Suppose the analyst fits a model with exposure only. The exposure-group means are \(0.8(12)+0.2(32)=16\%\) among exposed individuals and \(0.2(10)+0.8(30)=26\%\) among unexposed individuals. The resulting crude difference is
Yet within each risk group, the exposed mean is 2 percentage points higher: \(12\%-10\%=2\) points for \(Z=0\), and \(32\%-30\%=2\) points for \(Z=1\). If the target population is half lower-risk and half higher-risk, the model including \(Z\) predicts standardized means of
The standardized difference is \(22\%-20\%=2\) percentage points. In this hypothetical table, the opposite directions of the crude and within-group comparisons arise because the exposure groups have very different distributions of the measured risk indicator. Residual summaries from the exposure-only model would also differ strongly by \(Z\), flagging that the model leaves systematic outcome variation unexplained. This example illustrates why a measured covariate’s omission can matter; whether it belongs in a causal adjustment model depends on the causal question and assumptions, not on residual patterns alone.
What the Checks Can and Cannot Establish
A residual pattern helps locate a possible modeling problem; an alternative specification helps assess its consequences. If adding a nonlinear term removes a curved residual pattern, or an interaction removes systematic differences between exposure groups across strata, that is evidence that the original conditional-mean model was too restrictive for these data. If the standardized estimate also changes materially, report internally that the adjusted estimate depends on the outcome-model specification and investigate the plausible forms before drawing a conclusion.
Some distinctions are important:
- A covariate can predict the outcome without being a confounder. Conversely, omission of a measured confounder can distort an adjusted comparison.
- Residuals assess fit to observed outcomes under observed exposures. They do not verify exchangeability, positivity, consistency, or the absence of unmeasured confounding.
- A small change in the estimate across two models does not prove either model is correct; both may be misspecified.
- Added flexibility can improve fit while making predictions unstable, especially in covariate regions with few observations. Check that the alternative model is supported by the data.
A short justification explains why residuals are relevant. If the conditional mean is \(m(A,L)=E(Y\mid A,L)\), then an adequately specified fitted mean aims to make \(E\{Y-\hat m(A,L)\mid A,L\}\) close to zero. A systematic residual average within a covariate or exposure subgroup indicates that the predictions are consistently too high or too low in that subgroup, which is inconsistent with this goal. In finite samples, however, observed residual averages vary by chance, so patterns should be considered alongside model plausibility and estimate comparisons.
Common Mistake
Wrong reasoning: “The residual plot looks acceptable, so the outcome model is correct and the adjusted estimate is causal.” A residual plot can fail to reveal a real problem, and no residual plot establishes the causal identification assumptions.
Correction: Treat residual checks as targeted diagnostics. Compare plausible specifications for the same target estimand, examine whether they address the flagged pattern, and assess how their standardized predictions and estimates differ. State causal conclusions only to the extent justified by both the model and the observational assumptions.
Check Your Understanding
For each question, distinguish evidence about model fit from evidence about causal identification.
- What residual pattern against a continuous covariate could suggest that a straight-line term is inadequate?
- Why might residual summaries by exposure group within covariate strata reveal a missing interaction?
- In the interaction example, why is the standardized contrast for equal age-group weights not the same as the common-effect regression coefficient?
- What does a strong residual pattern across a measured covariate omitted from a model suggest, and what does it not establish?
- If two plausible outcome specifications produce noticeably different standardized estimates, what should the analyst conclude about the original estimate?