From a Coefficient Table to a Model Comparison
A multiple regression analysis is more than a list of coefficients. A useful analysis connects the question to the predictors, interprets estimates in context, checks whether the model is an adequate description of the data, and compares competing models when one contains all the terms of another. This workshop follows those steps for a hypothetical campus health dataset.
The outcome is resting systolic blood pressure, measured in millimeters of mercury (mm Hg). The candidate predictors are age, weekly physical activity, and residence type. Age and activity are quantitative; residence type is categorical. The aim is descriptive and predictive: the analysis concerns associations in this dataset, not evidence that changing a predictor would cause a change in blood pressure.
For inference using the usual regression standard errors and tests, the model also requires appropriate conditions. The conditional mean of the response should be linear in the included predictors; errors should be independent, have mean zero, and have a common variance; and errors should be approximately normally distributed for exact small-sample \(t\) and \(F\) inference. Predictors must not be perfectly linearly dependent. Study design is central to independence, while residual plots can help assess linearity and constant variance and a normal probability plot can help assess residual normality. These checks do not prove assumptions true.
Worked Example: Fit and Interpret a Mixed-Predictor Model
Worked Example: Fit and Interpret a Mixed-Predictor Model
Suppose a fictional campus health team records resting systolic blood pressure for 30 volunteers. For each person it also records age in years, physical activity in hours per week, and residence type. The residence categories are off campus and on campus. A few rows from the hypothetical dataset are shown below; the regression output described afterward uses all 30 records.
| Volunteer | Blood pressure (mm Hg) | Age (years) | Activity (hours/week) | Residence |
|---|---|---|---|---|
| A | 119 | 20 | 4 | Off campus |
| B | 124 | 22 | 2 | On campus |
| C | 116 | 19 | 7 | Off campus |
| D | 127 | 24 | 3 | On campus |
| E | 121 | 21 | 5 | Off campus |
Let \(D_i=1\) for an on-campus resident and \(D_i=0\) for an off-campus resident. A hypothetical software fit to all 30 volunteers gives
The age estimate is \(0.42\) mm Hg per year. Among people with the same recorded activity and residence category, the fitted mean blood pressure is 0.42 mm Hg higher for each additional year of age. The activity estimate is \(-0.65\) mm Hg per additional weekly activity hour, comparing people with the same age and residence category. These are adjusted associations, not causal effects.
The residence estimate compares on-campus with off-campus volunteers at the same age and activity level. The fitted mean is 3.80 mm Hg higher for on-campus residents. The intercept, 108.0 mm Hg, is the fitted mean for an off-campus person with age zero and zero activity. Since age zero is far outside this dataset's relevant range, that intercept is needed to define the equation but is not a useful campus-health summary.
The indicator coding matters: reversing which category is coded 1 would change the sign and reference interpretation of the residence coefficient, but not the model's fitted values. The units also matter: age is in years, activity in hours per week, and the response in mm Hg. Interpretation should name the predictor increase and the variables held fixed.
Check the Model Before Comparing It
A model comparison is only useful if the fitted models are plausible descriptions of the outcome pattern. For this example, suppose the residual-versus-fitted plot has residuals scattered around zero without a clear curve or widening spread. Suppose a normal probability plot is approximately straight, with no pronounced tail departure, and the leverage and influence review identifies no single observation that dominates the fit. These are favorable but not conclusive checks for the corresponding regression assumptions.
Independence cannot be established by looking at those plots. It should be considered from how the observations were obtained. If several records came from the same household, or if one person's measurement affected another's, treating all 30 errors as independent could be inappropriate. In this hypothetical example, assume each volunteer contributes one observation and the sampling process gives no reason to expect dependent errors. The conclusion is conditional on that design assumption.
Worked Example: Diagnose a Pattern Before Proceeding
Imagine the residual plot for the 30-person fit shows residuals that are mostly positive at low fitted values, mostly negative in the middle, and positive again at high fitted values. What should the analyst do before relying on the model's usual coefficient tests?
Interpret the pattern. A curved sequence in residuals suggests that the mean response may not be adequately represented by a straight-line combination of age, activity, and residence. A residual plot that simply has points above and below zero at random would be more consistent with the linear mean condition.
Do not treat the test as a repair. A partial F test compares the specified models; it does not make a misspecified linear model appropriate. The analyst should investigate whether the pattern reflects a needed nonlinear term or another systematic feature, guided by the study question and data. Any revised model should then be checked again.
Separate visual evidence from design evidence. A normal probability plot can reveal serious departures from normal residuals, but it cannot establish independent sampling. Conversely, no obvious curve in a residual plot does not guarantee the conditional mean is exactly linear. Diagnostics inform judgment; they are not automatic certificates of validity.
Comparing Nested Models
A reduced model is nested in a full model when every term in the reduced model also appears in the full model. The full model can therefore reproduce the reduced model by setting its additional coefficient or coefficients to zero. The partial \(F\) test asks whether the extra terms, considered together, reduce the residual variation enough to be more than would be expected from adding parameters alone.
The numerator is the reduction in residual sum of squares per added parameter. The denominator is the full model's estimated error variance, \(MSE_F=SSE_F/df_F\). The ratio compares improvement per added parameter with the residual variation left after fitting the larger model.
Why the Partial F Formula Works
The justification follows from the nested structure. Fitting extra terms cannot increase the least-squares residual sum of squares, so \(SSE_R-SSE_F\) measures the improvement attributable to those terms. If the null hypothesis is true, the added terms explain only random variation, and the reduction divided by the number added, \(q\), estimates the same error variance as \(SSE_F/df_F\). Under the model assumptions, their ratio has an \(F\) distribution under the null.
The residual degrees of freedom decrease by \(q\), since each added parameter uses one additional degree of freedom. Thus \(df_R=df_F+q\). If the added terms truly improve the conditional mean, the reduction in residual sum of squares tends to be large relative to the full model's residual mean square, producing a larger \(F\) statistic. This test concerns the added terms jointly; it does not say which coefficient is responsible without additional analysis.
Worked Example: Is Residence Useful as an Added Predictor?
Worked Example: Is Residence Useful as an Added Predictor?
Return to the hypothetical 30-volunteer dataset. Compare a reduced model with age and activity to the full model that also includes the on-campus indicator. Suppose the reduced model has \(SSE_R=42.0\), and the full model has \(SSE_F=36.0\). The full model estimates four parameters: an intercept and three slopes, so \(df_F=30-4=26\). The reduced model estimates three parameters, so \(df_R=30-3=27\). Thus \(q=1\).
The null and alternative hypotheses are \(H_0:\beta_{\text{residence}}=0\) and \(H_a:\beta_{\text{residence}}\ne0\), conditional on age and activity. Calculate the statistic:
The reference distribution is \(F(1,26)\). Its two-sided-equivalent upper-tail probability for this one added coefficient is approximately \(p=0.047\). At significance level \(\alpha=0.05\), reject the joint null that the added residence coefficient is zero. In this hypothetical analysis, residence provides statistically significant additional explanatory information after accounting for age and activity.
This result is consistent with the fitted residence estimate of 3.80 mm Hg, but the partial \(F\) test itself tests whether that coefficient is zero; the coefficient's sign and units come from the fitted equation. Statistical significance does not establish practical importance, eliminate possible confounding, or show that residence causes blood pressure to differ.
Making the Comparison Part of a Complete Analysis
The conclusion should combine the model specification, coefficient interpretation, diagnostic evidence, and model-comparison result. Here, the analysis supports retaining residence at the 5% significance level under the stated model and assumptions. The estimated difference is expressed in mm Hg and adjusted for age and activity. The residual checks described above show no conspicuous warning pattern, while independence rests on the sampling design.
A small p-value is not a measure of the size or usefulness of an effect. Nor does the comparison say that the full model is the best possible model: it only compares the particular nested pair tested. Conversely, a nonsignificant result would not prove that the added predictor has no association; it would mean the data do not provide sufficiently strong evidence against the specified null at the chosen threshold.
Name the response, quantitative predictors, categorical reference group, and the terms being compared.
Write the fitted equation and interpret each coefficient with its units, reference category, and held-fixed predictors.
Use residual plots for linearity and variance patterns, a normal probability plot for residual shape, and study design to assess independence.
Calculate the partial \(F\) statistic from the two residual sums of squares and the correct degrees of freedom, then interpret its p-value in context.
Report what the added terms contribute, while acknowledging diagnostics, design limitations, and the distinction between association and causation.
Common Mistakes
- Comparing unrelated models with the partial \(F\) formula. The models must be nested, and the full model must include all the reduced model's terms.
- Using the wrong degrees of freedom. The numerator degrees of freedom \(q\) equal the number of added parameters, not the number of observations. The denominator uses the full model's residual degrees of freedom.
- Interpreting a categorical coefficient without its reference group. A coefficient for an indicator compares the category coded 1 with the category coded 0, with other predictors held fixed.
- Calling the added predictor causal because its p-value is small. Regression adjustment does not by itself remove confounding or establish causality.
- Treating diagnostics as proof. Plots can reveal problems but cannot verify assumptions; independence particularly depends on the design and data collection.
Check Your Understanding
Use the definitions and workflow above to answer these questions.
- In a model with an on-campus indicator coded 1 and off-campus coded 0, what does its coefficient compare?
- A full model adds two parameters to a reduced model. What is the numerator degrees of freedom for the partial \(F\) test?
- Why does a curved residual pattern matter before interpreting the usual regression tests?
- For the worked comparison, verify the full model's residual degrees of freedom from the sample size and number of estimated parameters.
- What does a significant partial \(F\) test establish, and what does it not establish about causation?