A Worked Workshop in Regression Specification Checks

Introduction

A residual pattern is a clue, not a diagnosis by itself. A curve might indicate that a straight-line mean model is inadequate; different patterns across groups might suggest an interaction; and one extreme point might reflect an unusual observation—or a value entered in the wrong units. A careful specification check combines graphical evidence, comparisons of clearly defined models, and checks against how the data were recorded.

This workshop follows those steps using a hypothetical dataset. The outcome is a response score, \(Y\), and the predictors are \(x\), a measured exposure in hours, and \(g\), a group indicator coded 0 or 1. The goal is to decide whether the mean response needs a quadratic term, whether the response pattern differs by group, and whether an apparent anomaly is caused by a data-entry error.

Models and Assumptions for the Workshop

Let \(u=x-5.5\), so that \(u=0\) corresponds to an exposure of 5.5 hours. Centering makes the intercept and linear coefficient easier to describe; it does not change the fitted values when the same polynomial terms are included. Start with a straight-line model that allows the groups to have different mean levels:

M_0:\quad Y_i=\beta_0+\beta_1u_i+\beta_2g_i+\varepsilon_i.

The quadratic extension adds curvature, and the interaction extension allows the linear association with exposure to differ between groups:

M_q:\quad Y_i=\beta_0+\beta_1u_i+\beta_2g_i+\beta_3u_i^2+\varepsilon_i, \qquad M_f:\quad Y_i=\beta_0+\beta_1u_i+\beta_2g_i+\beta_3u_i^2+\beta_4u_ig_i+\varepsilon_i.

Here \(M_0\) is nested in \(M_q\), and \(M_q\) is nested in \(M_f\): setting the added coefficient or coefficients to zero recovers the smaller model. For the partial F comparisons below, assume independent errors with mean zero and a common variance; normal errors give the exact F reference distribution. The mean model being tested must be appropriately specified, the same observations and outcome must be used in both fits, and the model matrices must have full column rank. Residual plots can reveal concerns about these conditions, but do not prove that they hold.

Definition: A residual is the observed outcome minus its fitted value, \(e_i=y_i-\hat{y}_i\). A residual plot displays these differences against a variable or fitted value to look for structure left unexplained by the model. A nested-model comparison asks whether adding specified terms reduces residual variation enough to be compelling relative to the remaining variation.

First Read the Residual Patterns

Suppose the corrected, hypothetical dataset contains 30 observations, with \(x\) values from 1 to 10 hours and observations in both groups throughout that range. In the residual-versus-fitted plot from \(M_0\), residuals tend to be negative at low and high fitted values and positive in the middle. A smooth guide through the points would arch above zero. This is evidence that the straight-line mean pattern may be missing curvature.

Next, plot residuals against \(u\) separately for \(g=0\) and \(g=1\), or use different symbols for the two groups. Suppose the residual patterns suggest that the group difference changes with exposure: the group 1 residuals tend to be below those for group 0 at low exposure but above them at high exposure. That is a reason to examine an interaction. It is not proof that an interaction is needed. The plot can also reflect noise, an unusual point, or another omitted feature.

These plots answer different questions. A residual-versus-fitted plot is useful for an overall leftover mean pattern, while residuals against exposure within groups can make group-specific structure easier to see. In neither plot should a handful of points be treated as a substitute for checking the data or specifying the comparison to be made.

Worked Example: Turning a Plot Into a Checkable Question

For the hypothetical \(M_0\) fit, suppose the residual plot has the arch-shaped pattern described above. What is a focused next step?

The plot raises the question of whether adding \(u^2\) improves the mean model. Fit \(M_q\), which adds exactly one term and retains all terms in \(M_0\). Then compare \(M_0\) and \(M_q\) using their residual sums of squares. The plot motivates this targeted comparison; it does not, by itself, establish that the quadratic coefficient is nonzero.

For a possible interaction, inspect residuals against exposure within each group and compare \(M_q\) with \(M_f\). This comparison adds \(ug\) while retaining the quadratic term. Looking at the two questions separately makes clear which change in specification is associated with each reduction in residual variation.

Compare the Nested Fits

For a reduced model and a full model fitted to the same \(n\) observations, let \(RSS_R\) and \(RSS_F\) be their residual sums of squares. If the full model adds \(q\) coefficients and has \(p_F\) fitted coefficients, the partial F statistic is

F=\frac{(RSS_R-RSS_F)/q}{RSS_F/(n-p_F)}.

The numerator measures the reduction in residual sum of squares per added coefficient. The denominator estimates the error variance using the full model’s residuals. Because the full model can reproduce the reduced fit, least-squares fitting guarantees \(RSS_R\geq RSS_F\); equality is possible if the added terms do not improve the fit. Under the null hypothesis that all \(q\) added coefficients are zero, and the stated error assumptions, \(F\) has an \(F\) distribution with \(q\) and \(n-p_F\) degrees of freedom.

This comparison concerns the specified added terms, conditional on the terms already in the reduced model. It is not a test of whether every aspect of the full model is correct.

Worked Example: Testing the Quadratic Term

For the 30 corrected observations, suppose the fitted \(M_0\) has \(RSS_0=260\), and \(M_q\) has \(RSS_q=170\). The full quadratic model has four coefficients, so its residual degrees of freedom are \(30-4=26\). The comparison adds \(q=1\) coefficient, \(\beta_3\).

Hypotheses: \(H_0:\beta_3=0\) versus \(H_a:\beta_3\ne0\), conditional on the other terms in the model.

Calculation:

F=\frac{(260-170)/1}{170/(30-4)} =\frac{90}{170/26} =\frac{90}{6.538} =13.76.

The reference distribution is \(F_{1,26}\), giving a two-sided coefficient test with \(p\) approximately 0.001. At a 5% significance level, this is evidence against \(\beta_3=0\) under the model assumptions. In this hypothetical setting, the plot and the comparison both support retaining a quadratic term. The small p-value does not establish that the quadratic form is the only reasonable mean structure.

Worked Example: Testing the Group-by-Exposure Interaction

Suppose \(M_f\), which adds \(ug\) to \(M_q\), has \(RSS_f=132\). It has five coefficients and therefore \(30-5=25\) residual degrees of freedom. The reduced model \(M_q\) has \(RSS_q=170\); the comparison adds one coefficient, \(\beta_4\).

Hypotheses: \(H_0:\beta_4=0\) versus \(H_a:\beta_4\ne0\), conditional on the quadratic term and the other retained terms.

Calculation:

F=\frac{(170-132)/1}{132/(30-5)} =\frac{38}{132/25} =\frac{38}{5.280} =7.197.

With an \(F_{1,25}\) reference distribution, \(p\) is approximately 0.013. This is evidence that the interaction term improves the fit, conditional on \(M_q\), at the 5% level. It supports allowing the linear exposure association to differ between groups in this specification. The result is not evidence that group membership causes a change in response.

Check an Apparent Anomaly Against the Records

Before interpreting an extreme residual or a change in fitted coefficients, compare the data with their source. In the hypothetical records, one row has an exposure of 40 hours, while the original measurement form lists 4 hours. Other observations range from 1 to 10 hours, and the recorded value is inconsistent with the source. This is evidence of a coding error, specifically a misplaced decimal, rather than a reason to remove the observation or invent a special model term.

A large predictor value can have substantial influence on a fitted line or curve because the model uses its distance from the other predictor values. A residual pattern near the boundary may therefore change markedly if that value is wrong. But extremeness alone does not establish an error: a valid unusual observation must not be silently changed or discarded. Here the external record justifies correcting \(x=40\) to \(x=4\), documenting the change, and refitting all candidate models to the corrected data.

Worked Example: Sensitivity Fits After Verifying a Coding Error

Suppose the coefficient estimates from the corrected full model \(M_f\) are \(\hat{\beta}_0=50.2\), \(\hat{\beta}_1=1.8\), \(\hat{\beta}_2=3.1\), \(\hat{\beta}_3=1.4\), and \(\hat{\beta}_4=2.2\). These are hypothetical fitted estimates, in response-score units and the stated exposure units. The corrected-data nested comparisons above use \(RSS_0=260\), \(RSS_q=170\), and \(RSS_f=132\).

A sensitivity check can fit the models with the erroneous entry and compare them with the fits after the documented correction. Suppose the erroneous-entry full fit has a visibly different exposure curve and a residual sum of squares of 1,180. This sharp contrast flags the importance of the row, but it is the source record—not the RSS reduction—that establishes which exposure value is correct. Replacing 40 with the verified value 4 and refitting is the justified analysis; deleting the row merely because it changes the result is not.

In the corrected \(M_f\), the group 0 mean response at exposure \(x\) is \(\hat{y}=50.2+1.8u+1.4u^2\), where \(u=x-5.5\). For group 1, it is \(\hat{y}=50.2+3.1+(1.8+2.2)u+1.4u^2\).

Thus, at the centering point \(x=5.5\), the fitted group 1 mean is 3.1 response-score units higher than the fitted group 0 mean. The fitted local linear slope for group 0 is \(1.8+2(1.4)u\) score units per hour; for group 1 it is \(1.8+2.2+2(1.4)u\). The interaction coefficient means the difference between these local slopes is 2.2 score units per hour at any given exposure in this quadratic specification. These are model-based associations, not causal effects.

What the Workshop Establishes—and What It Does Not

In this example, the quadratic comparison addresses the curved residual pattern, the interaction comparison addresses a possible difference in exposure patterns between groups, and the source check resolves an implausible predictor value. The revised model is not selected by a sequence of p-values alone: the residual plots motivate specific checks, the nested tests quantify improvements for those terms, and the record verifies the data correction.

The partial F tests rely on the model assumptions stated earlier. Residual structure, unequal spread, dependence, or influential observations can undermine the usual reference distribution or the interpretation of the fit. The plots should therefore be reconsidered after refitting. In particular, a successful reduction in RSS is guaranteed or expected when terms are added, but that alone does not show that the added terms are useful or that assumptions are satisfied.

Key takeaway: Diagnose before revising: use targeted residual plots to identify a question, compare nested models that differ in the terms of interest, verify suspicious values against their sources, and interpret the refitted coefficients in context. A data correction requires evidence about the data, not merely a more attractive regression fit.

Common Mistakes

Calling a residual pattern proof. A curve or group-specific pattern suggests a possible missing term; test a clearly stated comparison and examine the refitted residuals.
Claiming the full model must have a strictly smaller RSS. Nested least-squares fits guarantee \(RSS_R\geq RSS_F\), but the values can be equal when the added terms do not improve the fit.
Comparing fits on different observations. A nested partial F comparison requires the reduced and full models to use the same outcome and observations.
Deleting an influential point because it changes the result. Check the measurement record. Correct a verified error; retain and investigate a valid observation rather than excluding it for convenience.
Interpreting an interaction as one constant group difference. In this model, the group difference depends on exposure: it is \(\beta_2+\beta_4u\), not just \(\beta_2\).

Check Your Understanding

Use the workshop’s notation and reasoning to answer these questions.

What feature in a residual-versus-fitted plot would motivate checking a quadratic term, and why is the plot not conclusive?
For the quadratic comparison, identify the reduced model, full model, added coefficient, and residual degrees of freedom used in the F statistic.
Why is the interaction comparison conditional on the quadratic term in this workshop?
What does the interaction coefficient \(\beta_4\) represent in units, and how does the group difference in fitted mean vary with exposure?
Why does verifying the original measurement record provide stronger grounds for correcting \(x=40\) than observing that the corrected model fits better?