From Residual Structure to a Model Extension
A generalized linear model can fit well against each predictor separately and still miss a systematic pattern involving predictors jointly. For example, a predictor might have a strong association with a response in one subgroup and a weak or reversed association in another. A model with only additive predictor effects cannot represent that changing association. The resulting residual pattern can point toward an omitted interaction.
Residuals can also vary systematically across levels of a predictor that is absent from the model. That pattern may suggest an omitted main effect. These are diagnostic clues, not proofs: unusual observations, an incorrect mean functional form, dependence, or a misspecified response distribution can also create structure. The practical approach is to use residuals to formulate a specific candidate extension and then assess that extension with a valid comparison.
For example, a Poisson log-link model with two binary predictors and no interaction has
where \(\mu_{ab}=E[Y\mid A=a,B=b]\). The model implies that the multiplicative effect of changing \(A\) from 0 to 1 is the same at both levels of \(B\): the rate ratio is \(\exp(\beta_A)\) for either value of \(B\). Adding \(\beta_{AB}ab\) permits that rate ratio to differ when \(B=1\). In a logistic model the same product term allows the odds ratio for \(A\) to vary across levels of \(B\).
What to Look For Across Predictor Combinations
A plot of residuals against one predictor can conceal a pattern that appears only after conditioning on another predictor. Instead of looking only at residuals by \(A\), inspect their organization across the combinations of \(A\) and \(B\), or across clinically or operationally meaningful subgroups. For binary responses, binned residual plots can help reveal calibration patterns; for discrete counts, compare residual behavior with the fitted means and the count scale. The earlier material on residuals against predictors and binned residual plots provides the basic graphical tools.
For a Poisson observation \(y_i\) with fitted mean \(\hat{\mu}_i\), the deviance residual is
with the convention \(y_i\log(y_i/\hat{\mu}_i)=0\) when \(y_i=0\). A positive residual indicates that the observed count exceeds its fitted mean; a negative residual indicates the reverse. A sequence of positive residuals in some combinations and negative residuals in others can be more informative than any single residual’s size. The exact interpretation depends on the fitted response distribution and the residual definition.
- Possible omitted main effect: residuals tend to be positive in one level of a candidate predictor and negative in another, across other predictors’ levels.
- Possible omitted interaction: the residual contrast across one predictor changes across levels of another, such as a positive-versus-negative pattern that reverses between strata.
- Important qualification: a visual pattern suggests a model term to investigate; it does not establish that the term is needed or that the proposed model is otherwise adequate.
When there are many predictors or many possible subgroupings, searching until a pattern appears can produce chance findings. Define the candidate comparison from the substantive question and diagnostic pattern, and treat follow-up significance levels cautiously if the same data were used to select the extension.
Nested Deviance Comparisons
Suppose a reduced model is nested within a full model: every mean structure allowed by the reduced model can be obtained from the full model by setting some added coefficients to zero. Let \(D_R\) and \(D_F\) be their residual deviances, and let \(q\) be the number of additional free parameters in the full model. For independent observations from a correctly specified likelihood model, under regularity conditions and when the null hypothesis is true, the likelihood-ratio statistic is approximately chi-square with \(q\) degrees of freedom.
The null hypothesis is that all coefficients added in the full model are zero. A large deviance reduction relative to the number of added parameters is evidence against that null. The comparison tests the added terms jointly; it does not establish that every individual coefficient is important.
Write the reduced model and the proposed extension. Confirm that the reduced model is obtained by constraints on the full model’s coefficients.
Set \(q\) equal to the number of independent coefficient restrictions being tested, not the number of observations or subgroups.
Subtract the full-model residual deviance from the reduced-model residual deviance.
Compare with a chi-square distribution with \(q\) degrees of freedom, while checking that likelihood-based inference is appropriate for the response and sampling structure.
The reason the deviance difference is the likelihood-ratio statistic is direct. For a specified response distribution, the deviance of a fitted model is twice the difference between the log likelihood of the saturated model and that of the fitted model. Write these log likelihoods as \(\ell_S\), \(\ell_R\), and \(\ell_F\). Then
so subtraction cancels the common saturated-model term:
The right-hand side is twice the log likelihood ratio in favor of the full model. Wilks’ theorem gives its approximate chi-square reference distribution under the null when the models are regular, the observations are independent, and the likelihood is correctly specified. The approximation can be poor with sparse data, boundary parameters, or very small samples. If extra-Poisson variation invalidates the assumed Poisson likelihood, the ordinary likelihood-ratio calibration should not be used automatically; an overdispersion diagnosis requires separate attention.
Worked Example: A Poisson Interaction Across Two Factors
A hypothetical service center records event counts in four equally exposed combinations of two binary operating conditions, \(A\) and \(B\). The observed counts are 14 for \((0,0)\), 6 for \((0,1)\), 6 for \((1,0)\), and 14 for \((1,1)\). Assume independent Poisson counts with equal exposure. A main-effects log-link model fits a mean of 10 to each cell; the model with the \(AB\) interaction fits the four observed cell means. Test whether adding the interaction improves fit at the 5% level.
Method. Compare the nested main-effects and interaction models by their deviance difference. The main-effects fitted means are 10 because each row and column total is 20, and the grand total is 40; under the no-interaction model, each fitted cell mean is \(20(20)/40=10\).
The reduced-model deviance is
Using \(\log(1.4)=0.3364722\) and \(\log(0.6)=-0.5108256\), the two contributions inside the brackets are \(28(0.3364722)=9.4212226\) and \(12(-0.5108256)=-6.1299075\). Their sum is \(3.2913151\), so \(D_R=6.5826303\). The interaction model is saturated for these four cell means, so \(D_F=0\). It adds one parameter, giving \(q=1\).
Thus \(G^2=6.5826303-0=6.5826303\). Against \(\chi^2_1\), the approximate p-value is 0.0103. Since \(0.0103<0.05\), the data provide evidence that the interaction improves fit, subject to the model assumptions and the limitations of a small four-cell illustration.
Check and common mistake. The fitted means under the reduced model sum to 40, matching the observed total, and the full model reproduces all four counts. A common mistake is to compare the full-model deviance with zero and stop there; the evidence for the interaction is the reduction from the reduced-model deviance, assessed with one degree of freedom.
Worked Example: Residual Evidence for an Omitted Predictor
A hypothetical monitoring study has four independent Poisson counts, each observed over one equal unit of exposure. The counts in four pre-specified site types are 3, 7, 13, and 17. The reduced model has an intercept only; the candidate full model assigns a separate mean to each site type. Calculate and assess the deviance improvement at the 5% level.
Method. The intercept-only Poisson estimate is the overall mean: \((3+7+13+17)/4=40/4=10\). The reduced model therefore fits 10 in each site type. The full model’s fitted means are 3, 7, 13, and 17, so it is saturated and has deviance zero. The full model adds three parameters, hence \(q=3\).
The reduced deviance is
The four terms inside the brackets are, respectively, \(3(-1.2039728)=-3.6119184\), \(7(-0.3566749)=-2.4967246\), \(13(0.2623643)=3.4107354\), and \(17(0.5306283)=9.0206803\). Their sum is \(6.3227727\), giving \(D_R=12.6455454\). The deviance difference is therefore 12.6455454 on 3 degrees of freedom, with an approximate p-value of 0.0055. This is below 0.05, so the site-type model improves fit in this illustration.
Check and common mistake. The site-type counts sum to 40, and the four fitted means in the reduced model also sum to 40. Do not mistake the four groups for four added parameters: the intercept-only model already has one parameter, so the full model adds three. In applied work, a saturated comparison with very few cells should be interpreted cautiously and supported by adequate replication and model checks.
Worked Example: Testing a Logistic Interaction From Model Output
A hypothetical insurer models whether an application results in a reported event using a logistic GLM. The model includes an age-band indicator, a monitoring-device indicator, and one adjustment variable. The main-effects model has residual deviance 92.4 on 96 residual degrees of freedom. Adding the age-band-by-device interaction gives residual deviance 84.8 on 95 residual degrees of freedom. Test the added interaction at the 5% level. The fitted interaction coefficient is 0.90.
Method. The models are nested and differ by one coefficient, so \(q=1\). The likelihood-ratio statistic is the deviance reduction:
The degrees of freedom also decrease from 96 to 95, confirming that one parameter was added. Under the null hypothesis of no interaction, \(G^2\) is approximately \(\chi^2_1\); its p-value is approximately 0.0058. Since this is below 0.05, there is evidence that the interaction improves the model’s fit.
For interpretation, the interaction coefficient corresponds to a ratio of odds ratios of \(\exp(0.90)\approx2.46\). Thus, under this fitted model, the age-band odds ratio for the event is about 2.46 times as large for one device status as for the reference device status. This is an interaction interpretation, not a statement that the odds themselves are multiplied by 2.46 for every applicant.
Check and common mistake. The deviance reduction is positive, as expected when a nested model is fitted by maximum likelihood. A common error is to use 96 degrees of freedom for the test; 96 is the reduced model’s residual degrees of freedom, whereas the comparison uses the one-degree difference between the models.
Interpreting the Evidence Responsibly
A statistically significant deviance reduction says that the added terms improve fit relative to the specified reduced model, under the likelihood assumptions. It does not guarantee good predictions, establish causality, or prove that the full model captures the true process. Consider whether the added term is plausible, whether estimated effects are stable, and whether the extension changes conclusions that matter for the application.
A non-significant result is not proof that the effect is exactly zero. The data may have limited power, especially for interactions, which often require more observations than main effects. Inspect effect estimates and uncertainty as well as the omnibus test, and avoid treating a lack of evidence as evidence of no effect.
- Compare genuinely nested models fitted to the same observations and response definition.
- Use the number of added parameters for the test degrees of freedom.
- Do not use the ordinary likelihood-ratio chi-square test without considering dependence, overdispersion, sparse data, and the response distribution.
- Use residual subgroup patterns to guide a focused comparison rather than testing every possible interaction indiscriminately.
Check Your Understanding
For each problem, identify what the residual pattern or model comparison supports. Use the stated assumptions when interpreting a likelihood-ratio result.
- In a Poisson model with binary predictors \(A\) and \(B\), residuals are mostly positive for \((A,B)=(0,0)\) and \((1,1)\), and mostly negative for \((0,1)\) and \((1,0)\). Which model feature is a natural candidate to investigate? (Answer: An \(A\)-by-\(B\) interaction, because the residual pattern changes across combinations rather than showing only a uniform main-effect shift.)
- A reduced GLM has residual deviance 31.7. A nested model adds two coefficients and has deviance 27.9. At the 5% level, is the improvement significant, using the 5% critical value \(\chi^2_{2,0.95}=5.991\)? (Answer: No; the reduction is \(31.7-27.9=3.8<5.991\), with 2 degrees of freedom.)
- A nested-model comparison gives deviances 18.4 and 10.1, with two added parameters. Find the likelihood-ratio statistic and approximate p-value. (Answer: \(G^2=8.3\) on 2 degrees of freedom; \(p=\exp(-8.3/2)\approx0.0158\).)
- A one-parameter interaction reduces deviance by 6.2. Using the approximate \(\chi^2_1\) reference distribution, is there evidence of improvement at the 5% level? (Answer: Yes; \(6.2>3.841\), and \(p\approx0.0128\).)
- Residuals are positive in both levels of \(A\) for one level of a candidate binary predictor \(C\), and negative in both levels of \(A\) for the other level of \(C\). What kind of extension is most directly suggested? (Answer: A main effect of \(C\), because the residual shift is consistent across levels of \(A\), rather than an \(A\)-by-\(C\) interaction pattern.)
The diagnostic reasoning and nested-model comparisons in this tutorial are aligned with the SOA Exam SRM (Statistics for Risk Modeling) syllabus.