Sequential and Partial Deviance Tests

Two Questions About Adding Model Terms

A deviance difference can answer more than one question, depending on which models are compared. A sequential test asks how much fit improves when a term is added at a particular point in a specified order. A partial test asks how much fit improves when a specified term is added while the other terms in the model are already present. These comparisons need not give the same statistic: the earlier terms in a sequential test and the retained terms in a partial test define different comparisons.

Let \(M\) be a model in a set of nested models fitted to the same data with the same likelihood. Write \(\ell(M)\) for its maximized log-likelihood and \(D(M)\) for its deviance. Since the saturated-model term is the same for these fits, a deviance reduction from a smaller model \(M_0\) to a larger model \(M_1\) is

D(M_0)-D(M_1)=2\{\ell(M_1)-\ell(M_0)\}.

Thus, for nested models, a deviance reduction is the likelihood-ratio statistic for that particular comparison. The central practical task is to identify the two models before interpreting the statistic.

Definitions: Sequential and Partial Comparisons

Definition: Consider an ordered sequence of nested models \(M_0\subset M_1\subset\cdots\subset M_k\), where each step adds one term or one specified block of terms. The sequential deviance reduction at step \(j\) is \(D(M_{j-1})-D(M_j)\). It tests the term or block added at that step, conditional on the terms already included in \(M_{j-1}\).

Definition: To test a specified term \(T\) partially in a larger model \(M_{\mathrm{full}}\), compare \(M_{\mathrm{full}}\) with the reduced model \(M_{-T}\) obtained by removing \(T\) and retaining the other terms. The partial deviance reduction for \(T\) is \(D(M_{-T})-D(M_{\mathrm{full}})\). It tests the contribution of \(T\) conditional on the terms retained in both models.

A term may represent several coefficients, as with a categorical predictor having several levels. The degrees of freedom for a comparison are the difference in the dimensions of the two model parameter spaces (under the regularity conditions below). Sequential and partial comparisons must use nested models. They must also use a common likelihood for the same data; otherwise their deviance difference does not have the stated likelihood-ratio interpretation.

What Sequential Differences Do and Do Not Tell You

The meaning of a sequential statistic depends on its position in the model-building order. If \(M_0\) contains an intercept, \(M_1\) adds age, and \(M_2\) then adds treatment, the second reduction compares \(M_1\) with \(M_2\). It asks about treatment after age is included. Reversing the order asks about age after treatment is included. Those are different comparisons whenever the improvement associated with one term depends on which other terms are present.

In contrast, a partial test for treatment in the model containing both age and treatment always compares the full model with the model containing age but not treatment. It does not depend on an ordering used to build the model. Its answer can still depend on which other terms are retained: changing those terms changes the reduced model and therefore changes the question.

Theorem: For any fixed nested sequence \(M_0\subset\cdots\subset M_k\), all sequential deviance reductions are nonnegative, and their sum is the total deviance reduction from \(M_0\) to \(M_k\). Reordering the terms can change the individual reductions, but not their sum if the starting and ending models stay the same.

Proof. A larger model contains every parameter value available in a nested smaller model. Maximizing the same log-likelihood over the larger parameter space therefore cannot give a lower maximum: \(\ell(M_j)\geq\ell(M_{j-1})\). Hence \(D(M_{j-1})-D(M_j)=2\{\ell(M_j)-\ell(M_{j-1})\}\geq0\). For the sum, the intermediate terms cancel:

\sum_{j=1}^k \{D(M_{j-1})-D(M_j)\} =D(M_0)-D(M_k).

The right-hand side depends only on the starting and ending models, not on the path between them. A different order can redistribute this fixed total among the steps. \(\square\)

Worked Example: Reordering Two Poisson Terms

Suppose four independent counts in a \(2\times2\) table are Poisson, with observed counts

\begin{array}{c|cc} & B=1 & B=2\\ \hline A=1 & 8 & 12\\ A=2 & 18 & 22 \end{array}

Fit loglinear models with an intercept, a row term \(A\), a column term \(B\), and then both terms (without an interaction). For the calculations, omit the common Poisson terms \(-\log(y!)\) from the log-likelihood. Every fitted model has fitted counts summing to 60, so its maximized log-likelihood is \(\sum y\log(\hat\mu)-60\).

The intercept-only model has fitted count \(60/4=15\) in every cell, giving \(\ell_0=60\log(15)-60=102.483\). The \(A\)-only model fits 10 in each cell of the first row and 20 in each cell of the second, giving \(\ell_A=20\log(10)+40\log(20)-60=105.881\). The \(B\)-only model fits 13 in each cell of the first column and 17 in each cell of the second, giving \(\ell_B=26\log(13)+34\log(17)-60=103.018\).

With both terms, the fitted counts are row total times column total divided by 60: \(26/3,\ 34/3,\ 52/3,\ 68/3\), in table order. Thus

\ell_{AB} =8\log(26/3)+12\log(34/3)+18\log(52/3)+22\log(68/3)-60 =106.425.

If \(A\) enters first, its sequential reduction is \(2(105.881-102.483)=6.796\); the later reduction for \(B\) is \(2(106.425-105.881)=1.088\). If \(B\) enters first, its reduction is \(2(103.018-102.483)=1.070\); the later reduction for \(A\) is \(2(106.425-103.018)=6.814\). Both paths sum to approximately \(7.884\), but their stepwise results differ. The partial test for \(A\) given \(B\) is \(2(\ell_{AB}-\ell_B)=6.814\); the partial test for \(B\) given \(A\) is \(2(\ell_{AB}-\ell_A)=1.088\). These match the corresponding last steps, because those steps compare exactly the same pairs of models.

Degrees of Freedom and Reference Distributions

For either kind of comparison, the null model is the smaller of the two nested models. Let \(q\) be the difference in parameter dimensions. Under the null hypothesis and the usual regularity conditions for Wilks' theorem, the likelihood-ratio statistic \(D(M_0)-D(M_1)\) converges in distribution to a chi-square random variable with \(q\) degrees of freedom. The conditions include an identifiable, smooth, finite-dimensional model; a true parameter in the interior of the parameter space; nonsingular Fisher information; and a fixed number of parameters as the sample size increases. Boundary parameters and other nonregular cases require separate calibration.

The degrees of freedom belong to the comparison, not to the label “sequential” or “partial.” For example, adding a two-coefficient factor gives a two-dimensional difference when both coefficients are freely added and the design is of full rank. A one-degree-of-freedom term and a multi-coefficient block should not be assigned the same reference distribution merely because each is described as one “term.”

Worked Example: A Partial Test for a Multi-Level Predictor

In a binomial logistic regression for a binary clinical outcome, suppose a three-level treatment factor is tested while age and site remain in the model. The reduced model includes age and site but omits treatment; the full model includes all three terms. Suppose the fitted log-likelihoods are \(-180.40\) for the reduced model and \(-176.95\) for the full model. Treatment contributes two coefficients, so \(q=2\).

The partial likelihood-ratio statistic is

G^2=2\{-176.95-(-180.40)\}=2(3.45)=6.90.

Using the asymptotic \(\chi^2_2\) reference distribution, the upper-tail probability is \(P(\chi^2_2\geq6.90)=\exp(-6.90/2)\approx0.0318\). At the 5% level this comparison gives evidence against the null that both treatment coefficients are zero, conditional on age and site. It does not test treatment before age or site enters the model; those would be sequential comparisons with different reduced models.

Why the Distinction Matters

Sequential reductions are useful when the order itself represents a planned set of questions—for example, whether a block of baseline covariates improves fit before a treatment term is added. But the order must be reported, because changing it can change the statistic attributed to each term. A partial test is appropriate when the question is explicitly whether one specified term contributes after adjustment for the other terms in the full model.

The term “after adjustment” has a precise model meaning here: the reduced and full models contain the same retained terms, and differ only by the tested term or block. If an interaction is present, deciding whether to retain its associated main effects is a substantive model-specification choice. The test statistic compares the models actually specified; it does not make that choice on its own.

Worked Example: Reading a Sequential Analysis

A logistic regression is fitted in the order \(M_0\): intercept; \(M_1\): intercept plus age; \(M_2\): intercept plus age and treatment. Suppose the maximized log-likelihoods are \(-250.0,\ -245.8,\) and \(-243.9\), respectively. Each added term has one coefficient.

The sequential reduction for age is \(2\{-245.8-(-250.0)\}=8.4\), with one degree of freedom. The sequential reduction for treatment after age is \(2\{-243.9-(-245.8)\}=3.8\), also with one degree of freedom. The total reduction is \(8.4+3.8=12.2=2\{-243.9-(-250.0)\}\).

The value 3.8 answers whether adding treatment improves fit after age is already included. To test age partially after treatment, one would instead fit the model containing treatment but not age, then compare that fit with \(M_2\). The reported three likelihoods alone do not supply that comparison: the missing reduced-model fit is required.

Common Mistakes

Calling a sequential statistic “the test for the term” without stating the order. It tests the term as it enters after the terms already included, and may change when the order changes.
Calling a sequential reduction a partial test. A partial test compares the full model with the model that removes just the specified term while retaining the other terms.
Assuming order changes the total reduction. For the same starting and ending models the sequential reductions telescope to the same total; it is their allocation across terms that can change.
Using a chi-square degrees of freedom equal to one for every term. Use the difference in parameter dimensions; a block can contribute more than one degree of freedom.
Comparing fits that do not share a likelihood. The deviance-difference likelihood-ratio interpretation requires the same data and a common likelihood for nested models.

Key takeaway: A sequential test is defined by a term’s position in an ordered sequence; a partial test is defined by the full and reduced models for a specified term. State those models explicitly before interpreting a deviance difference.

Check Your Understanding

For each question, identify which models are being compared before interpreting the deviance statistic.

In an ordered sequence, what does the reduction at a given step test, and what determines its conditioning terms?
Describe the reduced model required for a partial test of one term in a model containing several terms.
Why can reordering terms change individual sequential reductions without changing their sum?
A factor contributes three coefficients. What is the usual chi-square degrees of freedom for its likelihood-ratio comparison, assuming the model is regular and the design has full rank?
A sequential analysis reports a test for age before treatment is added. What additional fit is needed to test age partially while retaining treatment?