Absolute Fit Versus Nested-Model Comparison

Two Questions That Need Different Answers

A model can improve substantially on a simpler alternative and still describe the data poorly. These are not contradictory conclusions: they answer different questions. A nested-model comparison asks whether adding specified terms improves fit relative to a particular smaller model. An absolute-fit assessment asks whether the fitted model itself describes the observed data adequately.

For example, adding a group indicator may explain a large difference between two sets of observations. But if the fitted model still misses a pronounced pattern within those groups, its improvement over an intercept-only model does not make it adequate. A comparison is always relative to its chosen baseline; absolute fit is not.

Definition: Models $M_0$ and $M_1$ are nested if every distribution allowed by $M_0$ is also allowed by $M_1$. The smaller model $M_0$ is the null model for their comparison, and $M_1$ is the larger model. A nested-model test evaluates the restrictions that make $M_0$ a special case of $M_1$.

For a GLM with independent observations, let $\ell(\beta)$ denote the log-likelihood and let $\hat\beta_M$ maximize it subject to model $M$. When a saturated model is defined for the data, the deviance of $M$ is

D(M)=2\{\ell_{\mathrm{sat}}-\ell(\hat\beta_M)\},

where $\ell_{\mathrm{sat}}$ is the maximized log-likelihood of that saturated model. Smaller deviance means a model is closer in likelihood to the saturated fit. The deviance difference for two nested models has a separate interpretation: it measures the gain in maximized log-likelihood from $M_0$ to $M_1$.

What the Nested-Model Difference Measures

Theorem: If $M_0$ is nested in $M_1$, and both models use the same observations and likelihood, then $$ D(M_0)-D(M_1)=2\{\ell(\hat\beta_{M_1})-\ell(\hat\beta_{M_0})\}. $$ The right-hand side is the likelihood-ratio statistic for comparing the restrictions in $M_0$ against $M_1$.

The identity follows directly from the definition. Both deviances use the same saturated-model log-likelihood, so subtracting cancels it:

\begin{aligned} D(M_0)-D(M_1) &=2\{\ell_{\mathrm{sat}}-\ell(\hat\beta_{M_0})\} -2\{\ell_{\mathrm{sat}}-\ell(\hat\beta_{M_1})\}\\ &=2\{\ell(\hat\beta_{M_1})-\ell(\hat\beta_{M_0})\}. \end{aligned}

Because $M_1$ includes every parameter value allowed by $M_0$, its maximized log-likelihood cannot be smaller. Thus the difference is nonnegative. It is a measure of the improvement over this particular simpler model, not a direct measure of whether $M_1$ fits well in an absolute sense. $\square$

Under the usual regularity conditions for the likelihood-ratio test, the statistic is approximately chi-squared under $M_0$, with degrees of freedom equal to the difference in the models’ numbers of free parameters. These conditions include a correctly specified null model, identifiable parameters, a true null parameter in the interior of the parameter space, nonsingular information, and a suitable large-sample setting. This is Wilks’ theorem. The reference distribution tests the restrictions in $M_0$; it does not test whether $M_1$ is adequate.

Key takeaway: A small p-value for a nested-model comparison supports the conclusion that the added terms improve fit relative to the specified null model. It does not, on its own, support the conclusion that the larger model fits the data adequately.

Worked Examples

Worked Example: A Significant Logistic-Model Improvement

Suppose a logistic regression model $M_0$ includes an intercept and a treatment indicator. A larger model $M_1$ adds two prespecified age terms. Both models use the same observations. The fitted log-likelihoods are $\ell(\hat\beta_{M_0})=-94.6$ and $\ell(\hat\beta_{M_1})=-86.1$. What does the nested comparison say?

The added terms contribute two parameters, so the degrees of freedom are $2$. The likelihood-ratio statistic is

2\{(-86.1)-(-94.6)\}=2(8.5)=17.0.

Against a chi-squared distribution with $2$ degrees of freedom, the approximate upper-tail probability is $\exp(-17/2)\approx0.00020$. The comparison gives strong evidence that the age terms improve fit over $M_0$, under the test’s assumptions. It says nothing by itself about whether $M_1$ adequately describes the outcome data. That requires an absolute-fit assessment as well.

Worked Example: A Poisson Nested Comparison

Four independent Poisson counts are observed at four covariate settings: $0,1,2,3$. Model $M_0$ gives all four observations one common mean. Model $M_1$ gives the first two observations one mean and the last two another. Find the deviance reduction and interpret it.

Under $M_0$, the maximum-likelihood estimate is the overall mean, $(0+1+2+3)/4=1.5$. Under $M_1$, the two estimates are the group means $(0+1)/2=0.5$ and $(2+3)/2=2.5$. For a Poisson observation $y$ with fitted mean $\hat\mu$, its deviance contribution is

d(y,\hat\mu)=2\left\{y\log\left(\frac{y}{\hat\mu}\right)-(y-\hat\mu)\right\},

with $0\log(0/\hat\mu)=0$. The contributions under $M_1$, in observation order, are

\begin{aligned} d(0,0.5)&=1,\\ d(1,0.5)&=2\log 2-1\approx0.3863,\\ d(2,2.5)&=4\log(0.8)+1\approx0.1074,\\ d(3,2.5)&=6\log(1.2)-1\approx0.09393. \end{aligned}

Thus $D(M_1)\approx1.5876$. To get the nested-model difference directly, the two observed group totals are $1$ and $5$, while $M_0$ expects $3$ in each group. Therefore

\begin{aligned} D(M_0)-D(M_1) &=2\left\{1\log\left(\frac{1}{3}\right) +5\log\left(\frac{5}{3}\right)\right\}\\ &\approx2.9110. \end{aligned}

There is one added parameter in $M_1$, so the comparison has $1$ degree of freedom. Its approximate p-value is about $0.088$. The calculation quantifies the improvement over a common-mean model. It does not establish that $M_1$ fits adequately; the deviance remaining under $M_1$ is a different quantity.

Worked Example: Significant Improvement Alongside Poor Absolute Fit

Now consider four independent Poisson counts $0,100,200,300$, with the same $M_0$ and $M_1$ structures as in the previous example. The fitted means are $150$ for every observation under $M_0$, and $50,50,250,250$ under $M_1$.

The contributions to $D(M_1)$ are

\begin{aligned} d(0,50)&=100,\\ d(100,50)&=200\log 2-100\approx38.6294,\\ d(200,250)&=400\log(0.8)+100\approx10.7426,\\ d(300,250)&=600\log(1.2)-100\approx9.3929. \end{aligned}

Adding these gives $D(M_1)\approx158.7649$. The nested-model deviance reduction can be calculated from the group totals. The observed totals are $100$ and $500$; under $M_0$, each group’s expected total is $300$. Hence

\begin{aligned} D(M_0)-D(M_1) &=2\left\{100\log\left(\frac{100}{300}\right) +500\log\left(\frac{500}{300}\right)\right\}\\ &\approx291.103. \end{aligned}

The comparison has $1$ degree of freedom, and this deviance reduction is highly significant. Yet $M_1$ still assigns mean $50$ to both of the first two observations, despite seeing counts $0$ and $100$. It also assigns mean $250$ to both of the last two, despite counts $200$ and $300$. The remaining deviance of about $158.76$, with $4-2=2$ residual degrees of freedom in this simple count-table setting, is very large. Under the usual chi-squared approximation for absolute-fit assessment, the upper-tail probability is approximately $\exp(-158.7649/2)$, or about $3.3\times10^{-35}$. The approximation is an illustration of the distinction, not a substitute for checking whether its conditions are appropriate in an application.

So the two conclusions coexist: the group effect improves substantially on the common-mean model, and the two-mean model still leaves a striking pattern unexplained. The added group term helps, but it does not make the resulting model adequate.

Why the Distinction Matters

In applied GLM analysis, a nested-model test is useful when the scientific question is about a specified set of terms: for example, whether adding an interaction improves on a model without it. Absolute-fit checks address a broader question: whether the selected model’s assumptions and structure are plausible for the observed data. Evidence for one answer does not supply the other.

This distinction also clarifies why the choice of null model matters. A larger model can be strongly preferred to a very restrictive baseline while still missing structure that neither model represents. Conversely, a comparison may fail to reject the smaller model even when both models fit poorly. “No detectable improvement” is not equivalent to “adequate fit.”

Common mistake: “The likelihood-ratio test is significant, so the larger GLM fits the data well.” This reasoning confuses a relative comparison with an absolute assessment. The test asks whether the larger model improves on the stated nested null; it does not certify the larger model. Check absolute-fit diagnostics or an appropriate goodness-of-fit assessment separately, and interpret them in light of their assumptions.

State the comparison.
Name the smaller model, the larger model, and the terms or restrictions being tested.

Calculate the relative improvement.
Use $D(M_0)-D(M_1)$, equivalently twice the difference in maximized log-likelihoods, and use the parameter-count difference for the reference degrees of freedom when regularity conditions apply.

Assess the fitted model separately.
Inspect whether its remaining discrepancies and assumptions are compatible with an adequate description of the data.

Check Your Understanding

For each question, keep the relative comparison distinct from absolute fit.

In a nested comparison, what scientific question does $D(M_0)-D(M_1)$ answer?
Why does the saturated-model log-likelihood cancel from the deviance difference?
A larger model has a much smaller deviance than a nested smaller model. What conclusion is justified, and what conclusion is not?
If two additional parameters are added, what degrees of freedom are used for the usual likelihood-ratio reference distribution under its regularity conditions?
Give an example of how a model can improve on a baseline yet still leave important observed structure unexplained.

Absolute Fit Versus Nested-Model Comparison

What You'll Learn