Likelihood-Ratio Deviance for Loglinear Models

Why Compare a Model With the Saturated Model?

A loglinear model summarizes the cell means of a contingency table using a restricted set of parameters. The saturated model, by contrast, allows a separate mean for every cell. Comparing the fitted model with this cell-by-cell benchmark measures how much likelihood is lost by imposing the model’s restrictions. The resulting statistic is the likelihood-ratio deviance.

For Poisson cell counts, the deviance has a particularly direct form: it is twice a sum of contributions comparing each observed count with its fitted mean. Under appropriate conditions, its reference distribution is approximately chi-square when the proposed model is correct. The number of degrees of freedom reflects how many independent restrictions the model imposes on the cell means.

Definition and Derivation of the Deviance

Let $Y_1,\ldots,Y_m$ be independent Poisson counts, with observed values $y_1,\ldots,y_m$. A loglinear model specifies their means $\mu_i$ through a parameter vector $\beta$, commonly as $\log(\mu_i)=x_i^\mathsf{T}\beta$, possibly with a known offset. Write $\hat{\mu}_i$ for the fitted mean under this model. The saturated model allows each cell mean to vary freely.

Definition: The likelihood-ratio deviance of the fitted model against the saturated model is $$ D=2\{\ell_{\mathrm{sat}}-\ell_{\mathrm{fit}}\}, $$ where $\ell_{\mathrm{sat}}$ is the maximized log-likelihood in the saturated model and $\ell_{\mathrm{fit}}$ is the maximized log-likelihood in the proposed model. For independent Poisson cells, $$ D=2\sum_{i=1}^m\left\{y_i\log\left(\frac{y_i}{\hat{\mu}_i}\right)-(y_i-\hat{\mu}_i)\right\}, $$ with the convention $0\log(0/\hat{\mu}_i)=0$.

Here and below, a saturated mean for a cell with zero observed count is understood as a likelihood supremum at mean zero. This convention includes boundary values in the saturated model and makes the formula valid even when some $y_i=0$.

To derive the expression, the log-likelihood for independent Poisson counts, omitting no terms that depend on the means, is

\ell(\mu)=\sum_{i=1}^m\{y_i\log(\mu_i)-\mu_i-\log(y_i!)\}.

For a cell with $y_i>0$, differentiating $y_i\log(\mu_i)-\mu_i$ with respect to $\mu_i$ gives $y_i/\mu_i-1$, so the maximizing saturated mean is $\mu_i=y_i$. If $y_i=0$, the cell’s likelihood is proportional to $\exp(-\mu_i)$, whose supremum over $\mu_i\geq 0$ occurs at $\mu_i=0$. Substituting these saturated means gives

\ell_{\mathrm{sat}} =\sum_{i=1}^m\{y_i\log(y_i)-y_i-\log(y_i!)\}, $$ where \(0\log 0\) is interpreted as zero. Subtracting the fitted log-likelihood cancels the factorial terms and yields $$ 2(\ell_{\mathrm{sat}}-\ell_{\mathrm{fit}}) =2\sum_{i=1}^m\{y_i\log(y_i/\hat{\mu}_i)-(y_i-\hat{\mu}_i)\}.

This derivation also shows why the deviance is nonnegative: the saturated model contains every possible vector of cell means, so its maximized likelihood cannot be smaller than the maximized likelihood under a restricted model.

Worked Example: An Intercept-Only Model

Suppose three independent Poisson counts are observed as $20,30,40$, and the model requires all three means to be equal: $\log(\mu_i)=\beta$. The fitted mean is the average count, $\hat{\mu}_i=(20+30+40)/3=30$ for each cell.

The deviance is $$ D=2\left\{20\log(20/30)+30\log(30/30)+40\log(40/30)\right\}, $$ because the terms $-\sum_i(y_i-\hat{\mu}_i)$ sum to zero: both observed and fitted totals equal 90.

Using $\log(2/3)\approx-0.405465$ and $\log(4/3)\approx0.287682$, $$ D=2\{20(-0.405465)+30(0)+40(0.287682)\} =2(-8.10930+11.50728) \approx 6.796. $$ The saturated model has three free means; the equal-means model has one parameter. Its goodness-of-fit reference degrees of freedom are therefore $3-1=2$, provided the chi-square approximation is appropriate.

Degrees of Freedom and the Chi-Square Reference

The deviance is often compared with a chi-square distribution to assess goodness of fit. For a model with $m$ cells and $p$ identifiable parameters, the usual reference has $m-p$ degrees of freedom. More generally, $p$ means the local dimension of the model’s mean vectors, not merely the number of parameter symbols written in a formula. In a loglinear model $\log(\mu)=X\beta$, with $X$ an $m$-by-$p$ design matrix of full column rank, this dimension is $p$.

Theorem: Suppose the cell counts are independent Poisson variables and the true mean vector is in a smooth loglinear model of local dimension $p$. Assume all true cell means are positive, the model is locally identifiable with nonsingular information, and the fitted model has an interior, consistent maximum-likelihood estimator with probability tending to one. Under an increasing-information regime, such as multiplying every cell’s exposure by $t$ and letting $t\to\infty$, the deviance against the saturated model satisfies $$ D\xrightarrow{d}\chi^2_{m-p}. $$ For the exposure formulation, assume fixed cell design, positive baseline means, and a fixed full-rank model matrix. The result is an asymptotic reference, not an exact finite-sample distribution.

The dimension difference has a geometric interpretation. The saturated model can vary in $m$ directions, one for each cell. The fitted model can vary in $p$ local directions. If the model is correct, the remaining $m-p$ directions describe departures from the model, and the likelihood-ratio statistic measures those departures.

A derivation makes the role of the assumptions clearer. Write the true means as $\nu_i=tq_i$, where each $q_i>0$, and let $Z_i=(Y_i-\nu_i)/\sqrt{\nu_i}$. For independent Poisson counts, the central limit theorem gives $Z=(Z_1,\ldots,Z_m)^\mathsf{T}\xrightarrow{d}N_m(0,I_m)$. Consider local changes in the means, $\mu_i=\nu_i+\sqrt{\nu_i}\,\delta_i$, for bounded vectors $\delta$. A Taylor expansion of the Poisson log-likelihood about $\nu$ gives

2\{\ell(\mu)-\ell(\nu)\} =2Z^\mathsf{T}\delta-\delta^\mathsf{T}\delta+o_p(1).

The remainder is negligible for bounded local changes because $\nu_i$ grows proportionally to $t$, and the third-order terms are of smaller order than the quadratic terms. The regularity conditions ensure that the maximizing changes are local and that this expansion applies to the likelihood maxima.

In the saturated model, $\delta$ can range over all of $\mathbb{R}^m$. In the loglinear model $\log(\mu)=\log(tq)+X(\beta-\beta_0)$, a local parameter change $\beta=\beta_0+a/\sqrt{t}$ gives, to first order,

\delta \approx A a,\qquad A=\operatorname{diag}(\sqrt{q_1},\ldots,\sqrt{q_m})X.

Since every $q_i$ is positive and $X$ has rank $p$, the column space of $A$ has dimension $p$. Maximizing the quadratic expression $2Z^\mathsf{T}\delta-\delta^\mathsf{T}\delta$ over a subspace projects $Z$ onto that subspace. Therefore, the difference between the saturated and fitted maxima converges to the squared length of the projection of a standard normal vector onto the orthogonal complement of the model’s tangent space. That complement has dimension $m-p$. The squared length of $m-p$ independent standard normal coordinates has a $\chi^2_{m-p}$ distribution, giving the stated limit.

Worked Example: Deviance Under Two-Way Independence

Consider the table of observed counts

	Column 1	Column 2	Row total
Row 1	12	8	20
Row 2	6	14	20
Column total	18	22	40

Under the independence model, the fitted counts are row total times column total divided by the grand total. Thus they are $20(18)/40=9$, $20(22)/40=11$, $20(18)/40=9$, and $20(22)/40=11$.

The deviance is $$ D=2\left\{12\log(12/9)+8\log(8/11)+6\log(6/9)+14\log(14/11)\right\}. $$ The four terms inside braces are approximately $3.45218$, $-2.54763$, $-2.43279$, and $3.37627$. Their sum is $1.84803$, so $D\approx3.696$.

There are four cells. The independence model for a two-way table has local dimension $1+(2-1)+(2-1)=3$: one overall level and one independent contrast for each variable. Hence the reference degrees of freedom are $4-3=1$. This degree-of-freedom calculation is about the number of independent mean parameters; it does not assert that the finite-sample deviance is exactly chi-square.

Zero Counts, Interpretation, and Pitfalls

A zero observed count is not a reason to omit a cell from the deviance. Its contribution is obtained by the stated limiting convention: $$ 2\{0\log(0/\hat{\mu}_i)-(0-\hat{\mu}_i)\}=2\hat{\mu}_i. $$ The fitted mean remains part of the comparison, even though the saturated mean for that cell is zero.

Worked Example: A Zero Cell in an Equal-Means Model

Suppose the observed counts are $0,3,7$ under the equal-means model. The fitted mean for each cell is $\hat{\mu}_i=10/3$. The zero-count cell contributes $2(10/3)=6.6667$.

The other contributions are $$ 2\{3\log(3/(10/3))-(3-10/3)\} =2\{3\log(0.9)+1/3\}\approx0.0345 $$ and $$ 2\{7\log(7/(10/3))-(7-10/3)\} =2\{7\log(2.1)-11/3\}\approx3.0538. $$ Thus $D\approx6.6667+0.0345+3.0538=9.7550$, with $3-1=2$ degrees of freedom by parameter counting. The calculation is valid, but the fitted means are small, so the chi-square approximation may be poor.

The deviance matters because it turns a model’s overall lack of fit into a likelihood comparison with a clear benchmark. But its reference distribution depends on asymptotics: the cells must provide enough information for the likelihood to be well approximated by a quadratic near its maximum. Very sparse tables, boundary estimates, nonidentifiable parameters, or an incorrect model can undermine the usual chi-square calibration. A degrees-of-freedom calculation alone cannot establish that the approximation is trustworthy.

Mistake: counting parameter labels rather than identifiable directions. Two coefficients may not represent two distinct model directions if the design matrix is rank deficient. Use the rank, or the model’s local dimension, to find $p$.
Mistake: dropping zero-count cells from the sum. A zero contributes $2\hat{\mu}_i$; it is not an undefined logarithm after applying the $0\log 0=0$ convention.
Mistake: treating the chi-square reference as exact. The result is an asymptotic distribution under regularity and an increasing-information regime. Small fitted counts can make the approximation questionable.

Key takeaway: For independent Poisson cells, the deviance is twice the log-likelihood gain of the saturated model over the fitted model. With $m$ cells and a regular, identifiable model of dimension $p$, its asymptotic goodness-of-fit reference is $\chi^2_{m-p}$, subject to adequate information and the model being correctly specified.

Check Your Understanding

Use the definitions and conditions above to answer these questions.

Why does maximizing the saturated Poisson likelihood give fitted means equal to the observed counts, including the boundary interpretation when a count is zero?
A model has 8 cells and a full-rank design matrix with 5 columns. What is the usual asymptotic deviance reference distribution, and what conditions does that answer rely on?
Why does a zero observed count contribute $2\hat{\mu}_i$ to the deviance rather than being omitted?
In a two-way table with $r$ rows and $c$ columns, the independence model has dimension $1+(r-1)+(c-1)$. What is the deviance reference degrees of freedom against the saturated model?
Give one reason a deviance with the nominal degrees of freedom might not have a reliable chi-square approximation in a particular table.

Likelihood-Ratio Deviance for Loglinear Models

What You'll Learn