Sparse Cells and Deviance-Test Caution

Why Sparse Counts Need Special Caution

A likelihood-ratio deviance test compares the fit of nested loglinear models. Its familiar chi-square reference distribution is an approximation, not an automatic property of a deviance statistic. The approximation is justified by large-sample theory under regularity conditions. When some expected counts under the tested model remain small, individual cell contributions can be far from their large-count behavior, so the chi-square calibration may be poor.

This is different from a numerical fitting problem. An algorithm can converge accurately to the maximum-likelihood fit even when that fit has sparse expected counts. Conversely, an algorithm can fail to converge even in a setting where a chi-square approximation would otherwise be reasonable. One issue concerns the calculation of the fitted model; the other concerns how well a reference distribution describes the statistic.

Definition: For observed count $y_i$ and fitted mean $\hat{\mu}_i$, the cell’s likelihood-ratio deviance contribution is $$ d_i=2\left[y_i\log\left(\frac{y_i}{\hat{\mu}_i}\right)-(y_i-\hat{\mu}_i)\right], $$ with $0\log(0/\hat{\mu}_i)=0$. The model deviance is the sum of these contributions relative to the saturated model. A sparse fitted count is an expected count $\hat{\mu}_i$ that is small under the model being assessed.

In a comparison of nested models, the likelihood-ratio statistic is the difference between their deviances, $D(M_0)-D(M_1)$, where $M_0$ is the more restricted model and $M_1$ the larger model. Under the null hypothesis, the usual chi-square approximation has degrees of freedom equal to the difference in model dimensions. The relevant expected counts for assessing sparsity are those under the model being tested, not merely the observed counts.

What the Chi-Square Approximation Assumes

A standard justification comes from Wilks’ theorem. In a fixed-dimensional regular model, if the null model is true, the parameter is in the interior of the parameter space, the model is identifiable with nonsingular information, and the sample size grows, a likelihood-ratio statistic for $q$ independent restrictions converges in distribution to $\chi^2_q$. For Poisson loglinear models, one useful large-sample formulation holds the number of cells and model dimensions fixed while an exposure multiplier $t$ increases: the cell means are $t\mu_i$, with each $\mu_i>0$, and the true finite-dimensional parameter lies in the interior of the model.

These conditions describe a regime in which counts grow, rather than one in which an increasing table has many cells with persistently tiny means. They also exclude structural zeros from the ordinary positive-mean model. If a cell is structurally impossible, it must be handled as outside the sampling space; treating it as an ordinary cell with an estimated mean near zero does not restore the usual regularity conditions.

A useful local calculation explains why large expected counts help. Write $y=\mu+\delta$, where $\mu>0$, and suppose $|\delta|/\mu\leq 1/2$. The deviance contribution is

d(y,\mu)=2\mu h(x),\qquad x=\frac{\delta}{\mu},\qquad h(x)=(1+x)\log(1+x)-x.

The Pearson contribution for the same cell is $\delta^2/\mu$. The next result gives a precise comparison.

Result: If $\mu>0$ and $|\delta|/\mu\leq 1/2$, then $$ \left|d(\mu+\delta,\mu)-\frac{\delta^2}{\mu}\right| \leq \frac{4|\delta|^3}{3\mu^2}. $$

Proof. The function $h$ satisfies $h(0)=h'(0)=0$, $h''(0)=1$, and $h'''(x)=-1/(1+x)^2$. On $[-1/2,1/2]$, $|h'''(x)|\leq 4$. Taylor’s theorem with remainder therefore gives

\left|h(x)-\frac{x^2}{2}\right| \leq \frac{4|x|^3}{6} =\frac{2|x|^3}{3}.

Multiplying by $2\mu$ and substituting $x=\delta/\mu$ proves the bound. Thus, when a count is close to its fitted mean in relative terms, the deviance contribution is close to the Pearson contribution. For counts that fluctuate on the order of $\sqrt{\mu}$, the relative fluctuation typically shrinks as $\mu$ grows. This local expansion helps explain the large-count approximation; it does not by itself prove that a whole model-comparison statistic follows a chi-square distribution. $\square$

Worked Examples

Worked Example: A Sparse Cell in a Two-Way Independence Test

Consider the observed table below. Its row totals are 18 and 2, and its column totals are also 18 and 2. Under independence, the fitted counts are the row total times the column total, divided by the grand total of 20.

	Column 1	Column 2	Row total
Row 1	17	1	18
Row 2	1	1	2
Column total	18	2	20

The fitted counts are $16.2, 1.8, 1.8,$ and $0.2$. For example, the fitted count in the lower-right cell is $2(2)/20=0.2$. The independence model has one degree of freedom relative to the saturated model. The deviance is

\begin{aligned} D &=2\left[ 17\log\left(\frac{17}{16.2}\right) +\log\left(\frac{1}{1.8}\right) +\log\left(\frac{1}{1.8}\right) +\log\left(\frac{1}{0.2}\right) \right]\\ &\approx 2(0.8194-0.5878-0.5878+1.6094)\\ &\approx 2.5066. \end{aligned}

For comparison, the Pearson statistic is

\begin{aligned} X^2 &=\frac{(17-16.2)^2}{16.2} +\frac{(1-1.8)^2}{1.8} +\frac{(1-1.8)^2}{1.8} +\frac{(1-0.2)^2}{0.2}\\ &\approx 0.0395+0.3556+0.3556+3.2000\\ &\approx 3.9506. \end{aligned}

Using $\chi^2_1$ as a reference gives approximate upper-tail probabilities of about 0.113 for the deviance and 0.047 for the Pearson statistic. The difference is a warning that the sparse expected count matters; it is not a way to decide that one approximation must be correct. In particular, the cell with fitted count 0.2 is not in a regime where a large-count argument is persuasive. The nominal p-values should not be treated as precise evidence without assessing the approximation.

Worked Example: A Zero Observation Has a Finite Contribution

Suppose a fitted cell mean is $\hat{\mu}=0.2$, while its observed count is $y=0$. The deviance contribution is evaluated by the zero-count convention:

d(0,0.2)=2\left[0\log\left(\frac{0}{0.2}\right)-(0-0.2)\right]=0.4.

The corresponding Pearson contribution is $(0-0.2)^2/0.2=0.2$. These are not equal, and the zero observation does not make the deviance infinite. More generally, for $y=0$ and $\mu>0$, $d(0,\mu)=2\mu$. This follows directly from the convention in the definition; it is also the limit as $y$ decreases to zero because $y\log(y/\mu)$ tends to zero. A finite contribution is a statement about the statistic’s calculation, not a guarantee that its chi-square reference distribution is reliable.

Worked Example: Why a Rare-Count Contribution Is Not Automatically Chi-Square

Let $Y\sim\operatorname{Poisson}(0.2)$, and compare the fixed mean $0.2$ with the saturated one-cell fit. This simple calculation illustrates the behavior of a deviance contribution when its expected count stays small. The probabilities of $Y=0$ and $Y=1$ are

P(Y=0)=e^{-0.2}\approx 0.8187,\qquad P(Y=1)=0.2e^{-0.2}\approx 0.1637.

At $Y=0$, the deviance is $0.4$. At $Y=1$, it is

d(1,0.2) =2\left[\log(5)-0.8\right] \approx 1.6189.

At $Y=2$, it is $2[2\log(10)-1.8]\approx 5.6103$, an event with probability $e^{-0.2}(0.2)^2/2\approx 0.0164$. The statistic therefore takes discrete values with substantial probability on its first few values; in particular, it has probability about 0.8187 of equaling 0.4. A chi-square distribution is continuous and cannot reproduce that point mass. This one-cell calculation is an illustration of sparse-count behavior, not a substitute for deriving the degrees of freedom of a loglinear model comparison. It shows why a large-sample reference cannot be presumed when relevant means remain small.

Separate Fit Diagnostics from Calibration

A fitting or numerical problem concerns whether the reported fitted means and likelihood correspond to the intended model. Warning signs include failure of the fitting algorithm to converge, fitted means that change materially with tolerances or starting values, or a likelihood that does not stabilize. Such symptoms warrant checking the model specification, data, and computation. A sparse expected count, by itself, is not evidence that fitting failed: a fit can be numerically stable and still have an unreliable chi-square approximation.

A calibration problem concerns the distribution used to turn a computed statistic into a p-value. The fit may be correct, but sparse counts, many low-mean cells, or an increasing number of cells can make the large-sample approximation questionable. There is no universal expected-count cutoff that makes every deviance test valid or invalid. The number and arrangement of sparse cells, the model, and the way the sample size grows all matter.

When sparse counts are important, useful steps include examining the fitted means under the null model, reporting the deviance statistic separately from its approximate p-value, and avoiding claims that depend heavily on a borderline chi-square result. Depending on the design and scientific question, a parametric bootstrap can assess the null distribution by simulating tables from the fitted null model and refitting the compared models in each table. Conditional exact methods may be appropriate when the conditioning scheme matches the question. These methods have their own assumptions and computational demands; neither should be selected merely because a cell falls below a rule of thumb.

Common Mistake

Incorrect reasoning: “The model converged, so the chi-square p-value is trustworthy.” Convergence indicates that the fitting procedure met its numerical criterion; it does not establish that the finite-sample distribution of the test statistic is close to chi-square.

Correction: Check the fit and the reference distribution as separate questions. Confirm that the model was fitted as intended, then assess whether the tested model’s expected counts and sampling regime make the asymptotic calibration plausible. A stable estimate can coexist with a poor chi-square approximation, and a failed fit requires attention even if all expected counts are large.

Key takeaway: Sparse fitted counts primarily threaten the accuracy of chi-square calibration, not the arithmetic definition of deviance. Large-count asymptotics explain the reference distribution under regular conditions; when those conditions are doubtful, report the limitation and consider a design-appropriate alternative.

Check Your Understanding

Answer these questions by focusing separately on the fitted model and on the test’s reference distribution.

In a nested loglinear model comparison, which model’s expected counts are most relevant when assessing sparse-cell caution?
For $y=0$ and fitted mean $\mu>0$, derive the deviance contribution and compare it with the Pearson contribution.
Why does the local expansion of the deviance contribution support a large-count approximation without proving that every model-comparison statistic is chi-square?
A fitting algorithm converges, but several null-model fitted counts are very small. What concern remains, and why is it distinct from numerical convergence?
Give one reason that an expected-count cutoff should not be treated as a universal validity rule for deviance tests.