Why Compare Grouped-Binomial Models?
A grouped-binomial observation records a number of successes out of a known number of trials. For example, a clinic might report the number of patients responding to treatment among all patients in each age-and-dose group. A regression model relates each group’s success probability to its covariates, and a nested-model deviance test asks whether the added terms improve the fit enough to justify their extra parameters.
The test is a likelihood-ratio comparison. Its calculation is especially transparent when both models are fitted to the same grouped responses: subtract their residual deviances, count the additional parameters in the larger model, and calibrate the difference against a chi-square distribution when the usual likelihood regularity conditions hold.
Deviance and the Nested-Model Statistic
Write \(y_i\) for the observed number of successes in group \(i\), and let \(\hat p_i\) be the fitted success probability under a model. The grouped-binomial log-likelihood is
The saturated model assigns a separate probability to every group and, when \(0<y_i<m_i\), estimates it as \(y_i/m_i\). The residual deviance of a fitted model is twice the difference between the maximized saturated log-likelihood and the fitted model’s maximized log-likelihood.
The binomial coefficients in the likelihood do not need to be calculated separately for each model: they are identical across models fitted to the same data and cancel in likelihood differences. The following derivation makes the test statistic explicit.
Derivation. Let \(\ell_{\mathrm{sat}}\) be the maximized log-likelihood under the saturated model. By definition, \(D(M_j)=2\{\ell_{\mathrm{sat}}-\ell(\hat\beta_j)\}\) for \(j=0,1\). Therefore,
The saturated-model term cancels. Moreover, because the smaller model’s parameter space is contained in the larger model’s, maximization over the larger space cannot produce a lower maximized log-likelihood: \(\ell(\hat\beta_1)\geq\ell(\hat\beta_0)\). Thus \(D(M_0)-D(M_1)\geq0\), apart from numerical optimization error. This proves both the identity and the expected direction of the difference. \(\square\)
If the two models are regular, identifiable, and fitted to the same response data, Wilks’ theorem gives the usual large-sample reference distribution under the null hypothesis. In particular, suppose the null parameter is an interior point of the larger model’s parameter space, the model is correctly specified, and the Fisher information for the identifiable parameters is nonsingular. If the larger model has \(q_1\) free parameters and the smaller has \(q_0\), the statistic is asymptotically \(\chi^2_{q_1-q_0}\) under the null. This is an approximation, not an exact finite-sample distribution.
Worked Comparisons
Worked Example: Common Probability Versus Group-Specific Probabilities
Two independent groups each have 10 trials. The observed success counts are 1 and 9. Compare a model with one common success probability to a model with a separate probability for each group.
Under the smaller model, the maximum-likelihood estimate is the pooled success proportion, \(\hat p_0=(1+9)/(10+10)=0.5\). Under the larger, group-specific model, the estimates are \(\hat p_1=1/10=0.1\) and \(\hat p_2=9/10=0.9\). The larger model is saturated for these two groups.
For the first group, the likelihood ratio of its saturated fit to the common-probability fit is \((0.1/0.5)^1(0.9/0.5)^9=0.2(1.8)^9\). The second group gives the same ratio by symmetry. Hence
The larger model has two parameters and the smaller has one, so the degrees of freedom are \(2-1=1\). The upper-tail probability is approximately \(P(\chi^2_1\geq14.7226)=0.00012\). Under the regular large-sample calibration, this is evidence that the groups do not share a common success probability. With only two small groups, however, the asymptotic p-value should be interpreted cautiously.
Worked Example: Three Groups and a Two-Degree-of-Freedom Test
Three groups each have 10 trials, with success counts \(2,5,8\). Compare an intercept-only logistic model, which gives all groups one common probability, with a model having a separate probability for each group.
The pooled estimate under the smaller model is \((2+5+8)/30=0.5\). The larger model’s estimates are \(0.2,0.5,0.8\). The middle group contributes zero to the likelihood-ratio statistic because its fitted probability is unchanged. The first and third groups have equal contributions:
There are three free probabilities in the larger model and one common probability in the smaller, giving \(3-1=2\) degrees of freedom. Thus \(p=P(\chi^2_2\geq7.7098)=\exp(-7.7098/2)\approx0.0212\). The data provide evidence against a common probability at the 5% level under this calibration.
Worked Example: Interpreting a Three-Parameter Addition
Suppose a grouped-binomial logistic analysis compares a smaller model with linear predictor \(\beta_0+\beta_1x_i\) to a larger model that adds three terms, giving \(\beta_0+\beta_1x_i+\beta_2z_i+\beta_3w_i+\beta_4v_i\). The models are fitted to the same grouped responses. The reported residual deviances are \(D(M_0)=25.40000\) and \(D(M_1)=8.36958\).
The deviance difference is \(G^2=25.40000-8.36958=17.03042\). The smaller model has 2 coefficients and the larger has 5, so the reference distribution has \(5-2=3\) degrees of freedom. The upper-tail probability is
This small p-value is evidence against the joint null hypothesis \(\beta_2=\beta_3=\beta_4=0\), subject to the model and large-sample assumptions. It tests the three added terms jointly; it does not say which individual term is responsible for the improvement.
Assumptions and Common Pitfalls
The chi-square approximation is most defensible when the model is regular and the data contain enough information for the likelihood to be well approximated locally by a quadratic. Independence of the grouped responses and a common likelihood for the two fits are essential. Very sparse groups, fitted probabilities near zero or one, complete or quasi-complete separation, or parameters on a boundary can undermine the standard approximation. These concerns are especially relevant when the number of groups is small or trial counts are low.
A frequent mistake is to use the larger model’s residual deviance alone as the test statistic. Residual deviance measures lack of fit relative to the saturated model; it is the difference in residual deviances that compares the two nested regression models. Another mistake is to use the number of observations, groups, or added rows of a design matrix as the degrees of freedom without checking the parameter dimensions. The relevant difference is the difference in identifiable free parameters under the two models.
The comparison also requires the same grouped responses, trial counts, and likelihood formulation. If the models use different observations or represent different likelihoods, cancellation of the saturated-model term does not justify this ordinary nested deviance test. Finally, a small p-value is not a measure of effect size, nor does it guarantee that the larger model captures the data well; inspect its fit and scientific interpretation separately.
Check Your Understanding
Use the definitions and examples above to answer these questions.
- Why do the binomial coefficient terms cancel when two grouped-binomial models are compared on the same observations?
- A smaller model has 4 free parameters and a nested larger model has 7. What are the degrees of freedom for their likelihood-ratio test?
- If the smaller model has residual deviance 18.6 and the larger has residual deviance 11.2, what is the deviance-difference statistic?
- Why is the saturated-model deviance of the larger model, by itself, not the statistic for comparing the two regression models?
- Name two circumstances that can make the usual chi-square calibration questionable.