Grouped-Binomial Goodness of Fit

Why Grouped-Binomial Fit Needs Its Own Interpretation

A grouped-binomial data set records a number of successes out of a number of trials for each covariate pattern. For example, a table might show how many patients improved among those receiving each treatment at each risk level. The fitted probability is compared with the observed success proportion, but the comparison depends on the group sizes: a difference of two successes carries different weight in a group of ten than in a group of a thousand.

The Pearson and deviance statistics provide complementary summaries of these discrepancies. In grouped-binomial models, however, it is also essential to ask what counts as a group. Combining trials with identical covariates is not the same goodness-of-fit comparison as treating every trial as its own group. The model likelihood may retain the same information about its parameters, while the saturated-model benchmark—and therefore the residual fit statistic—changes.

Grouped-Binomial Setup and Fit Statistics

Suppose there are $m$ groups. In group $j$, $n_j$ independent trials produce $Y_j$ successes, where $Y_j\sim\operatorname{Bin}(n_j,p_j)$. Assume the group counts are independent. A GLM specifies $g(p_j)=x_j^\mathsf{T}\beta$, where $g$ is the link function and $x_j$ is the covariate vector. After fitting the model, write the estimated probability as $\hat p_j$, the fitted number of successes as $\hat\mu_j=n_j\hat p_j$, and the fitted number of failures as $n_j-\hat\mu_j$.

Definition: For grouped-binomial data, the Pearson goodness-of-fit statistic and residual deviance are $$ X^2=\sum_{j=1}^m\frac{(Y_j-\hat\mu_j)^2}{n_j\hat p_j(1-\hat p_j)} $$ and $$ D=2\sum_{j=1}^m\left[ Y_j\log\!\left(\frac{Y_j}{\hat\mu_j}\right) +(n_j-Y_j)\log\!\left(\frac{n_j-Y_j}{n_j-\hat\mu_j}\right) \right]. $$ Terms with an observed count of zero are defined by continuity as zero. These formulas assume fitted probabilities strictly between zero and one; boundary fits require separate care.

The two statistics use the same observed and fitted counts but measure discrepancy differently. Pearson’s statistic squares a residual and scales it by its fitted binomial variance. Deviance compares the fitted model’s likelihood with that of a saturated model, which assigns a separate success probability to each group. Both statistics are nonnegative; their sizes are interpreted relative to a reference distribution only when its conditions are reasonably met.

Worked Example: An Intercept-Only Fit

Three groups each contain $10$ trials, with observed success counts $4,6,8$. Fit the intercept-only binomial model. Its estimated probability is the pooled proportion: $$ \hat p=\frac{4+6+8}{10+10+10}=\frac{18}{30}=0.6. $$ Thus each group has $\hat\mu_j=6$ fitted successes, $4$ fitted failures, and fitted variance $10(0.6)(0.4)=2.4$.

For Pearson’s statistic, the three contributions are $$ \frac{(4-6)^2}{2.4}=1.6667,\qquad \frac{(6-6)^2}{2.4}=0,\qquad \frac{(8-6)^2}{2.4}=1.6667. $$ Therefore $X^2=3.3333$.

For deviance, the first group contributes $$ 2\left[4\log(4/6)+6\log(6/4)\right]=1.6219. $$ The second group contributes zero because its observed and fitted counts agree. The third contributes $$ 2\left[8\log(8/6)+2\log(2/4)\right]=1.8303. $$ Hence $D=1.6219+0+1.8303=3.4522$. There are $m=3$ groups and one fitted coefficient, so the residual degrees of freedom are $3-1=2$. This is a small illustration, not by itself a guarantee that a chi-square approximation is accurate.

Where the Deviance Formula Comes From

The binomial likelihood for group $j$, apart from its combinatorial factor, is proportional to $p_j^{Y_j}(1-p_j)^{n_j-Y_j}$. Its maximizer over an unrestricted group probability is the observed proportion $Y_j/n_j$, with the natural boundary interpretation when all trials have the same outcome. The saturated model uses that maximizer separately for every group.

Result: The grouped-binomial deviance is twice the difference between the saturated-model log-likelihood and the fitted-model log-likelihood. It equals the displayed count-based formula, and each group’s contribution is nonnegative.

Derivation. The log-likelihood, including the binomial coefficient, is $$ \ell(p)=\sum_{j=1}^m\left[ \log {n_j\choose Y_j} +Y_j\log(p_j)+(n_j-Y_j)\log(1-p_j) \right]. $$ For an interior observed proportion, the saturated estimate is $\tilde p_j=Y_j/n_j$; for $Y_j=0$ or $n_j$, it is $0$ or $1$, respectively. The binomial coefficient is identical in the fitted and saturated log-likelihoods, so it cancels. Subtracting the fitted log-likelihood from the saturated log-likelihood and multiplying by two gives $$ 2\sum_{j=1}^m\left[ Y_j\log\!\left(\frac{Y_j/n_j}{\hat p_j}\right) +(n_j-Y_j)\log\!\left(\frac{(n_j-Y_j)/n_j}{1-\hat p_j}\right) \right], $$ which is the stated expression after simplifying the ratios.

For nonnegativity, set $a=Y_j/n_j$ and $b=\hat p_j$. The group contribution is $2n_j[a\log(a/b)+(1-a)\log((1-a)/(1-b))]$. The inequality $\log u\leq u-1$ for $u>0$, applied to $b/a$ and $(1-b)/(1-a)$ when $0<a<1$, gives $$ a\log(b/a)+(1-a)\log((1-b)/(1-a)) \leq b-a+(1-b)-(1-a)=0. $$ Negating both sides proves the contribution is nonnegative. The endpoint cases $a=0$ and $a=1$ follow directly, or by continuity. Equality holds when $a=b$. Thus $D\geq0$, with zero contribution in a group exactly when its observed and fitted proportions agree. $\square$

Calculating from Observed and Expected Counts

A reliable calculation starts with a group-by-group table. Record $n_j$, $Y_j$, the observed failures $n_j-Y_j$, $\hat p_j$, and the fitted counts $\hat\mu_j$ and $n_j-\hat\mu_j$. The Pearson denominator is the fitted variance of the success count, not the total number of trials. The deviance uses both the success and failure counts, so a discrepancy in either is reflected in its contribution.

Worked Example: Comparing Contributions with Unequal Group Sizes

Consider two groups with $(n_1,Y_1)=(20,12)$ and $(n_2,Y_2)=(10,2)$. Suppose a fitted model gives $\hat p_1=0.5$ and $\hat p_2=0.3$. These supplied fitted probabilities give expected success counts $10$ and $3$, and expected failure counts $10$ and $7$.

The Pearson contributions are $$ \frac{(12-10)^2}{20(0.5)(0.5)}=\frac{4}{5}=0.8000,\qquad \frac{(2-3)^2}{10(0.3)(0.7)}=\frac{1}{2.1}=0.4762. $$ Thus $X^2=1.2762$.

For the first group, the deviance contribution is $$ 2\left[12\log(12/10)+8\log(8/10)\right] =2\left[12(0.182322)+8(-0.223144)\right] =0.8054. $$ For the second group, it is $$ 2\left[2\log(2/3)+8\log(8/7)\right] =2\left[2(-0.405465)+8(0.133531)\right] =0.5146. $$ Therefore, calculating from the unrounded contributions gives $D=1.32006239\ldots=1.3201$ to four decimal places. The two totals differ because the statistics scale and aggregate the discrepancies differently; they are not expected to match exactly in a finite sample.

Degrees of Freedom and the Role of Grouping

When the model has $r$ identifiable fitted coefficients and the $m$ group probabilities are the saturated benchmark, the residual degrees of freedom are $m-r$. Under the null model, the usual chi-square reference for $X^2$ and $D$ is an asymptotic approximation: the groups must be independent, the model must be correctly specified and regular, the true probabilities must be interior, the design rank must be $r$, and the information must increase so that the likelihood and Pearson approximations apply. A small residual degrees of freedom count or poor expected-count behavior can make this reference unreliable; the statistics remain descriptive discrepancies even when the reference is doubtful.

The number of groups is not merely a formatting choice. It determines what the saturated model is allowed to fit. If trials share an identical covariate vector, their individual outcomes can be added to form a binomial count without changing the parameter-dependent part of the likelihood. To see this, for a common probability $p$, the likelihood of a particular sequence of $n$ binary outcomes is $p^y(1-p)^{n-y}$. There are ${n\choose y}$ sequences with $y$ successes, so the grouped count has probability ${n\choose y}p^y(1-p)^{n-y}$. The extra factor does not depend on $p$; therefore the likelihood for estimating $p$ or regression coefficients is unchanged by aggregation within genuinely identical covariate patterns.

The saturated benchmark does change. A grouped saturated model gives one probability to all trials in a covariate pattern, whereas an individual-level saturated model can give a separate probability to each trial. Consequently, the residual deviance and its residual degrees of freedom can differ even though the fitted regression parameters are based on equivalent likelihood information. Grouping trials across different covariate values is more consequential: it treats them as having a common modeled probability and can conceal variation that the finer covariates would have represented.

Worked Example: Two Covariate Groups Versus Individual Trials

Suppose two covariate patterns each have $10$ trials. Their observed success counts are $2$ and $8$. An intercept-only model estimates $\hat p=(2+8)/20=0.5$. With the two patterns as groups, each fitted success and failure count is $5$.

The grouped Pearson statistic is $$ X^2=\frac{(2-5)^2}{2.5}+\frac{(8-5)^2}{2.5}=3.6+3.6=7.2, $$ because each group’s fitted variance is $10(0.5)(0.5)=2.5$. The grouped deviance is $$ D=2\left[2\log(2/5)+8\log(8/5)\right] +2\left[8\log(8/5)+2\log(2/5)\right] =7.7098. $$ There are two groups and one fitted intercept, giving one residual degree of freedom.

If instead every binary trial is treated as its own group, the saturated model can assign a separate probability to each observed zero or one. Its fitted likelihood is then higher, and the deviance against it is not $7.7098$. In fact, under the intercept-only fit, each of the $20$ individual observations contributes $-2\log(0.5)=1.3863$, giving an individual-level deviance of $27.7259$. This is a different benchmark, not evidence that one arithmetic calculation is wrong. The grouped statistic assesses discrepancies between the two pattern-level proportions and the common fitted probability.

Common Mistake and Practical Interpretation

Using the wrong denominator. The grouped Pearson contribution divides by $n_j\hat p_j(1-\hat p_j)$, the fitted variance of $Y_j$, not by $n_j$ or by the fitted number of successes alone.
Omitting failures from the deviance. A binomial group contributes a success term and a failure term. Leaving out the failure term generally gives the wrong result.
Treating the grouping as irrelevant. Aggregation within identical covariate patterns preserves the parameter-dependent likelihood, but it changes the saturated model and the residual fit comparison. Combining unlike patterns can also discard relevant structure.
Reading a chi-square value as a verdict by itself. The reference approximation depends on regularity and information conditions. A calculated statistic should be accompanied by its degrees of freedom, the grouping definition, and an assessment of whether the approximation is credible.

The key practical lesson is to report not just a Pearson statistic or deviance, but also what was counted as a group, how the fitted success and failure counts were obtained, and which saturated benchmark defines the comparison. These details make clear what kind of lack of fit the summary can detect.

Check Your Understanding

Use the definitions and reasoning above to answer these questions.

For a group with $n_j=25$ and fitted probability $\hat p_j=0.4$, what is the fitted success count and the fitted variance used in the Pearson denominator?
Why does the grouped-binomial deviance include terms for both successes and failures?
State the usual residual degrees of freedom when there are $m$ groups and $r$ identifiable fitted coefficients.
Explain why aggregation within identical covariate patterns can preserve likelihood information about regression coefficients while changing the residual deviance benchmark.
What modeling assumption is made when trials with different covariate patterns are combined into one binomial group?