Why Grouped Outcomes Need Their Trial Counts
A data set may record one row for each risk class, clinic, or rating band rather than one row for each individual trial. For example, a row might report 18 claims among 120 policies with a specified characteristic. The response is not simply “18” or “15%”: the number of policies observed determines how much information that proportion carries. Grouped-binomial logistic regression models both the observed successes and the number of trials behind them.
The key distinction is between a binomial denominator and a generic observation weight. A denominator states how many Bernoulli trials produced a count. It directly defines the response distribution. A generic weight may change how a row contributes to a fitting procedure, but it does not by itself establish that the row represents that many independent trials.
The model assumes that, conditional on the predictors, trials within a group have the same success probability and are independent Bernoulli trials. Thus their sum has a binomial distribution. Groups are also treated as independent conditional on the model. If trials within a group have different probabilities or dependence that the model does not account for, the binomial assumption may not be suitable.
A group’s observed proportion is \(y_j/n_j\), where \(y_j\) is its observed success count. This proportion is useful to report, but it is not the full grouped response: two groups with the same proportion and different denominators do not generally provide equal information.
Likelihood, Score, and the Role of the Denominator
For observed counts \(y_j\), the probability of the data in group \(j\) is the binomial probability, including its combinatorial coefficient. The likelihood for independent groups is therefore:
The coefficient \({n_j\choose y_j}\) does not depend on \(\boldsymbol{\beta}\), but \(n_j\) appears in the powers of both \(p_j\) and \(1-p_j\). It is part of the binomial model, not an optional adjustment to the response.
To see why the score has this form, omit terms in the log-likelihood that do not involve the coefficients. The contribution of group \(j\) is \(y_j\log(p_j)+(n_j-y_j)\log(1-p_j)\). Write \(\eta_j=\boldsymbol{x}_j^\mathsf{T}\boldsymbol{\beta}\) and use the logistic relationship \(p_j=\exp(\eta_j)/[1+\exp(\eta_j)]\). Then the derivatives of \(\log(p_j)\) and \(\log(1-p_j)\) with respect to \(\eta_j\) are \(1-p_j\) and \(-p_j\), respectively. Consequently, the derivative of the group contribution with respect to \(\eta_j\) is:
Since the derivative of \(\eta_j\) with respect to \(\boldsymbol{\beta}\) is \(\boldsymbol{x}_j\), multiplying by that vector and summing over groups gives the stated score. Differentiating again gives the expected information contribution \(n_jp_j(1-p_j)\boldsymbol{x}_j\boldsymbol{x}_j^\mathsf{T}\). In particular, when a group’s observed proportion differs from its fitted probability, increasing \(n_j\) increases the magnitude of its score contribution.
Worked Examples
Worked Example: Predicting Counts From a Grouped Logistic Model
A hypothetical insurer groups policies by the policyholder’s age \(x\). In the age-30 group, 27 of 80 policies had a specified claim during the year; in the age-50 group, 13 of 20 did. Suppose a fitted model is \(\operatorname{logit}(p)=-2+0.05x\). Find the fitted probability and expected number of claims for each group, and interpret the slope.
Method: calculate the linear predictor, transform it through the inverse-logit function, and multiply by the trial count to obtain the expected count. The recorded proportions are \(27/80=0.3375\) and \(13/20=0.65\). These observed proportions are not substituted for the model’s fitted probabilities.
For age 30, the linear predictor is \(-2+0.05(30)=-0.5\). Thus the fitted probability is \(1/(1+\exp(0.5))=0.3775\), to four decimal places. The fitted expected claim count is \(80(0.3775407)=30.2033\), or 30.203 claims to three decimal places.
For age 50, the linear predictor is \(-2+0.05(50)=0.5\). The fitted probability is \(\exp(0.5)/[1+\exp(0.5)]=0.6225\), to four decimal places. The fitted expected count is \(20(0.6224593)=12.4492\), or 12.449 claims to three decimal places.
The slope means that a one-year increase in age multiplies the modeled odds by \(\exp(0.05)=1.0513\), approximately a 5.13% increase in odds per year. It does not mean that the probability increases by 0.05. The check is that each fitted expected count is \(n_jp_j\), the mean of a binomial count under the stated model. The common mistake avoided here is treating a fitted probability as the expected count.
Worked Example: Same Proportion, Different Information
Two independent groups have the same predictor vector, and each has an observed success proportion of 0.20. Group A has 2 successes in 10 trials; Group B has 20 successes in 100 trials. At a candidate fitted probability \(p=0.10\), calculate each group’s intercept-score contribution and expected information contribution. Compare their roles.
Method: use the grouped-binomial score \(y_j-n_jp\) and intercept information \(n_jp(1-p)\). Group A’s fitted expected count is \(10(0.10)=1\). Its score contribution is \(2-1=1\), and its information contribution is \(10(0.10)(0.90)=0.9\).
Group B’s fitted expected count is \(100(0.10)=10\). Its score contribution is \(20-10=10\), and its information contribution is \(100(0.10)(0.90)=9\). Both groups have an observed proportion 0.10 above the candidate probability, but Group B’s score and information contributions are ten times Group A’s.
The check is that the trial count increases from 10 to 100 while the proportion and fitted probability remain unchanged, so the displayed score and information each increase by a factor of \(100/10=10\). The common mistake avoided is treating the two rows as equally informative simply because their observed proportions match.
Worked Example: Why Equal Row Weights Can Give a Different Fit
A hypothetical screening program reports two groups for a common-probability, intercept-only model. Group A has 1 success in 5 trials, and Group B has 8 successes in 10 trials. Find the maximum-likelihood estimate using the binomial denominators. Then compare it with the estimate obtained by taking the unweighted average of the two group proportions.
Method: maximize the grouped-binomial likelihood for the correct estimate, then calculate the unweighted row average to expose the difference. In an intercept-only model, all trials share probability \(p\). Across the two groups there are \(1+8=9\) successes and \(5+10=15\) trials. The log-likelihood, apart from terms not involving \(p\), is \(9\log(p)+6\log(1-p)\). Its derivative is \(9/p-6/(1-p)\). Setting this equal to zero gives \(9(1-p)=6p\), so \(p=9/15=0.60\).
The corresponding intercept is \(\log\{0.60/(1-0.60)\}=\log(1.5)=0.4055\), to four decimal places. The two observed proportions are \(1/5=0.20\) and \(8/10=0.80\). Their unweighted average is \((0.20+0.80)/2=0.50\), not 0.60. The binomial estimate checks against the pooled success fraction, \(9/15\); the larger group contributes more trials.
The unweighted average would give the two group rows equal influence despite their different trial counts. It is not the grouped-binomial maximum-likelihood estimate. The common mistake avoided is using the average of group proportions without accounting for their denominators.
Representing Grouped Data and Avoiding Weight Confusion
Grouped data can be represented in equivalent ways when the model and software specify the binomial response correctly. A row may contain the success count \(y_j\) and failure count \(n_j-y_j\), or it may contain the proportion \(y_j/n_j\) together with the binomial denominator \(n_j\). In the latter representation, \(n_j\) identifies the number of Bernoulli trials represented by that proportion.
Some software uses a prior-weight field to encode the denominator when the response is a proportion. In that particular setup, the weight \(n_j\) is serving as the binomial trial count. This does not make every quantity called a “weight” a binomial denominator. A frequency weight, a precision weight, or another software-specific weight has a meaning determined by the fitting procedure and data specification; it cannot be assumed to describe independent Bernoulli trials.
| Data supplied for a group | What it states | What to check |
|---|---|---|
| Successes \(y_j\) and trials \(n_j\) | A binomial count from \(n_j\) trials | Use \(n_j-y_j\) as the failure count |
| Proportion \(y_j/n_j\) with denominator \(n_j\) | A sample proportion based on \(n_j\) trials | Confirm the fitting method uses the denominator in the binomial model |
| Proportion with an unspecified ordinary weight | Not enough information to identify a binomial response | Determine what the weight means before interpreting the fit |
Grouping is appropriate when the trials combined in a row share the modeled predictors and the same conditional success probability. If two records have different predictor values, combining their counts into one row generally discards information relevant to the model. Conversely, records with identical predictor values and a common modeled probability can be represented by a grouped count without changing the binomial likelihood for the coefficients.
Before interpreting a model summary, check that the response scale and denominator have been specified as intended. A success count without its trial count is insufficient to define the grouped-binomial distribution. A proportion without its denominator may be fitted as if each row contributes equally, which can change both the fitted coefficients and their uncertainty. This distinction is especially important when groups differ greatly in size.
Check Your Understanding
For each problem, identify the grouped-binomial quantity being used before calculating.
- A group has 40 trials and 6 successes. Under a logistic model with linear predictor \(-1+0.3x\), find the fitted probability and expected success count when \(x=2\). Give the probability to four decimal places and the expected count to three decimal places. (Answer: \(p=0.4013\); expected count \(=16.052\).)
- An intercept-only grouped-binomial model has 4 successes in 20 trials and 24 successes in 80 trials. Find the maximum-likelihood estimate of the common probability and the fitted intercept, rounded to four decimal places. (Answer: \(\hat p=0.28\); intercept \(=\log(7/18)=-0.9445\).)
- Two groups have the same predictor vector and observed proportion 0.15. One has 200 trials and the other has 20. At fitted probability \(p=0.15\), what is the ratio of their intercept information contributions, larger group to smaller group? (Answer: \(10\).)
- A grouped row records 7 successes in 25 trials. State the failure count and write the response as a success-failure pair. (Answer: 18 failures; \((7,18)\).)
- Two independent groups have identical predictor values and success probabilities under the model. Group A has 3 successes in 12 trials, and Group B has 5 successes in 8 trials. If they are combined, what are the pooled success count, trial count, and observed proportion? (Answer: 8 successes, 20 trials, and observed proportion \(0.40\).)