Logistic Regression for Grouped Success Probabilities

From Grouped Counts to a Logistic Model

A grouped binary data set records, for each group, how many trials resulted in success and how many trials were observed in total. Logistic regression models the success probability for each group through its covariates. If groups have different covariate values, the model assigns them different probabilities; if they have different trial totals, their likelihood contributions have different exponents.

The goal is to specify both parts precisely: the probability model for each group’s count and the way covariates determine that group’s success probability. Once those are stated, the joint likelihood follows directly from the binomial probability mass function and independence across groups.

Definition: For group $i$, let $Y_i$ be the number of successes among $m_i$ trials, and let $y_i$ be its observed value, where $i=1,\ldots,G$ and $0\leq y_i\leq m_i$. Let $x_i$ be a specified $p$-dimensional covariate vector, including an intercept component if the model has an intercept. A grouped logistic regression model is $$ Y_i\mid x_i \sim \operatorname{Binomial}(m_i,\pi_i),\qquad \operatorname{logit}(\pi_i)=x_i^\mathsf{T}\beta, $$ where $\beta$ is a $p$-dimensional coefficient vector, $\pi_i$ is the success probability in group $i$, and $\operatorname{logit}(\pi)=\log\{\pi/(1-\pi)\}$. The model assumes that, conditional on the covariates, the group counts are independent and each group’s trials have a common success probability $\pi_i$.

The trial total $m_i$ and covariate vector $x_i$ play different roles. The trial total determines how many Bernoulli outcomes contribute to group $i$’s count; the covariates determine its probability. The total number of successes $y_i$ is the observed response. In particular, the response count is not just the proportion $y_i/m_i$: the denominator is part of the data model.

Turning a Linear Predictor into a Probability

Write $\eta_i=x_i^\mathsf{T}\beta$ for the linear predictor in group $i$. Solving the logit equation for $\pi_i$ gives the logistic, or inverse-logit, function.

Result: For any finite real $\eta_i$, the logit-linear model assigns $$ \pi_i=\frac{\exp(\eta_i)}{1+\exp(\eta_i)}. $$ This probability is strictly between 0 and 1, and its logit is exactly $\eta_i$.

To verify the result, the right-hand side is positive and less than 1 because its denominator exceeds its positive numerator. Also, $$ \frac{\pi_i}{1-\pi_i} = \frac{\exp(\eta_i)/(1+\exp(\eta_i))} {1/(1+\exp(\eta_i))} =\exp(\eta_i), $$ so taking logarithms gives $\operatorname{logit}(\pi_i)=\eta_i$. Conversely, if $0<\pi_i<1$, setting $\eta_i=\log\{\pi_i/(1-\pi_i)\}$ and applying the displayed formula recovers $\pi_i$. Thus the logit maps probabilities in $(0,1)$ one-to-one to finite real linear predictors.

Worked Example: A One-Covariate Dose Model

Suppose three dose groups have doses $x_i=0,1,2$, trial totals $m_i=10,20,8$, and observed success counts $y_i=2,9,5$. Consider the logit-linear model $$ \operatorname{logit}(\pi_i)=\beta_0+\beta_1x_i. $$ For example, $\beta_0=-1$ and $\beta_1=0.7$ give linear predictors $-1$, $-0.3$, and $0.4$ at the three doses. The probabilities are therefore $$ \pi_1=\frac{\exp(-1)}{1+\exp(-1)}\approx 0.2689,\qquad \pi_2=\frac{\exp(-0.3)}{1+\exp(-0.3)}\approx 0.4256,\qquad \pi_3=\frac{\exp(0.4)}{1+\exp(0.4)}\approx 0.5987. $$

Under this specified model, the count distributions are $Y_1\sim\operatorname{Binomial}(10,\pi_1)$, $Y_2\sim\operatorname{Binomial}(20,\pi_2)$, and $Y_3\sim\operatorname{Binomial}(8,\pi_3)$, with the counts independent. The first group has 10 trials and probability about 0.2689; the second has 20 trials and probability about 0.4256. The difference in trial totals does not alter the form of the linear predictor, but it does alter the binomial distribution and likelihood contribution.

Deriving the Joint Likelihood

For fixed covariates and trial totals, the binomial probability mass function gives the probability of each observed group count. Independence across groups makes the probability of the complete observed vector the product of those group probabilities.

Result: If $Y_1,\ldots,Y_G$ are independent conditional on their covariates, with $Y_i\mid x_i\sim\operatorname{Binomial}(m_i,\pi_i)$ and $\operatorname{logit}(\pi_i)=x_i^\mathsf{T}\beta$, the likelihood for $\beta$, evaluated at counts $y_1,\ldots,y_G$, is $$ L(\beta) = \prod_{i=1}^{G} \binom{m_i}{y_i}\pi_i^{y_i}(1-\pi_i)^{m_i-y_i}. $$ Equivalently, $$ L(\beta) = \prod_{i=1}^{G} \binom{m_i}{y_i} \frac{\exp\{y_i x_i^\mathsf{T}\beta\}} {\left[1+\exp\{x_i^\mathsf{T}\beta\}\right]^{m_i}}. $$

For each group, the binomial mass at its observed count is $\binom{m_i}{y_i}\pi_i^{y_i}(1-\pi_i)^{m_i-y_i}$. Multiplying these masses is justified by conditional independence. Substituting $\pi_i=\exp(\eta_i)/\{1+\exp(\eta_i)\}$ and $1-\pi_i=1/\{1+\exp(\eta_i)\}$ gives $$ \binom{m_i}{y_i} \left(\frac{\exp(\eta_i)}{1+\exp(\eta_i)}\right)^{y_i} \left(\frac{1}{1+\exp(\eta_i)}\right)^{m_i-y_i} = \binom{m_i}{y_i} \frac{\exp(y_i\eta_i)}{[1+\exp(\eta_i)]^{m_i}}. $$ Taking the product over $i$ proves the stated likelihood.

Because the likelihood is positive for finite coefficients and valid counts, its logarithm is defined. Taking logarithms of the product yields the log-likelihood $$ \ell(\beta) = \sum_{i=1}^{G} \left[ \log\binom{m_i}{y_i} +y_i x_i^\mathsf{T}\beta -m_i\log\{1+\exp(x_i^\mathsf{T}\beta)\} \right]. $$

This expression displays the role of the unequal trial totals: $m_i$ multiplies the normalizing term for group $i$. The term $\log\binom{m_i}{y_i}$ depends on the observed data, not on $\beta$. It is part of the likelihood as written, even though it is constant with respect to the coefficients.

Worked Example: Evaluating a Likelihood with Unequal Group Sizes

Continue with the three dose groups and coefficient values $\beta_0=-1,\ \beta_1=0.7$. The observed data are $(m_i,y_i)=(10,2),(20,9),(8,5)$, and the probabilities rounded to four decimal places are $0.2689,\ 0.4256,\ 0.5987$. Using these displayed probabilities, the first group contributes $$ \binom{10}{2}(0.2689)^2(1-0.2689)^8 = 45(0.2689)^2(0.7311)^8 \approx 0.265588. $$

The second group contributes $$ \binom{20}{9}(0.4256)^9(1-0.4256)^{11} = \binom{20}{9}(0.4256)^9(0.5744)^{11} \approx 0.172814. $$ The third contributes $$ \binom{8}{5}(0.5987)^5(1-0.5987)^3 = 56(0.5987)^5(0.4013)^3 \approx 0.278382. $$

The groups are modeled as independent, so their joint likelihood at these coefficient values is the product: $$ L(\beta)\approx (0.265588)(0.172814)(0.278382)\approx 0.012777. $$ To four significant figures this is $0.01278$. These are likelihood contributions for the observed counts, not probabilities of the coefficient values. The calculation uses each group’s own trial total and success probability.

Several Covariates and Group-Level Design

The covariate vector can contain more than one predictor. For example, it may include an intercept, an indicator for treatment, and a centered continuous measurement. Each group receives a row of covariates, and that row’s inner product with the same coefficient vector determines the group’s linear predictor. The likelihood formula remains unchanged; only the calculation of $x_i^\mathsf{T}\beta$ differs.

Worked Example: Treatment and a Continuous Covariate

Suppose each group is described by a treatment indicator $t_i$ and an age $a_i$, measured in years. Use age centered at 50 and the model $$ \operatorname{logit}(\pi_i) = \beta_0+\beta_1t_i+\beta_2(a_i-50). $$ Let $(\beta_0,\beta_1,\beta_2)=(-1,0.5,0.1)$. Consider three independent groups: 10 control trials at age 50 with 3 successes; 12 treatment trials at age 50 with 6 successes; and 8 treatment trials at age 60 with 5 successes.

For the control group, $t_1=0$ and $a_1-50=0$, so $\eta_1=-1$ and $\pi_1=\exp(-1)/\{1+\exp(-1)\}\approx0.2689$. For the first treatment group, $\eta_2=-1+0.5= -0.5$, giving $\pi_2\approx0.3775$. For the second treatment group, $\eta_3=-1+0.5+0.1(10)=0.5$, giving $\pi_3\approx0.6225$.

Writing $Y_i$ for the group counts, the model states $Y_1\sim\operatorname{Binomial}(10,\pi_1)$, $Y_2\sim\operatorname{Binomial}(12,\pi_2)$, and $Y_3\sim\operatorname{Binomial}(8,\pi_3)$. The likelihood at these data is $$ L(\beta)= \binom{10}{3}\pi_1^3(1-\pi_1)^7 \binom{12}{6}\pi_2^6(1-\pi_2)^6 \binom{8}{5}\pi_3^5(1-\pi_3)^3, $$ with each $\pi_i$ given by its calculated inverse-logit. For instance, the second group’s exponent on $1-\pi_2$ is 6 because it has $12-6=6$ failures. Covariates determine $\pi_i$, while the observed count and trial total determine the powers in that group’s likelihood term.

Assumptions and Common Mistakes

A grouped binomial logistic model is appropriate when the trials represented within a group share a success probability conditional on the modeled covariates, and the groups are independent under the model. If trials within a group have systematically different probabilities that the model does not account for, treating their total as one binomial count can be inappropriate. The model also presumes the stated linear form for the logit; it does not assert that the probability itself is linear in the covariates.

Mistake: treating a proportion as an equally precise response regardless of its denominator. A proportion of $2/10$ and a proportion of $20/100$ are both 0.2, but they arise from different numbers of trials. In the binomial likelihood their trial totals determine different powers and different sampling distributions.
Mistake: using a trial total as though it were a replication weight on the proportion. The response is the count $y_i$ with binomial size $m_i$, or equivalently the proportion together with its denominator in a correctly specified binomial likelihood. A proportion alone does not identify that likelihood.
Mistake: assigning one probability to every group. With covariates, group $i$’s probability is $\operatorname{logit}^{-1}(x_i^\mathsf{T}\beta)$. Different covariate vectors can give different probabilities even when the trial totals match.
Mistake: dropping the binomial coefficient without saying why. It is required for the full likelihood of the observed counts. It is constant in $\beta$, so it may be omitted when writing a likelihood only up to proportionality as a function of $\beta$, but that is a different convention.

Key takeaway: For each group, specify a binomial count with its own trial total and a probability obtained by applying the inverse-logit to its covariate-based linear predictor. Under conditional independence, multiply the resulting binomial masses to obtain the grouped logistic likelihood.

Check Your Understanding

Use the model and likelihood definitions above to answer the following.

For a group with 7 successes from 15 trials, what are $y_i$, $m_i$, and the number of failures?
If $x_i^\mathsf{T}\beta=0$, what is the group’s success probability?
Why does the likelihood term for a group with 20 trials have a different exponent on $1-\pi_i$ from a group with 10 trials, even if their observed proportions match?
Write the likelihood contribution for a group with 4 successes from 9 trials and success probability $\pi_i$.
State the conditional independence and within-group probability assumptions needed to form the joint likelihood as a product of binomial terms.