Grouped Bernoulli Data and Aggregation Constants

Why Grouped Counts Have a Different Likelihood Constant

Suppose a study records a binary outcome for every participant, but reports only the number of successes in each group. The group count can be modeled directly with a binomial distribution when participants in a group share the same success probability. Its likelihood includes a combinatorial factor that is absent from the likelihood of one particular ordered sequence of individual outcomes. Understanding this difference helps distinguish a change in likelihood’s parameter-independent constant from a change in the model itself.

The key condition is that every Bernoulli observation within a group has the same probability. Under this condition, all individual outcome sequences with the same number of successes have the same probability. The grouped count’s probability is the sum of those sequence probabilities.

Definition: In group \(i\), let \(Y_{ij}\) be the binary outcome for individual \(j\), where \(j=1,\ldots,m_i\). Assume the outcomes are independent and \(Y_{ij}\sim\operatorname{Bernoulli}(\pi_i)\) for every \(j\) in that group. The number of successes is \(S_i=\sum_{j=1}^{m_i}Y_{ij}\), with observed value \(s_i\). The group has a common success probability \(\pi_i\), while different groups may have different probabilities.

In a GLM, \(\pi_i\) may depend on group-level covariates through a linear predictor and link, for example \(\operatorname{logit}(\pi_i)=\mathbf{x}_i^\mathsf{T}\boldsymbol{\beta}\). The derivation below concerns the likelihood for the observed outcomes, conditional on the covariates and group sizes. The essential assumption is that all members of a given group have the same \(\pi_i\).

From an Individual Sequence to a Group Count

For a particular observed sequence \(y_{i1},\ldots,y_{im_i}\), the Bernoulli likelihood is the product of the individual probabilities. If the sequence contains \(s_i\) successes and \(m_i-s_i\) failures, independence gives

L_i^{\mathrm{ind}}(\pi_i) =\prod_{j=1}^{m_i}\pi_i^{y_{ij}}(1-\pi_i)^{1-y_{ij}} =\pi_i^{s_i}(1-\pi_i)^{m_i-s_i}.

This is the probability of that particular sequence. For example, when \(m_i=4\) and \(s_i=3\), the sequence \(1,0,1,1\) has probability \(\pi_i^3(1-\pi_i)\). The count \(S_i=3\), however, occurs for any of the \(\binom{4}{3}\) possible placements of the one failure. Since all those sequences have the same probability, their probabilities add.

Result: Under independent Bernoulli outcomes with a common probability \(\pi_i\) within group \(i\), the grouped count has distribution \(S_i\sim\operatorname{Binomial}(m_i,\pi_i)\). Its probability mass function at \(s_i\) is \(\binom{m_i}{s_i}\pi_i^{s_i}(1-\pi_i)^{m_i-s_i}\), for \(s_i=0,\ldots,m_i\).

To see the result directly, consider all sequences of length \(m_i\) containing exactly \(s_i\) ones. There are \(\binom{m_i}{s_i}\) such sequences: choosing the sequence is equivalent to choosing which \(s_i\) positions contain successes. Each sequence has probability \(\pi_i^{s_i}(1-\pi_i)^{m_i-s_i}\). The events represented by the distinct sequences are disjoint, and their union is the event \(S_i=s_i\). Adding their probabilities therefore gives

P(S_i=s_i) =\binom{m_i}{s_i}\pi_i^{s_i}(1-\pi_i)^{m_i-s_i}.

This derivation requires independence and a common probability within the group. It does not follow merely from the fact that the observations are binary.

Worked Example: One Sequence Versus One Count

Four independent individuals each have success probability \(\pi=0.3\). Compare the probability of the specific sequence \(1,0,1,1\) with the probability that exactly three of the four outcomes are successes.

The sequence has three successes and one failure, so its probability is

$$ 0.3^3(1-0.3)=0.027(0.7)=0.0189. $$

There are \(\binom{4}{3}=4\) sequences with exactly three successes. Thus the count probability is

\binom{4}{3}(0.3)^3(0.7)^1 =4(0.0189)=0.0756.

The count probability is four times the sequence probability because it includes four disjoint possible placements of the failure. The factor four depends on the observed group size and success count, not on \(\pi\).

Likelihoods for Several Groups

If groups are independent and each satisfies the common-probability assumption, multiply their count probabilities. For observed counts \(s_i\) from \(g\) groups, the grouped likelihood is

L_{\mathrm{group}}(\boldsymbol{\theta}) =\prod_{i=1}^{g}\binom{m_i}{s_i} \pi_i(\boldsymbol{\theta})^{s_i} \{1-\pi_i(\boldsymbol{\theta})\}^{m_i-s_i}.

Here \(\boldsymbol{\theta}\) denotes the model parameters, which may be regression coefficients determining the group probabilities. The likelihood for one particular individual-level outcome sequence in every group is

L_{\mathrm{ind}}(\boldsymbol{\theta}) =\prod_{i=1}^{g} \pi_i(\boldsymbol{\theta})^{s_i} \{1-\pi_i(\boldsymbol{\theta})\}^{m_i-s_i}.

Consequently, for the same model and observed group counts,

L_{\mathrm{group}}(\boldsymbol{\theta}) =C\,L_{\mathrm{ind}}(\boldsymbol{\theta}), \qquad C=\prod_{i=1}^{g}\binom{m_i}{s_i}.

The factor \(C\) is positive and fixed once the observed group sizes and counts are fixed. It does not depend on \(\boldsymbol{\theta}\). Taking logarithms gives the corresponding relationship between log-likelihoods:

\ell_{\mathrm{group}}(\boldsymbol{\theta}) =\ell_{\mathrm{ind}}(\boldsymbol{\theta})+\log C.

Thus the grouped and individual log-likelihoods have the same parameter-dependent part. The constant term is the sum of the logarithms of the binomial coefficients.

Worked Example: The Constant for Two Groups

Suppose two independent groups have sizes \(m_1=5\) and \(m_2=3\), with observed success counts \(s_1=2\) and \(s_2=1\). Let their success probabilities be \(\pi_1=0.4\) and \(\pi_2=0.6\). For any individual sequence consistent with these counts, the probability is

$$ (0.4)^2(0.6)^3(0.6)^1(0.4)^2 =(0.4)^4(0.6)^4 =0.00331776. $$

There are \(\binom{5}{2}=10\) possible placements of the successes in group 1 and \(\binom{3}{1}=3\) in group 2. There are \(10\cdot3=30\) joint sequences consistent with both counts. The grouped likelihood is therefore

$$ 30(0.00331776)=0.0995328. $$

Equivalently, multiplying the two binomial probabilities gives

\left\{\binom{5}{2}(0.4)^2(0.6)^3\right\} \left\{\binom{3}{1}(0.6)(0.4)^2\right\} =30(0.00331776)=0.0995328.

The multiplier 30 is determined by the observed counts and group sizes. If \(\pi_1\) or \(\pi_2\) changes, the individual-sequence likelihood and the grouped likelihood change by the same multiplicative factor.

What the Constant Does—and Does Not—Change

A parameter-independent factor changes the numerical likelihood value, but not its maximizing parameter. The same fact applies to the log-likelihood, scores, and any derivatives that exist. This is why one may often omit the binomial coefficients when maximizing a grouped binomial likelihood. The precise statement is as follows.

Theorem: Let \(\Theta\) be a parameter space, and suppose two likelihoods for the same fixed observations satisfy \(L_1(\boldsymbol{\theta})=C L_2(\boldsymbol{\theta})\) for every \(\boldsymbol{\theta}\in\Theta\), where \(C>0\) does not depend on \(\boldsymbol{\theta}\). Then their sets of maximizers over \(\Theta\) are identical. Wherever both log-likelihoods are defined and differentiable, their score vectors and all corresponding derivatives of positive order are identical.

Proof. For every \(\boldsymbol{\theta}\in\Theta\), multiplication by the positive constant \(C\) preserves order. Hence for any \(\boldsymbol{\theta}_1,\boldsymbol{\theta}_2\in\Theta\),

L_1(\boldsymbol{\theta}_1)\geq L_1(\boldsymbol{\theta}_2) \quad\Longleftrightarrow\quad L_2(\boldsymbol{\theta}_1)\geq L_2(\boldsymbol{\theta}_2).

The two likelihoods therefore have exactly the same maximizers, including when a maximum occurs at a boundary of \(\Theta\). On any region where the log-likelihoods are defined,

\ell_1(\boldsymbol{\theta}) =\log C+\ell_2(\boldsymbol{\theta}).

Since \(\log C\) is constant in \(\boldsymbol{\theta}\), its derivatives of every positive order are zero. Differentiating the identity proves equality of the scores and of all higher derivatives wherever those derivatives exist. \(\square\)

Worked Example: Same Maximum From Grouped and Individual Data

Suppose a common probability \(\pi\) applies to two groups: the first has \(m_1=4\) trials and \(s_1=3\) successes, and the second has \(m_2=6\) trials and \(s_2=2\) successes. The individual-sequence likelihood is

L_{\mathrm{ind}}(\pi)=\pi^{3+2}(1-\pi)^{(4-3)+(6-2)} =\pi^5(1-\pi)^5.

The grouped likelihood includes the factor

\binom{4}{3}\binom{6}{2}=4\cdot15=60, \qquad L_{\mathrm{group}}(\pi)=60\pi^5(1-\pi)^5.

For \(0<\pi<1\), the individual log-likelihood has derivative

\frac{d}{d\pi}\ell_{\mathrm{ind}}(\pi) =\frac{5}{\pi}-\frac{5}{1-\pi}.

Setting this derivative to zero gives \(5(1-\pi)=5\pi\), so \(\hat{\pi}=0.5\). The second derivative is \(-5/\pi^2-5/(1-\pi)^2\), which is negative throughout \(0<\pi<1\); therefore this stationary point is the maximum. The grouped log-likelihood adds \(\log(60)\), so it has the same derivative and the same maximum. At \(\pi=0.5\), the individual likelihood is \(0.5^{10}=1/1024\), while the grouped likelihood is \(60/1024=0.05859375\). The likelihood values differ, but the estimate does not.

When Aggregation Is Not Valid

The binomial coefficient works because every sequence with the same count has the same probability. If individuals in one reported group have different success probabilities, the probability of a sequence depends on which individuals succeeded. The count probability must then sum different terms; it generally cannot be written as a binomial coefficient times one shared probability term.

For two independent Bernoulli outcomes with probabilities \(0.2\) and \(0.8\), for example, the probability of exactly one success is

$$ (0.2)(1-0.8)+(1-0.2)(0.8)=0.04+0.64=0.68. $$

This is not the probability from a binomial model with a common probability of either \(0.2\) or \(0.8\). More importantly, replacing the two outcomes by their count discards which individual succeeded, information that may matter when the probabilities are modeled separately. Grouping is appropriate when the group members share the modeled success probability, such as when they have the same covariate values in the specified model.

Common Mistake and Practical Consequences

Mistake: “The binomial coefficient is irrelevant, so the grouped and individual likelihoods are identical.”
Correction: They are not numerically identical. For a fixed observed count, the grouped likelihood is the individual-sequence likelihood multiplied by \(\binom{m_i}{s_i}\). The factor is irrelevant to maximizing over the parameters, but it changes the likelihood value and log-likelihood by a constant.

This distinction matters when describing what probability the likelihood represents. An individual-level likelihood assigns probability to the particular observed sequence. A grouped likelihood assigns probability to the event that the count takes its observed value, combining all compatible sequences. For a fixed grouping, the combinatorial factors are the same across candidate parameter values, so they do not change likelihood maximization or likelihood ratios between those parameter values.

However, the grouped and individual likelihood values should not be compared as though they were probabilities assigned to the same recorded outcome: one concerns a particular sequence and the other concerns a set of sequences. When comparing models for the same grouped data, retain a consistent representation; the binomial coefficients are then common to the candidate models. The parameter-independent term is not permission to ignore the data representation or the common-probability assumption.

Key takeaway: With independent Bernoulli outcomes sharing a probability within each group, a count has a binomial likelihood. Its combinatorial factor counts the individual sequences represented by that count. For fixed observations, the factor does not affect parameter estimates or likelihood derivatives, but it does affect the likelihood’s numerical value.

Check Your Understanding

Use the common-probability and independence assumptions explicitly in your reasoning.

For \(m=6\) and \(s=2\), how many individual sequences correspond to the observed count? What factor relates the count probability to the probability of one such sequence?
Explain why the binomial coefficient does not change the maximum-likelihood estimate when group sizes and observed counts are fixed.
If two people in a group have different success probabilities, why is the group count not generally binomial with one shared probability?
Distinguish the event whose probability is given by an individual-sequence likelihood from the event whose probability is given by a grouped-count likelihood.
For several independent groups, write the parameter-independent factor relating the grouped likelihood to the likelihood of one compatible individual-level sequence.