Binomial Coefficients and Likelihood Comparisons

Why a Binomial Coefficient Appears

A grouped binomial observation records a number of successes, not the order in which successes and failures occurred. If $m$ trials contain $y$ successes, there are $\binom{m}{y}$ distinct ordered sequences with that count. The probability of the count therefore includes a factor that would not appear in the probability of one specified sequence.

This distinction matters when interpreting a likelihood. For estimating regression coefficients from fixed grouped data, the factor is constant with respect to those coefficients. For reporting the likelihood of the observed counts, or comparing likelihood values for different representations of the data, it may not be harmless to omit it.

Definition: Suppose $Y_1,\ldots,Y_g$ are independent grouped binomial counts, with $Y_i\sim\operatorname{Bin}(m_i,p_i(\beta))$. Here $m_i$ is the fixed number of trials in group $i$, $y_i$ is its observed success count, and $p_i(\beta)$ is the model probability determined by coefficient vector $\beta$. The likelihood for the observed counts is $$ L(\beta)=\prod_{i=1}^{g}\binom{m_i}{y_i}p_i(\beta)^{y_i}\{1-p_i(\beta)\}^{m_i-y_i}. $$

The counts satisfy $m_i\in\{0,1,2,\ldots\}$ and $y_i\in\{0,\ldots,m_i\}$. The model probabilities lie in $[0,1]$, and the trial totals and observed counts are treated as fixed when maximizing over $\beta$. A factor $\binom{m_i}{y_i}$ depends on the observed data, but not on $\beta$.

For probabilities strictly between zero and one, the log-likelihood separates into a data-only term and a parameter-dependent term:

\ell(\beta) = \underbrace{\sum_{i=1}^{g}\log\binom{m_i}{y_i}}_{C(y,m)} + \sum_{i=1}^{g} \left[ y_i\log p_i(\beta)+(m_i-y_i)\log\{1-p_i(\beta)\} \right].

Write the second sum as $\ell^*(\beta)$. Then $\ell(\beta)=C(y,m)+\ell^*(\beta)$. The binomial coefficients do not disappear from the probability model; they are simply an additive constant in the log-likelihood for this fixed dataset. At a boundary probability, a zero count contributes $0\log 0=0$ by convention, while assigning positive probability to an impossible observed outcome gives likelihood zero and log-likelihood $-\infty$.

What the Coefficients Do—and Do Not—Change

Theorem: For fixed grouped observations $(m_i,y_i)$, multiplying the likelihood by $\prod_i\binom{m_i}{y_i}$ does not change its maximizing coefficient values. It also does not change likelihood ratios between models fitted to those same observations.

Proof. Each $\binom{m_i}{y_i}$ is positive and independent of $\beta$, so $L(\beta)=K L^*(\beta)$, where $K=\prod_i\binom{m_i}{y_i}>0$ and $L^*(\beta)$ is the likelihood with those factors omitted. For any two coefficient values $\beta_a,\beta_b$, $L(\beta_a)\geq L(\beta_b)$ if and only if $L^*(\beta_a)\geq L^*(\beta_b)$, since both sides are multiplied by the same positive $K$. Thus their sets of maximizers agree, including when a maximum is attained only on a boundary or the supremum is not attained.

For two candidate models $A$ and $B$ fitted to the same counts, their log-likelihoods are $C+\ell_A^*$ and $C+\ell_B^*$. Subtracting gives $\ell_A-\ell_B=\ell_A^*-\ell_B^*$, so the likelihood ratio is also unchanged. This is why dropping the coefficients is valid for estimating coefficients or forming a likelihood ratio between models on the same grouped dataset. $\square$

Worked Example: A Single Group and Its Coefficient Estimate

Observe $y=2$ successes in $m=4$ trials, with a common success probability $p$. The full likelihood is

L(p)=\binom{4}{2}p^2(1-p)^2=6p^2(1-p)^2.

For $0<p<1$, the log-likelihood is $\ell(p)=\log 6+2\log p+2\log(1-p)$. Its derivative is

\frac{d\ell(p)}{dp}=\frac{2}{p}-\frac{2}{1-p} =\frac{2(1-2p)}{p(1-p)}.

The derivative is zero at $p=1/2$, positive when $p<1/2$, and negative when $p>1/2$. The likelihood is zero at $p=0$ and $p=1$, so its maximum occurs at $p=1/2$. Omitting the factor 6 gives $L^*(p)=p^2(1-p)^2$; the derivative of its log has the same sign and the same maximum. But the maximum likelihood values differ: $L(1/2)=6/16=3/8$, whereas $L^*(1/2)=1/16$.

Comparing Models on the Same Grouped Data

The theorem applies to any collection of candidate models that use the same observed group counts. For example, consider one model that assigns a common probability across groups and another that permits different probabilities. Their fitted probabilities can differ, but both likelihoods include the same observed-data factor $K$. The absolute log-likelihoods shift by the same amount, leaving their difference unchanged.

Worked Example: A Likelihood-Ratio Comparison

Suppose two independent groups have $(m_1,y_1)=(5,2)$ and $(m_2,y_2)=(4,1)$. Compare two specified probability patterns: Model A has $p_1=p_2=0.5$; Model B has $p_1=0.4$ and $p_2=0.25$. The common combinatorial factor is

K=\binom{5}{2}\binom{4}{1}=10\cdot4=40.

For Model A, the probability terms are $0.5^2(0.5)^3=1/32$ in the first group and $0.5(0.5)^3=1/16$ in the second. Thus

L_A=40\left(\frac{1}{32}\right)\left(\frac{1}{16}\right) =\frac{5}{64}=0.078125.

For Model B, the terms are $0.4^2(0.6)^3=0.03456$ and $0.25(0.75)^3=0.10546875$. Therefore

$$ L_B=40(0.03456)(0.10546875)=0.1458. $$

The likelihood ratio is $L_B/L_A=0.1458/0.078125=1.86624$. If the factor 40 is omitted from both likelihoods, the ratio remains $0.003645/0.001953125=1.86624$. The absolute values change, but the comparison does not.

The same cancellation applies to differences of maximized log-likelihoods when the models use the same response counts. Consequently, comparisons based on such differences are unchanged by omitting the common factor. The individual log-likelihood values, however, are not the same as those from the full binomial likelihood. If a reported criterion includes the maximized log-likelihood itself, omitting a data-only constant shifts its reported value even when the ranking of models on this fixed dataset is unaffected.

Grouped Counts and Individual Sequences Are Different Outcomes

A grouped count and a particular ordered sequence are not the same observed event. For a specified sequence of $m$ independent Bernoulli trials with success probability $p$, its probability is $p^y(1-p)^{m-y}$. The event “there are $y$ successes” consists of all $\binom{m}{y}$ sequences with that count. Because those sequences are mutually exclusive and each has the same probability, their probabilities add.

Result: For $m$ independent Bernoulli trials with common success probability $p$, the probability of observing exactly $y$ successes is $\binom{m}{y}p^y(1-p)^{m-y}$.

Proof. Let $\mathcal{S}_y$ be the set of ordered binary sequences of length $m$ containing exactly $y$ successes. Choosing the $y$ positions occupied by successes gives $|\mathcal{S}_y|=\binom{m}{y}$. Each sequence in $\mathcal{S}_y$ has probability $p^y(1-p)^{m-y}$, by independence. The sequences are disjoint events whose union is the event that the total count is $y$. Adding their probabilities gives

P(Y=y) =\sum_{s\in\mathcal{S}_y}P(s) =\binom{m}{y}p^y(1-p)^{m-y}.

This establishes why the coefficient belongs in the grouped-count likelihood: the observed event is the count, not one particular ordering. $\square$

Worked Example: The Same Trials, Two Likelihood Values

Suppose four independent trials each have success probability $p=1/2$, and the observed sequence is success, failure, success, failure. The probability of that particular ordered sequence is

\left(\frac12\right)^2\left(\frac12\right)^2=\frac{1}{16}.

If the data are recorded only as the count $y=2$, there are $\binom{4}{2}=6$ possible sequences with two successes. The probability of that grouped outcome is

\binom{4}{2}\left(\frac12\right)^2\left(\frac12\right)^2 =6\cdot\frac{1}{16}=\frac{3}{8}.

The grouped likelihood is six times the likelihood of the particular ordered record because it assigns probability to six possible sequences. Neither value is a computational mistake; each refers to a different observed event.

When Dropping the Coefficient Is a Mistake

A data-only factor is safe to omit only when the task is invariant to that factor. For fixed grouped counts, estimation and likelihood ratios across models share the same factor. But a likelihood value is a probability density or mass assigned to a specified observation under a specified representation. Changing the representation can change the event being assigned probability and the factor that relates its likelihood to another representation.

In particular, do not compare the full likelihood of grouped counts directly with the likelihood of one individual-level ordering as though they were likelihoods for the same observation. Likewise, if data are regrouped or the observed counts change, the product of binomial coefficients need not stay constant. A comparison across those representations cannot justify dropping the coefficients merely because they were constant in one fixed-data optimization.

Common mistake: “The binomial coefficient does not involve the regression coefficients, so it can always be discarded.” The first clause is correct for fixed counts, but the conclusion is too broad. The factor can be discarded when maximizing over coefficients or comparing models on the same grouped observations; it cannot be discarded when the goal is the full likelihood of those counts or when comparing values tied to different data representations.

Key takeaway: Keep the grouped binomial coefficient when the likelihood must represent the probability of the observed counts. You may omit it for coefficient estimation or same-data likelihood-ratio comparisons because it is a common positive factor, not because it is absent from the probability model.

Check Your Understanding

Use the distinction between a count and an ordered sequence to answer these questions.

For fixed $m$ and $y$, explain why $\binom{m}{y}$ does not change the maximizing value of a model coefficient.
Two models are fitted to the same grouped counts. Why is their likelihood ratio unchanged if both likelihoods omit the binomial coefficients?
For five trials and two successes, how many ordered sequences correspond to the grouped count? What factor relates the count probability to the probability of one specified sequence?
Why is it generally inappropriate to compare the likelihood of a grouped count with the likelihood of one ordered individual-level record as if they represented the same observation?
Can the absolute maximized log-likelihood change when the binomial coefficients are omitted, even if the coefficient estimates do not? Explain.