Maximum Likelihood Estimation for Binomial Counts

Why Binomial Counts Can Be Combined

A data set may report the number of successes from several groups, time periods, or batches, with a different number of trials in each. If every trial has the same success probability and the counts are independent, these observations provide evidence about one common parameter. The maximum likelihood estimate weights each observed proportion by its number of trials; it is not generally the unweighted average of the proportions.

The model must be justified before pooling. In particular, a shared probability is an assumption about the trial outcomes, not a consequence of having counts in the same table. If the groups have different success probabilities, this tutorial's one-parameter likelihood is not the appropriate model.

Definition: Let \(X_1,\ldots,X_k\) be independent counts, with \(X_i\sim\operatorname{Binomial}(n_i,p)\). The trial sizes \(n_i\) are known positive integers, and the unknown success probability \(p\) is common to all groups, with \(0\leq p\leq 1\). Each count records the number of successes in \(n_i\) Bernoulli trials. The model assumes the trials within each group have success probability \(p\), and that the group counts are independent.

Write \(x_i\) for the observed value of \(X_i\). Then \(x_i\) is an integer from 0 to \(n_i\). Define the total number of successes and the total number of trials by \(x_{\text{tot}}=\sum_{i=1}^k x_i\) and \(n_{\text{tot}}=\sum_{i=1}^k n_i\). These totals will determine the estimate when the common-probability model applies.

Constructing the Likelihood

For one binomial count, the probability of observing \(x_i\) successes in \(n_i\) trials is \(\binom{n_i}{x_i}p^{x_i}(1-p)^{n_i-x_i}\). Because the observed counts are independent, their joint probability is the product of these probabilities. Viewed as a function of \(p\), with the observed counts fixed, that joint probability is the likelihood.

L(p) =\prod_{i=1}^{k}\binom{n_i}{x_i}p^{x_i}(1-p)^{n_i-x_i} =\left(\prod_{i=1}^{k}\binom{n_i}{x_i}\right) p^{x_{\text{tot}}}(1-p)^{n_{\text{tot}}-x_{\text{tot}}}.

The binomial coefficients depend on the observed data, but not on \(p\). They are part of the likelihood as written; for maximizing over \(p\), they can be treated as a positive constant. The exponents combine because multiplying powers with the same base adds their exponents. Thus the likelihood's dependence on \(p\) is governed by the total successes and total failures, where the total failures are \(n_{\text{tot}}-x_{\text{tot}}\).

Theorem: Under the independent binomial model with a common success probability \(p\) and known trial sizes, a maximum likelihood estimator is \(\widehat p=x_{\text{tot}}/n_{\text{tot}}\). If all observed trials are failures, this gives the boundary estimate \(\widehat p=0\); if all are successes, it gives \(\widehat p=1\).

Deriving and Checking the Estimate

When there is at least one success and at least one failure, the maximum is in the interior \(0<p<1\). Taking the logarithm simplifies the product likelihood to a sum. The logarithm is strictly increasing, so maximizing the log-likelihood gives the same value of \(p\) as maximizing the likelihood.

Collect the terms that depend on \(p\).
Taking the logarithm of the likelihood gives a constant term from the binomial coefficients, plus \(x_{\text{tot}}\log p+(n_{\text{tot}}-x_{\text{tot}})\log(1-p)\).

Differentiate and set the score to zero.
The derivative is \(x_{\text{tot}}/p-(n_{\text{tot}}-x_{\text{tot}})/(1-p)\). Equating it to zero and multiplying by \(p(1-p)\) gives \(x_{\text{tot}}(1-p)=(n_{\text{tot}}-x_{\text{tot}})p\).

Solve for the candidate.
Expanding the equation gives \(x_{\text{tot}}=n_{\text{tot}}p\), so \(p=x_{\text{tot}}/n_{\text{tot}}\).

Verify it is a maximum.
The second derivative is \(-x_{\text{tot}}/p^2-(n_{\text{tot}}-x_{\text{tot}})/(1-p)^2\), which is negative throughout \(0<p<1\) when both totals are positive. The log-likelihood is therefore strictly concave there, and its stationary point is the unique interior maximum.

This derivation also explains why the trial sizes matter: the denominator is the number of trials across all groups, not the number of groups. For boundary data, the interior score equation does not have a solution. The likelihood itself settles the issue. If \(x_{\text{tot}}=0\), then \(L(p)\) is proportional to \((1-p)^{n_{\text{tot}}}\), which is maximized at \(p=0\). If \(x_{\text{tot}}=n_{\text{tot}}\), it is proportional to \(p^{n_{\text{tot}}}\), which is maximized at \(p=1\).

Worked Example: Combining Counts With Unequal Trial Sizes

A hypothetical equipment study records successful starts in three independent batches. The batch sizes and success counts are \((n_1,x_1)=(12,9)\), \((n_2,x_2)=(20,11)\), and \((n_3,x_3)=(8,6)\). Assume every start has the same success probability \(p\). Find the MLE and check that it is a maximum.

Method and totals. Use the common-probability binomial likelihood and pool the successes and trials: \(x_{\text{tot}}=9+11+6=26\), while \(n_{\text{tot}}=12+20+8=40\). The total failures are \(40-26=14\).

Likelihood. The three binomial coefficients are \(\binom{12}{9}=220\), \(\binom{20}{11}=167{,}960\), and \(\binom{8}{6}=28\). Their product is \(220(167{,}960)(28)=1{,}034{,}633{,}600\). Therefore \(L(p)=1{,}034{,}633{,}600p^{26}(1-p)^{14}\).

Estimate. The score equation is \(26/p-14/(1-p)=0\). Multiplying by \(p(1-p)\) gives \(26(1-p)=14p\), so \(26=40p\), and \(\widehat p=26/40=0.65\).

Maximum check. At \(p=0.65\), the score is \(26/0.65-14/0.35=40-40=0\). The second derivative of the log-likelihood there is \(-26/(0.65)^2-14/(0.35)^2\), which is negative. Thus the candidate is the unique maximum. The estimate is 0.65, or 65% when expressed as a percentage.

Common mistake avoided. The method uses the total 26 successes out of 40 starts. It does not give each batch equal weight merely because there are three batches.

Why the Weighted Proportion Matters

Each observed proportion \(x_i/n_i\) summarizes one batch, but those proportions need not be equally informative: they are based on different numbers of trials. The pooled proportion can be written as a weighted average, \(\sum_i (n_i/n_{\text{tot}})(x_i/n_i)\). The weights add to 1 and are proportional to the batch sizes. This is exactly the total successes divided by the total trials.

Worked Example: Avoiding an Unweighted Average

A hypothetical screening process is assessed in three independent groups, under the assumption of a common probability of a positive result. In groups of 15, 25, and 10 trials, the observed positive counts are 3, 14, and 8. Find the MLE and show why averaging the three group proportions equally gives a different answer.

Method and pooled estimate. The total number of positive results is \(3+14+8=25\). The total number of trials is \(15+25+10=50\), so the common-probability MLE is \(\widehat p=25/50=0.50\).

Likelihood check. The likelihood is \(\binom{15}{3}\binom{25}{14}\binom{10}{8}p^{25}(1-p)^{25}\). Its score is \(25/p-25/(1-p)\), which equals zero at \(p=0.50\). The second derivative is negative for \(0<p<1\), so this is the unique maximum.

Compare the unweighted calculation. The group proportions are \(3/15=0.20\), \(14/25=0.56\), and \(8/10=0.80\). Their unweighted average is \((0.20+0.56+0.80)/3=1.56/3=0.52\), not 0.50. That calculation assigns the same weight to a group of 10 trials as to a group of 25; the likelihood instead reflects how many trials produced each count.

Common mistake avoided. Averaging group proportions without regard to their denominators is not the MLE for a shared binomial probability when the trial sizes differ.

Boundary Data and the Shared-Probability Assumption

The formula \(\widehat p=x_{\text{tot}}/n_{\text{tot}}\) includes the two boundary cases. For example, if every observed trial is a failure, then \(x_{\text{tot}}=0\) and the likelihood is largest at \(p=0\). If every observed trial is a success, then \(x_{\text{tot}}=n_{\text{tot}}\) and the likelihood is largest at \(p=1\). These are valid estimates in the parameter space \([0,1]\), even though the interior score equation cannot be used to obtain them.

The common-\(p\) assumption is essential. If each group has its own unknown probability, the observations are not described by the one-parameter likelihood above, and pooling into one estimate would answer a different question. In an application, consider whether the groups differ in a way that could change the success probability before treating their counts as evidence about the same \(p\).

Worked Example: All Observed Trials Are Successes

A hypothetical reliability check records 7 successes in 7 trials in one batch and 11 successes in 11 trials in another independent batch. Assume both batches share the same success probability \(p\). Find the MLE, and explain why solving an interior score equation is not the right method.

Method and totals. There are \(7+11=18\) successes among \(7+11=18\) trials, so the proposed estimate from the pooled proportion is \(18/18=1\).

Likelihood and boundary check. Each binomial coefficient is 1, so \(L(p)=p^7p^{11}=p^{18}\) for \(0\leq p\leq1\). Since \(p^{18}\) increases as \(p\) increases on this interval, its maximum occurs at \(p=1\). Thus \(\widehat p=1\).

Why the interior equation does not apply. The log-likelihood is \(18\log p\) for \(0<p\leq1\), with derivative \(18/p\), which is positive and never zero in the interior. There is no interior stationary point; the maximum is at the allowed endpoint.

Common mistake avoided. Do not reject a boundary estimate just because the score equation has no interior root. Check the likelihood at the parameter-space boundary.

Exam Traps and Practical Checks

The estimator is simple, but a short list of checks prevents common errors:

Keep counts and trial sizes distinct. The numerator is total successes; the denominator is total trials. A count is not a proportion until it is divided by its own trial size.
Use the common-probability model only when warranted. Independence of the counts does not by itself imply that their success probabilities are equal.
Do not average group proportions equally when trial sizes differ. The likelihood weights each group's proportion according to its trial size.
Check the endpoints. If there are no observed successes or no observed failures, the MLE is at a boundary and should be verified from the likelihood.
Keep the data-only factors in perspective. Binomial coefficients belong in the likelihood, but do not affect which value of \(p\) maximizes it.

This method is useful whenever several independent binomial counts share one probability parameter, including hypothetical studies that combine batches or periods under a defensible common-rate assumption. It reduces the observations to two totals while preserving the sample-size weighting. The result is aligned with the SOA VEE Mathematical Statistics syllabus.

Check Your Understanding

For each problem, state the pooled estimate under the stated common-probability model. Check both the total successes and the total trials.

A hypothetical device is tested in two independent batches of 18 and 42 trials, with 12 and 21 successes. Find the MLE of the shared success probability. (Answer: \(33/60=0.55\))
Three independent groups have trial sizes 10, 15, and 25, with observed success counts 0, 6, and 14. Find the MLE and identify whether the estimate is interior or at a boundary. (Answer: \(20/50=0.40\), an interior estimate)
A study has groups of 5 and 45 trials, with 4 and 27 successes. Find the pooled MLE and the unweighted average of the two group proportions. (Answer: pooled MLE \(31/50=0.62\); unweighted average \((0.80+0.60)/2=0.70\))
Two independent checks produce 0 successes in 12 trials and 0 successes in 8 trials. Under a shared-probability model, find the MLE and state why the interior score equation does not give it. (Answer: \(\widehat p=0\); the likelihood is \((1-p)^{20}\), maximized at the boundary \(p=0\))
Two batches have sizes 16 and 24 and success counts 8 and 18. Find the MLE and verify that the score is zero at your answer. (Answer: \(\widehat p=26/40=0.65\); the score is \(26/0.65-14/0.35=40-40=0\))

What You'll Learn