Binomial Trial Counts in the Likelihood

From Success Probabilities to Success Counts

In a binomial regression, an observation can summarize several trials that share a success probability. For observation $i$, let $m_i$ be the number of trials and $Y_i$ the number of successes. The regression model describes the probability of success on each trial; the expected number of successes is then determined by both that probability and the number of trials. Keeping those two quantities distinct is essential when constructing the likelihood.

Assume the observations are independent conditional on their covariates and trial counts. For each $i=1,\ldots,n$, take $m_i$ to be a fixed positive integer, and suppose $Y_i$ is the success count from $m_i$ Bernoulli trials with a common probability $p_i$. Thus $Y_i$ has a binomial distribution. A GLM specifies $p_i$ through a linear predictor, for example through the logit link $ \operatorname{logit}(p_i)=\log\{p_i/(1-p_i)\}=x_i^\mathsf{T}\beta$. The count mean is $m_i p_i$, not $p_i$.

Definition: For observation $i$, $m_i$ is the number of binomial trials, $Y_i$ is the observed number of successes, and $p_i$ is the probability of success on any one of those trials. The response model is $Y_i\sim\operatorname{Binomial}(m_i,p_i)$, with $0\leq Y_i\leq m_i$. Its expected success count is $E[Y_i]=m_i p_i$.

The trial count is part of the response distribution. It scales the mean count while the regression model determines the success probability. In particular, if $g$ is the chosen link, the model is $g(p_i)=x_i^\mathsf{T}\beta$; it is not generally $g(m_i p_i)=x_i^\mathsf{T}\beta$. The latter would put a different quantity on the link scale.

The Likelihood for Observation-Specific Trial Counts

For an observed success count $y_i$, the binomial probability mass is the probability of obtaining that count in $m_i$ trials. Under conditional independence across observations, the joint likelihood is the product of these probabilities. The binomial coefficient is included in the full likelihood, even though it does not depend on the regression coefficients when $m_i$ and $y_i$ are fixed.

Definition: Given fixed trial counts $m_i$, observed counts $y_i$, and success probabilities $p_i$, the likelihood is $$ L(\beta)=\prod_{i=1}^n {m_i\choose y_i}p_i(\beta)^{y_i}\{1-p_i(\beta)\}^{m_i-y_i}, $$ where $0\leq y_i\leq m_i$ and $p_i(\beta)$ is specified by the regression model. Its log-likelihood is $$ \ell(\beta)=\sum_{i=1}^n\left[\log {m_i\choose y_i}+y_i\log p_i(\beta)+(m_i-y_i)\log\{1-p_i(\beta)\}\right]. $$

When the model uses a link $g$, the probability is $p_i=g^{-1}(x_i^\mathsf{T}\beta)$, where the inverse is defined on the relevant range of the link. For the logit link, $p_i=\exp(x_i^\mathsf{T}\beta)/\{1+\exp(x_i^\mathsf{T}\beta)\}$. The count $m_i$ appears in the likelihood through the number of failures $m_i-y_i$ as well as through the range of possible counts. It does not replace the modeled probability.

Worked Example: Two Observations With Different Trial Counts

Suppose two independent observations have trial counts $m_1=4$, $m_2=2$, observed success counts $y_1=3$, $y_2=0$, and specified probabilities $p_1=0.6$, $p_2=0.25$. Construct and evaluate the likelihood.

For the first observation, the probability of three successes in four trials is ${4\choose3}(0.6)^3(0.4)^1=4(0.216)(0.4)=0.3456$. For the second, the probability of zero successes in two trials is ${2\choose0}(0.25)^0(0.75)^2=1(1)(0.5625)=0.5625$.

Independence gives the product $$ L=(0.3456)(0.5625)=0.1944. $$ Equivalently, the likelihood is $4(0.6)^3(0.4)(0.75)^2=0.1944$. The two observations use their own trial counts and probabilities; the likelihood is not calculated by treating each observed count as a single Bernoulli response.

How Trial Counts Enter the Mean

For a binomial count, increasing the number of trials while holding the success probability fixed increases the expected count in direct proportion. This statement follows from the representation of a binomial count as a sum of Bernoulli indicators.

Result: If $Y_i=\sum_{j=1}^{m_i}B_{ij}$, where the $B_{ij}$ are independent Bernoulli random variables with $P(B_{ij}=1)=p_i$, then $E[Y_i]=m_i p_i$ and $\operatorname{Var}(Y_i)=m_i p_i(1-p_i)$.

To verify the result, linearity of expectation gives $E[Y_i]=\sum_{j=1}^{m_i}E[B_{ij}]=\sum_{j=1}^{m_i}p_i=m_i p_i$. For the variance, independence makes all cross-covariances zero, so $\operatorname{Var}(Y_i)=\sum_{j=1}^{m_i}\operatorname{Var}(B_{ij})$. Each indicator has variance $p_i(1-p_i)$, hence $\operatorname{Var}(Y_i)=m_i p_i(1-p_i)$. This derivation shows why a probability and a count mean are not interchangeable: they differ by the number of trials. $\square$

Worked Example: Applying a Logit Model to Counts

Consider a model with $\operatorname{logit}(p_i)=\beta_0+\beta_1 x_i$, where $\beta_0=0$ and $\beta_1=\log 3$. There are two observations: observation 1 has $x_1=0$, $m_1=2$, $y_1=1$; observation 2 has $x_2=1$, $m_2=4$, $y_2=3$. Find the fitted probabilities, expected counts, and likelihood.

For observation 1, the linear predictor is $0+(\log 3)0=0$, so $p_1=1/2$. Its expected count is $m_1p_1=2(1/2)=1$. For observation 2, the predictor is $\log 3$, so its odds are $3$, giving $p_2=3/(1+3)=3/4$. Its expected count is $m_2p_2=4(3/4)=3$.

The likelihood contribution from observation 1 is ${2\choose1}(1/2)(1/2)=1/2$. The contribution from observation 2 is ${4\choose3}(3/4)^3(1/4)=4(27/64)(1/4)=27/64$. Therefore, $$ L=(1/2)(27/64)=27/128=0.2109375. $$ The linear predictor determines each probability, while the corresponding trial count converts that probability into an expected success count and specifies the binomial likelihood contribution.

The Score for Regression Coefficients

The score describes how the log-likelihood changes as the regression coefficients change. For a differentiable link, it can be obtained by differentiating the binomial log-likelihood through $p_i(\beta)$. The following derivation assumes $0<p_i<1$, that $g$ is differentiable with $g'(p_i)\neq0$, and that $p_i$ depends on $\beta$ through $g(p_i)=x_i^\mathsf{T}\beta$.

Result: Under those conditions, the score vector is $$ \frac{\partial\ell(\beta)}{\partial\beta} =\sum_{i=1}^n x_i\frac{y_i-m_i p_i}{p_i(1-p_i)g'(p_i)}. $$ For the logit link, $g'(p)=1/\{p(1-p)\}$, so this simplifies to $$ \frac{\partial\ell(\beta)}{\partial\beta} =\sum_{i=1}^n x_i(y_i-m_i p_i). $$

To derive the general expression, first differentiate the $i$th log-likelihood contribution with respect to $p_i$: $$ \frac{\partial\ell_i}{\partial p_i} =\frac{y_i}{p_i}-\frac{m_i-y_i}{1-p_i} =\frac{y_i-m_i p_i}{p_i(1-p_i)}. $$ Differentiating $g(p_i)=x_i^\mathsf{T}\beta$ with respect to the coefficient vector gives $g'(p_i)\,\partial p_i/\partial\beta=x_i$, and therefore $\partial p_i/\partial\beta=x_i/g'(p_i)$. Applying the chain rule and summing over observations yields the stated score. For the logit link, substituting $g'(p_i)=1/\{p_i(1-p_i)\}$ cancels the denominator, giving $\sum_i x_i(y_i-m_i p_i)$. $\square$

The factor $m_i$ is explicit in the logit score through the fitted count $m_i p_i$. Thus a residual for this likelihood is naturally the observed count minus the expected count, $y_i-m_i p_i$, rather than the observed proportion minus the fitted probability by itself.

Worked Example: Checking the Intercept Score

Suppose an intercept-only logit model is evaluated at $\beta_0=0$, so $p=1/2$. The data are $(m_1,y_1)=(5,4)$ and $(m_2,y_2)=(3,1)$. Compute the score and identify the direction in which the likelihood initially increases.

For an intercept-only model, each $x_i=1$, so the score is $\sum_i(y_i-m_i p)$. The first contribution is $4-5(1/2)=1.5$, and the second is $1-3(1/2)=-0.5$. The total score is $1.5-0.5=1$.

Because the score at $\beta_0=0$ is positive, a sufficiently small increase in the intercept increases the log-likelihood locally. The total observed count is $4+1=5$, whereas the total fitted count at this intercept is $(5+3)(1/2)=4$; their difference is the same score, $5-4=1$. This check uses the trials in both fitted counts.

Common Mistake: Treating the Success Proportion as the Count

A frequent error is to put the observed proportion $y_i/m_i$ into the likelihood as though it were a single Bernoulli outcome, or to model the expected count $m_i p_i$ directly with a logit link. Both moves change the response model. In a binomial GLM, the link applies to $p_i$, and the random response is the count $Y_i$ with $m_i$ trials.

For example, if $y_i=3$ successes are observed in $m_i=4$ trials, $y_i/m_i=0.75$ is the observed proportion, while $Y_i$ is the random count and has mean $4p_i$. Modeling $p_i$ as $0.75$ in a particular data set does not make the observation a single Bernoulli trial. The likelihood contribution still describes the probability of the count among four trials.

Key takeaway: Specify the probability model first: $g(p_i)=x_i^\mathsf{T}\beta$. Then use $Y_i\sim\operatorname{Binomial}(m_i,p_i)$, with mean $m_i p_i$, and include the observed count and trial count in the likelihood.

Check Your Understanding

Use the binomial count model and distinguish its probability and count scales.

If $m_i=10$ and $p_i=0.3$, what is the expected success count? What is the expected success proportion?
Write the likelihood contribution for an observation with $m_i=5$, $y_i=2$, and success probability $p_i$.
In a logit model, does the linear predictor describe $p_i$ or $m_i p_i$? State the corresponding expected count.
For an intercept-only logit model, $m_1=4,y_1=3$ and $m_2=2,y_2=0$, what is the score at $\beta_0=0$?
Explain why replacing a binomial count $Y_i$ by a single Bernoulli response changes the likelihood model.