How Trial Counts Determine Binomial Information

Why the Number of Trials Matters

A binomial response records how many successes occurred in a specified number of trials. Two groups can have the same estimated success probability but very different amounts of data behind that estimate: 8 successes out of 10 trials and 80 out of 100 trials are not equally informative. In a generalized linear model, this difference appears directly in the score and Fisher information.

This tutorial derives those quantities for a binomial observation with a logit mean. The key conclusion is precise: holding the linear predictor and covariates fixed, the information contributed by an observation is proportional to its trial count. For regression, that information is a matrix, so the design determines which coefficient combinations are well informed.

Binomial Observations With a Logit Mean

Definition: For observation $i$, let $Y_i$ be a binomial count from $m_i$ trials, with success probability $p_i$. A binomial-logit regression model specifies $$ Y_i\sim\operatorname{Binomial}(m_i,p_i),\qquad \operatorname{logit}(p_i)=\log\left(\frac{p_i}{1-p_i}\right)=\eta_i=x_i^\mathsf{T}\beta, $$ where $x_i$ is a fixed covariate vector and $\beta$ is the coefficient vector. Thus $p_i=\exp(\eta_i)/(1+\exp(\eta_i))$. The observations are assumed independent, and the trial counts and covariates are fixed.

For finite $\eta_i$, the probability satisfies $0<p_i<1$. The binomial likelihood contribution, including its combinatorial factor, is

L_i(\beta)=\binom{m_i}{y_i}p_i^{y_i}(1-p_i)^{m_i-y_i}.

The combinatorial factor does not depend on $\beta$. It matters to the full probability model, but it contributes zero to the score and to derivatives of the log-likelihood with respect to $\beta$.

Worked Example: A Single Group at a Specified Coefficient

Suppose $Y\sim\operatorname{Binomial}(20,p)$, with an intercept-only logit model $\operatorname{logit}(p)=\beta$. The observed count is $y=12$. Calculate the score and Fisher information at $\beta=0$.

At $\beta=0$, $p=\exp(0)/(1+\exp(0))=1/2$. The log-likelihood is

\ell(\beta)=\log\binom{20}{12}+12\beta-20\log(1+\exp(\beta)).

Differentiation gives $U(\beta)=d\ell/d\beta=y-mp$. Therefore the score at zero is $12-20(1/2)=2$. The Fisher information is $mp(1-p)=20(1/2)(1/2)=5$. The score is calculated from the observed count; the information is the expected curvature at the specified coefficient.

Score and Information: Derivation

The logistic function has derivative $dp_i/d\eta_i=p_i(1-p_i)$, and $d\eta_i/d\beta=x_i$. The log-likelihood contribution can therefore be written in a form that makes its derivatives transparent:

\ell_i(\beta)=\log\binom{m_i}{y_i}+y_i\eta_i-m_i\log(1+\exp(\eta_i)).

Differentiating with respect to the coefficient vector gives the score contribution $U_i(\beta)$. Differentiating again gives the negative Hessian, also called the observed information contribution:

U_i(\beta)=\frac{\partial\ell_i}{\partial\beta} =x_i(y_i-m_ip_i),\qquad J_i(\beta)=-\frac{\partial^2\ell_i}{\partial\beta\,\partial\beta^\mathsf{T}} =m_ip_i(1-p_i)x_ix_i^\mathsf{T}.

Theorem: Suppose $Y_i\sim\operatorname{Binomial}(m_i,p_i)$, with finite $m_i$, fixed $x_i$, finite $\beta$, and $\operatorname{logit}(p_i)=x_i^\mathsf{T}\beta$. Then the score has mean zero, and both the expected Fisher information and the score covariance are $$ I_i(\beta)=E[J_i(\beta)] =\operatorname{Var}(U_i(\beta)) =m_ip_i(1-p_i)x_ix_i^\mathsf{T}. $$ The observed information $J_i(\beta)$ equals this expression for every possible observed count $y_i$.

Proof. The log-likelihood expression above follows by substituting $p_i=\exp(\eta_i)/(1+\exp(\eta_i))$ and $1-p_i=1/(1+\exp(\eta_i))$ into the binomial probability. Since $d\eta_i/d\beta=x_i$, its first derivative is $x_i(y_i-m_ip_i)$. Differentiating $p_i$ with respect to $\beta$ gives $p_i(1-p_i)x_i$. Applying the product and chain rules to the score therefore gives the stated $J_i(\beta)$. This expression has no $y_i$ in it, so taking its expectation leaves it unchanged.

Under the binomial model, $E[Y_i]=m_ip_i$ and $\operatorname{Var}(Y_i)=m_ip_i(1-p_i)$. Consequently,

$$ E[U_i]=x_i(E[Y_i]-m_ip_i)=0, $$

and, because $x_i$ is fixed,

\operatorname{Var}(U_i) =x_ix_i^\mathsf{T}\operatorname{Var}(Y_i) =m_ip_i(1-p_i)x_ix_i^\mathsf{T}.

These calculations establish the asserted equality. All expectations are finite sums under the finite binomial support, so no interchange of a limit and an expectation is required. $\square$

For independent observations, log-likelihoods and scores add. Hence the full score and Fisher information are

U(\beta)=\sum_i x_i(y_i-m_ip_i),\qquad I(\beta)=\sum_i m_ip_i(1-p_i)x_ix_i^\mathsf{T}.

The information matrix is positive semidefinite: for any vector $a$, $a^\mathsf{T}I(\beta)a=\sum_i m_ip_i(1-p_i)(a^\mathsf{T}x_i)^2\geq 0$. It is positive definite only when the covariate vectors provide information in every coefficient direction; a large trial count cannot repair a rank-deficient design.

Trial Counts Scale Information

Theorem: For a binomial-logit observation with fixed covariate vector $x$ and fixed coefficient value $\beta$, multiplying the trial count $m$ by a positive constant $c$ multiplies its score by $c$ for the same success proportion $y/m$, and multiplies its Fisher information by exactly $c$. In particular, $$ I(\beta)=mp(1-p)xx^\mathsf{T}. $$

Proof. The information formula has $m$ as a factor, while $p$ is determined by the fixed $x$ and $\beta$. Replacing $m$ by $cm$ therefore replaces $I$ by $cI$. For the score, if the success proportion $y/m$ is held fixed, then replacing both $y$ and $m$ by $cy$ and $cm$ gives $x(cy-cmp)=c\,x(y-mp)$. The information scaling does not require a particular observed count, because the information formula does not depend on that count. $\square$

This is an exact statement about the model's information at a specified coefficient and design. It does not say that every estimate in every sample has an error that shrinks by a fixed factor. When the full information matrix is invertible, its inverse describes the usual local covariance approximation for a regular maximum-likelihood estimate; multiplying all trial counts by $c$, with the probabilities and design held fixed, divides that approximation by $c$. The approximation and its interpretation depend on the model and estimation conditions, not on the scaling identity alone.

Worked Example: Information in a Two-Group Regression

Consider a logistic regression with an intercept and a binary group indicator, $x_1=(1,0)^\mathsf{T}$ and $x_2=(1,1)^\mathsf{T}$. The first group has $m_1=10$ trials and $y_1=2$; the second has $m_2=20$ and $y_2=12$. Evaluate the score and information at $\beta_0=\log(0.25)$ and $\beta_1=\log(4)$.

For group 1, $\eta_1=\log(0.25)$, so $p_1=0.2$. Its residual count is $y_1-m_1p_1=2-10(0.2)=0$, giving score contribution $(0,0)^\mathsf{T}$. Its information contribution is

10(0.2)(0.8) \begin{pmatrix}1&0\\0&0\end{pmatrix} = \begin{pmatrix}1.6&0\\0&0\end{pmatrix}.

For group 2, $\eta_2=\log(0.25)+\log(4)=0$, so $p_2=0.5$. Its residual is $12-20(0.5)=2$, giving score contribution $2(1,1)^\mathsf{T}=(2,2)^\mathsf{T}$. Its information contribution is $20(0.5)(0.5)\begin{pmatrix}1&1\\1&1\end{pmatrix}$, or $\begin{pmatrix}5&5\\5&5\end{pmatrix}$. Adding the contributions gives

U(\beta)=\begin{pmatrix}2\\2\end{pmatrix},\qquad I(\beta)=\begin{pmatrix}6.6&5\\5&5\end{pmatrix}.

The determinant is $6.6(5)-25=8$, so this information matrix is positive definite. Both groups contribute, and their distinct covariate vectors allow the intercept and group effect to be distinguished.

Worked Example: Quadrupling the Trials

Suppose an intercept-only model has $p=0.3$. Compare the information from $m=10$ trials with that from $m=40$ trials.

With an intercept, $x=1$, so the information is $mp(1-p)=m(0.3)(0.7)=0.21m$. Thus the two information values are $2.1$ and $8.4$, respectively. The second is exactly four times the first. Their inverse-information values are $1/2.1\approx0.4762$ and $1/8.4\approx0.1190$, with the latter one quarter as large. These inverse values illustrate the change in local variance scale when the model conditions for that approximation hold; they are not a claim that every realized estimate will have exactly that variance.

Worked Example: A Covariate Changes the Information Direction

For one group, let $m=12$, $p=0.5$, and $x=(1,2)^\mathsf{T}$. Compute the information matrix and identify the coefficient direction informed by this group.

The scalar multiplier is $mp(1-p)=12(0.5)(0.5)=3$. Since

xx^\mathsf{T}= \begin{pmatrix}1&2\\2&4\end{pmatrix}, \qquad I=3xx^\mathsf{T}= \begin{pmatrix}3&6\\6&12\end{pmatrix},

the matrix has determinant $36-36=0$ and rank one. For a coefficient change $a=(a_0,a_1)^\mathsf{T}$, its information contribution is $3(a_0+2a_1)^2$. Thus this observation informs the predictor combination $a_0+2a_1$, but alone cannot distinguish arbitrary changes in the intercept from changes in the slope that leave this combination unchanged. Increasing $m$ multiplies the matrix but does not change its rank.

Interpretation and a Common Pitfall

The factor $p_i(1-p_i)$ also matters. At fixed trial count, it is largest at $p_i=1/2$ and becomes small as $p_i$ approaches zero or one. Therefore a group with many trials can still contribute little local information when its modeled probability is near a boundary. The factor $x_ix_i^\mathsf{T}$ determines how that information is distributed across coefficient directions.

Common mistake: “A group has four times as many trials, so every coefficient estimate must have exactly one-quarter the standard error.” The correction is that the group contributes four times the Fisher information only when the model probability and covariate are held fixed. Overall coefficient uncertainty depends on the sum of information across all observations, the design's rank and structure, and the estimation conditions. A larger count adds information in the directions represented by its covariates; it cannot by itself guarantee a particular standard error or resolve confounding.

The trial count here is part of the binomial sampling model: it specifies how many Bernoulli trials produced the observed count. The resulting factor of $m_i$ follows from the binomial likelihood and its variance. That specific derivation is why trial counts have a clear information interpretation in this model.

Check Your Understanding

Use the formulas and conditions developed above to answer these questions.

For $Y\sim\operatorname{Binomial}(m,p)$ with an intercept-only logit model, what are the score and Fisher information?
Why does the observed information for a binomial-logit observation not depend on the observed success count?
If the trial count is doubled while the covariate and coefficient remain fixed, what happens to the information contribution?
Why can increasing the trial count fail to make a multi-coefficient information matrix invertible?
At what success probability is $p(1-p)$ largest, and what does this imply about information at fixed $m$ and $x$?