Information Contributions of Binomial Groups

Why Information Depends on the Group

A grouped-binomial record reports a number of successes out of a known number of trials. In logistic regression, that record contributes information about the regression coefficients through two features: how many trials it represents and the covariate vector at which its success probability is modeled. Its fitted probability matters too. The derivation makes these roles explicit and helps distinguish a group’s trial total from a generic replication weight.

We consider fixed covariates and independent groups. The information below is a matrix because the regression coefficient usually has several components. A group can inform some coefficient directions strongly and contribute nothing to others.

Definition: For group $i$, let $Y_i$ be the number of successes among $m_i$ trials, where $m_i$ is a fixed positive integer. Let $x_i$ be a fixed $q$-dimensional covariate vector, and let $\boldsymbol{\beta}\in\mathbb{R}^q$ be the coefficient vector. The grouped-logistic model is $$ Y_i\mid x_i \sim \operatorname{Binomial}(m_i,p_i), \qquad \operatorname{logit}(p_i)=x_i^\mathsf{T}\boldsymbol{\beta}, $$ where $p_i=(1+\exp(-x_i^\mathsf{T}\boldsymbol{\beta}))^{-1}$. For finite $\boldsymbol{\beta}$ and finite $x_i$, $0<p_i<1$.

The vector $x_i$ includes an intercept component when the model has an intercept. The information contribution is a $q\times q$ matrix. We use the Fisher information definition $I_i(\boldsymbol{\beta})=E_{\boldsymbol{\beta}}[U_iU_i^\mathsf{T}]$, where $U_i$ is the score vector for this group. When the regularity conditions for the likelihood hold, it also equals the negative expected Hessian of the group log-likelihood. Here the support $Y_i\in\{0,\ldots,m_i\}$ is fixed, and the log-likelihood is smooth in $\boldsymbol{\beta}$, so both expressions are well-defined.

Deriving One Group’s Information

For a realized count $y_i$, the group likelihood includes the binomial coefficient. That term matters for the probability model, but it does not depend on $\boldsymbol{\beta}$.

L_i(\boldsymbol{\beta}) = \binom{m_i}{y_i}p_i^{y_i}(1-p_i)^{m_i-y_i}, \qquad \ell_i(\boldsymbol{\beta}) = \log\binom{m_i}{y_i} +y_i\log(p_i)+(m_i-y_i)\log(1-p_i).

Write $\eta_i=x_i^\mathsf{T}\boldsymbol{\beta}$. The logistic function satisfies $\partial p_i/\partial\eta_i=p_i(1-p_i)$. Differentiating the log-likelihood with respect to $\eta_i$ gives $y_i(1-p_i)-(m_i-y_i)p_i=y_i-m_ip_i$. The chain rule then gives the coefficient score.

Theorem: Under the grouped-logistic model, the score, Hessian, and Fisher information contribution of group $i$ are $$ U_i(\boldsymbol{\beta})=x_i(y_i-m_ip_i), \qquad \frac{\partial^2\ell_i}{\partial\boldsymbol{\beta}\,\partial\boldsymbol{\beta}^\mathsf{T}} =-m_ip_i(1-p_i)x_ix_i^\mathsf{T}, \qquad I_i(\boldsymbol{\beta})=m_ip_i(1-p_i)x_ix_i^\mathsf{T}. $$

Proof. Since $\partial\eta_i/\partial\boldsymbol{\beta}=x_i$, the derivative of $\ell_i$ is $x_i(y_i-m_ip_i)$. Differentiating again uses $\partial p_i/\partial\boldsymbol{\beta}=p_i(1-p_i)x_i$, giving the displayed Hessian. It does not depend on the observed count, so its negative is also its negative expectation.

For the score-covariance definition, $E[Y_i]=m_ip_i$ and $\operatorname{Var}(Y_i)=m_ip_i(1-p_i)$ for a binomial random variable. Thus $E[U_i]=0$, and $$ E[U_iU_i^\mathsf{T}] =x_ix_i^\mathsf{T}E[(Y_i-m_ip_i)^2] =m_ip_i(1-p_i)x_ix_i^\mathsf{T}. $$ This agrees with the negative expected Hessian. $\square$

The expression has a simple structure: a scalar $m_ip_i(1-p_i)$ multiplies the covariate outer product $x_ix_i^\mathsf{T}$. The trial total scales the contribution linearly. The factor $p_i(1-p_i)$ is largest at $p_i=0.5$ and smaller when the probability is nearer either end of the unit interval. The covariates determine which combinations of coefficients this group informs.

Worked Information Calculations

Worked Example: One Intercept-Only Group

Suppose one group has $m=20$ trials and fitted probability $p=0.30$. Consider an intercept-only logistic model, so $x=1$. Calculate its information contribution.

The scalar multiplier is $mp(1-p)=20(0.30)(0.70)=4.2$. Since $xx^\mathsf{T}=1$, the information is the $1\times1$ matrix $$ I=4.2(1)(1)=4.2. $$

As a check, the count variance is $\operatorname{Var}(Y)=20(0.30)(0.70)=4.2$. The score is $Y-6$, whose variance is therefore 4.2, matching the information calculation.

Worked Example: A Group With an Intercept and Slope

Suppose a group has $m=12$, fitted probability $p=0.50$, and covariate vector $x=(1,2)^\mathsf{T}$. The two coefficients are an intercept and a slope. Find the group’s information matrix.

First, $mp(1-p)=12(0.50)(0.50)=3$. Next, $$ xx^\mathsf{T} = \begin{pmatrix}1\\2\end{pmatrix} \begin{pmatrix}1&2\end{pmatrix} = \begin{pmatrix}1&2\\2&4\end{pmatrix}. $$

Therefore, $$ I=3\begin{pmatrix}1&2\\2&4\end{pmatrix} =\begin{pmatrix}3&6\\6&12\end{pmatrix}. $$

This single group’s matrix has rank one: its outer product describes information in the coefficient direction represented by $x$, not separate information in every possible direction in the two-dimensional coefficient space.

Worked Example: Adding Two Group Contributions

Consider two independent groups in a logistic model with an intercept and one predictor. Their covariates, trial totals, and fitted probabilities are $x_1=(1,0)^\mathsf{T}$, $m_1=10$, $p_1=0.20$, and $x_2=(1,1)^\mathsf{T}$, $m_2=8$, $p_2=0.75$. Calculate the total information.

For group 1, the scalar is $10(0.20)(0.80)=1.6$, and $$ I_1=1.6 \begin{pmatrix}1&0\\0&0\end{pmatrix} = \begin{pmatrix}1.6&0\\0&0\end{pmatrix}. $$ For group 2, the scalar is $8(0.75)(0.25)=1.5$, and $$ I_2=1.5 \begin{pmatrix}1&1\\1&1\end{pmatrix} = \begin{pmatrix}1.5&1.5\\1.5&1.5\end{pmatrix}. $$

Adding the matrices gives $$ I_{\mathrm{total}}=I_1+I_2 =\begin{pmatrix}3.1&1.5\\1.5&1.5\end{pmatrix}. $$ Its determinant is $3.1(1.5)-(1.5)^2=4.65-2.25=2.4>0$. Thus this sum has full rank for the two coefficients. Neither group alone has full rank, but their different covariate vectors allow the combined information to cover two coefficient directions.

Information Across Independent Groups

Suppose there are $n$ independent grouped records, each following the stated model with fixed $m_i$ and $x_i$. The total log-likelihood is the sum of the group log-likelihoods, so its score is the sum of the group scores. Independence makes the information contributions additive.

Theorem: For independent grouped-binomial records under the logistic model, $$ I(\boldsymbol{\beta}) = \sum_{i=1}^{n}m_ip_i(1-p_i)x_ix_i^\mathsf{T}. $$ For any vector $v\in\mathbb{R}^q$, the information in direction $v$ is $$ v^\mathsf{T}I(\boldsymbol{\beta})v = \sum_{i=1}^{n}m_ip_i(1-p_i)(x_i^\mathsf{T}v)^2 \geq 0. $$

Proof. Let $U=\sum_{i=1}^n U_i$ be the total score. Each group score has mean zero, and independent responses imply independent group scores. For $i\ne j$, this gives $E[U_iU_j^\mathsf{T}]=E[U_i]E[U_j^\mathsf{T}]=0$. Expanding $E[UU^\mathsf{T}]$ therefore leaves only the diagonal terms: $$ I(\boldsymbol{\beta}) =E[UU^\mathsf{T}] =\sum_{i=1}^n E[U_iU_i^\mathsf{T}] =\sum_{i=1}^{n}m_ip_i(1-p_i)x_ix_i^\mathsf{T}. $$ Multiplying on the left by $v^\mathsf{T}$ and on the right by $v$ yields the stated sum of nonnegative terms, since $m_i>0$, $0<p_i<1$, and every squared term is nonnegative. Hence the information matrix is positive semidefinite. $\square$

The directional expression also identifies when a coefficient direction receives no information: its contribution is zero exactly when $x_i^\mathsf{T}v=0$ for every group. The trial totals and probabilities set the nonnegative weights, while the covariate vectors determine which directions are represented. A large total alone cannot make the information full rank if the covariate design does not vary in the relevant directions.

Why the Trial Total and Probability Both Matter

A group with more trials contributes more information when its covariate vector and probability are held fixed. But information does not simply equal the trial total: the probability factor matters. For example, with $m=10$, $p=0.50$, and $x=1$, the intercept information is $10(0.50)(0.50)=2.5$. With the same $m$ and $x$, but $p=0.10$, it is $10(0.10)(0.90)=0.9$. These are model-based information contributions, not measures of how many rows appear in a data table.

If the coefficients are estimated and one wants information at the fitted model, substitute the fitted probabilities $\hat p_i$ into the formula: $$ \hat I=\sum_{i=1}^{n}m_i\hat p_i(1-\hat p_i)x_ix_i^\mathsf{T}. $$ This is the usual plug-in information matrix. For an individual group, the Hessian calculation shows that the observed information and the expected information are equal at a given parameter value.

Common Mistake: Dropping or Duplicating the Trial Total

A frequent error is to use $p_i(1-p_i)x_ix_i^\mathsf{T}$ for a whole group, forgetting that the count represents $m_i$ trials. The opposite error is to multiply that contribution by $m_i$ a second time after already accounting for the binomial count. The correct multiplier for a grouped count is exactly $m_ip_i(1-p_i)$.

The same result applies if the response is written as the observed proportion $Y_i/m_i$, provided its variance and mean model are handled correctly. Its variance is $p_i(1-p_i)/m_i$, not $p_i(1-p_i)$; using the proportion does not erase the information from its denominator. The binomial trial total is part of the sampling model, and it must be retained.

Key takeaway: One grouped-logistic record contributes $m_ip_i(1-p_i)x_ix_i^\mathsf{T}$ units of Fisher information. The trial total scales the contribution, the fitted probability determines its scalar precision factor, and the covariate vector determines its coefficient directions.

Check Your Understanding

Use the contribution formula and the assumptions stated above to answer these questions.

For a group with $m=15$, $p=0.40$, and $x=1$, calculate the intercept information.
For a two-coefficient model and $x=(1,3)^\mathsf{T}$, write down $xx^\mathsf{T}$ and state the maximum rank of that group’s information contribution.
Explain why independent group information matrices add, rather than requiring cross-product terms in the total information.
What does $v^\mathsf{T}Iv=0$ imply about the covariate vectors and direction $v$, when every group has a positive trial total and an interior fitted probability?
A grouped response is recorded as a proportion rather than a success count. Which variance factor must retain the trial total, and why does the denominator still matter?