Score Equations for Grouped Logistic Regression

From grouped counts to score equations

In grouped logistic regression, each row represents a group of trials with a common modeled success probability. The response is recorded as a success count, not as a collection of separate binary entries. The score equations connect those observed counts to the model’s fitted counts: their residuals, observed successes minus fitted successes, must balance across the covariates.

This balance is especially useful for checking a fitted model and understanding what its likelihood equations require. It also makes clear why a group’s trial total matters: a fitted success count is the fitted probability multiplied by the number of trials in that group.

Definition: Suppose there are $n$ independent groups. For group $i$, let $Y_i$ be the number of successes in $m_i$ trials, where $m_i$ is a positive integer and $Y_i\in\{0,\ldots,m_i\}$. Let $\mathbf{x}_i$ be a fixed $q$-dimensional covariate vector, including an intercept if the model has one. Under grouped logistic regression, $$ Y_i\sim \operatorname{Binomial}(m_i,\pi_i),\qquad \operatorname{logit}(\pi_i)=\mathbf{x}_i^\mathsf{T}\boldsymbol{\beta},\qquad \pi_i=\frac{\exp(\mathbf{x}_i^\mathsf{T}\boldsymbol{\beta})}{1+\exp(\mathbf{x}_i^\mathsf{T}\boldsymbol{\beta})}, $$ where $\boldsymbol{\beta}\in\mathbb{R}^q$. The fitted success count in group $i$ at $\boldsymbol{\beta}$ is $m_i\pi_i$.

The model assumes that, conditional on the covariates, the group counts are independent and each has the stated binomial distribution. For a finite parameter vector, every $\pi_i$ lies strictly between zero and one. The derivations below concern the likelihood as a function of finite $\boldsymbol{\beta}$; they do not assume that a finite maximizer exists.

Deriving the score vector

The grouped-data likelihood includes one binomial probability for each group. The binomial coefficient depends on the observed data and trial total, but not on $\boldsymbol{\beta}$. Retaining it gives the full likelihood; dropping it changes the log-likelihood only by a parameter-free constant.

L(\boldsymbol{\beta})= \prod_{i=1}^{n}{m_i\choose y_i}\pi_i^{y_i}(1-\pi_i)^{m_i-y_i}, \qquad \ell(\boldsymbol{\beta})= \sum_{i=1}^{n}\left[ \log {m_i\choose y_i} +y_i\log(\pi_i) +(m_i-y_i)\log(1-\pi_i) \right].

Here $y_i$ denotes the observed value of $Y_i$. To differentiate, write $\eta_i=\mathbf{x}_i^\mathsf{T}\boldsymbol{\beta}$, so that $\pi_i=\exp(\eta_i)/(1+\exp(\eta_i))$. The log-likelihood contribution depending on $\eta_i$ can then be written as $y_i\eta_i-m_i\log(1+\exp(\eta_i))$.

Theorem (Grouped-logistic score): For finite $\boldsymbol{\beta}$, the gradient of the grouped-binomial log-likelihood is $$ \mathbf{U}(\boldsymbol{\beta}) =\frac{\partial\ell(\boldsymbol{\beta})}{\partial\boldsymbol{\beta}} =\sum_{i=1}^{n}\mathbf{x}_i\left(y_i-m_i\pi_i\right). $$ Thus each group contributes its covariate vector multiplied by its observed successes minus its fitted successes.

Proof. Since $\eta_i=\mathbf{x}_i^\mathsf{T}\boldsymbol{\beta}$, its gradient with respect to $\boldsymbol{\beta}$ is $\mathbf{x}_i$. Also,

\frac{d}{d\eta_i}\left[y_i\eta_i-m_i\log(1+\exp(\eta_i))\right] =y_i-m_i\frac{\exp(\eta_i)}{1+\exp(\eta_i)} =y_i-m_i\pi_i.

The binomial coefficient is constant in $\boldsymbol{\beta}$, so its derivative is zero. Applying the chain rule to each group’s log-likelihood contribution and summing the gradients gives

\frac{\partial\ell(\boldsymbol{\beta})}{\partial\boldsymbol{\beta}} =\sum_{i=1}^{n} \frac{\partial\ell_i}{\partial\eta_i} \frac{\partial\eta_i}{\partial\boldsymbol{\beta}} =\sum_{i=1}^{n}(y_i-m_i\pi_i)\mathbf{x}_i.

The log-likelihood is differentiable for all finite $\boldsymbol{\beta}$, because $0<\pi_i<1$ and the displayed terms are finite. This proves the score formula. $\square$

The likelihood equations as residual balances

A finite interior maximum of a differentiable log-likelihood must have zero gradient. Consequently, if a finite maximum-likelihood estimate $\hat{\boldsymbol{\beta}}$ exists, it satisfies the score equations. Writing $\hat{\pi}_i$ for the fitted probability at that estimate gives the equations in vector and component form below.

Likelihood equations: A finite interior maximum-likelihood estimate must satisfy $$ \sum_{i=1}^{n}\mathbf{x}_i\left(y_i-m_i\hat{\pi}_i\right)=\mathbf{0}. $$ For covariate coordinate $j=1,\ldots,q$, the corresponding scalar equation is $$ \sum_{i=1}^{n}x_{ij}\left(y_i-m_i\hat{\pi}_i\right)=0, \qquad \hat{\pi}_i= \frac{\exp(\mathbf{x}_i^\mathsf{T}\hat{\boldsymbol{\beta}})} {1+\exp(\mathbf{x}_i^\mathsf{T}\hat{\boldsymbol{\beta}})}. $$

These equations are necessary conditions for a finite interior maximum, not a guarantee that such an estimate exists. Nor do the equations require every group’s observed count to equal its fitted count. Instead, the residuals must sum to zero after weighting by each covariate coordinate.

Result (covariate balance): At any finite parameter vector satisfying the score equations, the vector of group residuals $r_i=y_i-m_i\hat{\pi}_i$ has zero weighted sum for each modeled covariate. If the model includes an intercept, then the residuals themselves sum to zero: $$ \sum_{i=1}^{n}r_i=0. $$

Proof. Let $x_{ij}$ be coordinate $j$ of $\mathbf{x}_i$. The $j$th coordinate of the score is $\sum_i x_{ij}r_i$. Setting the score vector to zero sets each of these coordinate sums to zero, proving the covariate-balance statement. For an intercept, the corresponding coordinate is $x_{i1}=1$ for every group $i$. Its score equation is therefore $\sum_i r_i=\sum_i(y_i-m_i\hat{\pi}_i)=0$, as claimed. $\square$

The intercept equation says that the total observed number of successes equals the total fitted number of successes. It does not say that the unweighted mean of the group proportions equals the unweighted mean of the fitted probabilities. The score is built from counts, so groups with different trial totals contribute according to their observed and fitted success counts.

Worked examples

Worked Example: Evaluating a Score Vector

Consider two groups with covariates $\mathbf{x}_1=(1,0)^\mathsf{T}$ and $\mathbf{x}_2=(1,1)^\mathsf{T}$. Their trial totals and observed successes are $(m_1,y_1)=(5,3)$ and $(m_2,y_2)=(4,2)$. Evaluate the score at $\boldsymbol{\beta}=(0,\log 2)^\mathsf{T}$.

For group 1, $\eta_1=1(0)+0(\log 2)=0$, so $\pi_1=1/2$. Its fitted count is $5(1/2)=2.5$, and its residual is $3-2.5=0.5$.

For group 2, $\eta_2=1(0)+1(\log 2)=\log 2$, so $\pi_2=2/3$. Its fitted count is $4(2/3)=8/3$, and its residual is $2-8/3=-2/3$.

The two group contributions are $(1,0)^\mathsf{T}(0.5)=(0.5,0)^\mathsf{T}$ and $(1,1)^\mathsf{T}(-2/3)=(-2/3,-2/3)^\mathsf{T}$. Therefore,

\mathbf{U}(\boldsymbol{\beta}) =\begin{pmatrix}0.5\\0\end{pmatrix} +\begin{pmatrix}-2/3\\-2/3\end{pmatrix} =\begin{pmatrix}-1/6\\-2/3\end{pmatrix}.

The score is not zero at this parameter value, so it does not satisfy the likelihood equations. Notice that the second group contributes its residual in both coordinates because both of its covariate values are one.

Worked Example: Solving an Intercept-Only Model

Suppose the model has an intercept only, with two groups: $m_1=12,y_1=5$ and $m_2=18,y_2=8$. The common success probability is $\pi=\exp(\beta_0)/(1+\exp(\beta_0))$. Find the finite maximum-likelihood estimate.

The intercept score equation is

0=(5-12\pi)+(8-18\pi)=13-30\pi.

Solving gives $\hat{\pi}=13/30$, which is strictly between zero and one. Since $\operatorname{logit}(\pi)=\beta_0$,

\hat{\beta}_0=\log\left(\frac{13/30}{17/30}\right) =\log\left(\frac{13}{17}\right).

The fitted success counts are $12(13/30)=5.2$ and $18(13/30)=7.8$. Their sum is $13$, matching the observed total $5+8=13$, as required by the intercept equation. The individual group residuals are $5-5.2=-0.2$ and $8-7.8=0.2$; they need not each be zero.

This stationary point is the maximum: as a function of the common probability, the log-likelihood has derivative $13/\pi-17/(1-\pi)$, which is zero only at $13/30$, positive below that value, and negative above it. Thus the likelihood increases up to this interior point and decreases afterward.

Worked Example: Solving a Model With a Binary Predictor

Let $x_i$ be a binary predictor, with an intercept and slope. There are two groups: at $x=0$, $m_0=10,y_0=3$; at $x=1$, $m_1=12,y_1=6$. The model is $\operatorname{logit}(\pi_i)=\beta_0+\beta_1x_i$. Solve the likelihood equations.

The intercept equation adds the two group residuals, and the slope equation includes only the $x=1$ group:

(3-10\hat{\pi}_0)+(6-12\hat{\pi}_1)=0, \qquad 6-12\hat{\pi}_1=0.

The slope equation gives $\hat{\pi}_1=6/12=1/2$. Substituting into the intercept equation gives $3-10\hat{\pi}_0=0$, hence $\hat{\pi}_0=3/10$. Both probabilities are interior. Their logits give

\hat{\beta}_0=\log\left(\frac{3/10}{7/10}\right)=\log(3/7), \qquad \hat{\beta}_0+\hat{\beta}_1 =\log\left(\frac{1/2}{1/2}\right)=0, $$ $$ \hat{\beta}_1=-\log(3/7)=\log(7/3).

Checking the equations directly, the fitted counts are $10(3/10)=3$ and $12(1/2)=6$. Both group residuals are zero, so their intercept- and slope-weighted sums are zero as well. In this particular design, each predictor level has its own fitted probability; that special feature is why both residuals vanish, rather than merely balancing in aggregate.

Common mistake: using proportions without trial totals

A frequent error is to replace each count residual $y_i-m_i\hat{\pi}_i$ with the proportion residual $y_i/m_i-\hat{\pi}_i$, then give every group equal weight. That is not the grouped-binomial score. The correct expression can be factored as

y_i-m_i\hat{\pi}_i =m_i\left(\frac{y_i}{m_i}-\hat{\pi}_i\right).

Thus the difference between observed and fitted proportions is multiplied by the group’s number of trials. For example, a proportion discrepancy of $0.1$ corresponds to a count residual of $1$ in a group of 10 trials, but a count residual of $10$ in a group of 100 trials. Omitting $m_i$ changes the likelihood equations and treats those discrepancies as equally consequential, despite their different numbers of trials.

The group-level interpretation also matters. A covariate attached to a group describes the group-level relationship modeled by the data; the score equations do not, by themselves, identify individual-level relationships within groups. The fitted-count residuals are aggregated at the same level as the observed counts.

Key takeaway: For grouped logistic regression, the score is the sum of covariate vectors multiplied by observed successes minus fitted successes. At a finite interior maximum, each modeled covariate has zero residual-weighted sum; with an intercept, observed and fitted total successes agree.

Check Your Understanding

Use the grouped score formula to answer the following.

For a group with $m_i=20$, $y_i=7$, and fitted probability $\hat{\pi}_i=0.4$, what is its fitted count and count residual?
Why does an intercept score equation imply equality of total observed and fitted successes?
In the binary-predictor example, why did the slope equation involve only the group with $x=1$?
What factor is missing if one uses a group’s proportion residual without weighting it by its trial total?
Do the likelihood equations generally require every group residual to be zero? Explain the distinction between groupwise equality and covariate-weighted balance.