Grouped Proportions, Trial Totals, and Weights

Counts and Proportions Describe the Same Groups

Grouped binary data are often reported either as the number of successes in each group or as the proportion of successes. These are two summaries of the same observations, but they do not carry the same variance when groups have different numbers of trials. A proportion based on 100 trials is generally more precise than the same proportion based on 5 trials.

Let group $i$ contain $m_i$ trials, with $Y_i$ successes and observed proportion $P_i=Y_i/m_i$. Assume $m_i$ is a known positive integer, and that groups are independent. In a grouped binomial model, each trial in group $i$ has success probability $\mu_i$. In a quasi-binomial model, the mean is the same, but the variance is allowed to include a dispersion factor $\phi>0$.

Definition: For group $i$, the success count is $Y_i$, the trial total is $m_i$, and the success proportion is $P_i=Y_i/m_i$. Under a quasi-binomial mean–variance specification, $$ E[Y_i]=m_i\mu_i,\qquad \operatorname{Var}(Y_i)=\phi m_i\mu_i(1-\mu_i), $$ so $$ E[P_i]=\mu_i,\qquad \operatorname{Var}(P_i)=\frac{\phi\mu_i(1-\mu_i)}{m_i}. $$ Here $0<\mu_i<1$. For an ordinary binomial model, $\phi=1$.

The proportion variance follows by scaling the count: $P_i=Y_i/m_i$, so its expectation is $E[Y_i]/m_i=\mu_i$, and its variance is $\operatorname{Var}(Y_i)/m_i^2=\phi\mu_i(1-\mu_i)/m_i$. Thus, the trial total changes precision, not the expected proportion. In a quasi model, the stated variance is the working variance specification; it does not by itself assert a full probability distribution for $Y_i$.

Why Trial Totals Act as Weights

Suppose the group means follow a regression model with linear predictor $\eta_i=x_i^\mathsf{T}\beta$ and link $g(\mu_i)=\eta_i$, where $x_i$ is the covariate vector for group $i$. Write $\mu_i'=d\mu_i/d\eta_i$. The quasi-score contribution for a response with mean $a_i$ and variance $V_i$ is proportional to $x_i\mu_i'(a_i-\mu_i)/V_i$. This form makes the count–proportion relationship explicit.

Theorem: For a fixed link and the same group-level covariates, the quasi-score equation based on counts $Y_i$ is identical to the equation based on proportions $P_i=Y_i/m_i$ when each proportion’s contribution is multiplied by its trial total $m_i$. Consequently, the two formulations have the same quasi-score roots, provided they use the same model and a common dispersion factor.

Write the count contribution.
The mean is $m_i\mu_i$, its derivative with respect to $\eta_i$ is $m_i\mu_i'$, and its variance is $\phi m_i\mu_i(1-\mu_i)$. The contribution is therefore proportional to $x_i\mu_i'(Y_i-m_i\mu_i)/\{\phi\mu_i(1-\mu_i)\}$.

Write the proportion contribution.
The proportion mean is $\mu_i$, its derivative is $\mu_i'$, and its variance is $\phi\mu_i(1-\mu_i)/m_i$. Its contribution is proportional to $x_i\mu_i'(P_i-\mu_i)m_i/\{\phi\mu_i(1-\mu_i)\}$.

Substitute the observed proportion.
Since $m_i(P_i-\mu_i)=Y_i-m_i\mu_i$, the contributions in Steps 1 and 2 are equal for each group, not merely equal after summing over groups.

In particular, the estimating equation for either representation is

\sum_i \frac{x_i\mu_i'}{\phi\mu_i(1-\mu_i)}(Y_i-m_i\mu_i)=0.

Equivalently, using proportions, the residual term is $m_i(P_i-\mu_i)$. Since a common positive factor $1/\phi$ does not affect the roots, the same coefficient estimates result if the same fixed or estimated common dispersion factor is used in both representations. The corresponding variance-based information also matches: the proportion variance contains the factor $1/m_i$, so its inverse-variance contribution contains $m_i$.

Worked Example: Computing a Group’s Proportion Variance

A group has $m_i=25$ trials, and its fitted success probability is $\mu_i=0.4$. Suppose the quasi-binomial dispersion is $\phi=1.5$. Find the fitted mean and variance of the count and of the proportion.

The fitted mean count is $m_i\mu_i=25(0.4)=10$. The count variance is

\operatorname{Var}(Y_i)=1.5(25)(0.4)(0.6)=1.5(25)(0.24)=9.

The proportion has fitted mean $E[P_i]=0.4$. Its variance is the count variance divided by $25^2$:

\operatorname{Var}(P_i)=\frac{9}{625}=0.0144 =\frac{1.5(0.4)(0.6)}{25}.

The standard deviation of the proportion is $\sqrt{0.0144}=0.12$. The dispersion factor inflates the binomial variance, while the trial total still reduces proportion variance by a factor of $25$ relative to the variance function $\phi\mu_i(1-\mu_i)$.

What Goes Wrong Without the Weights?

If proportions are entered as if they had equal precision, the score no longer contains $m_i(P_i-\mu_i)$; it contains only $P_i-\mu_i$. That treats a proportion based on a small group as having the same influence as a proportion based on a large group. The result is generally a different estimating equation and a different fit.

Worked Example: An Intercept-Only Fit With Unequal Trial Totals

Two groups have $Y_1=1$ success out of $m_1=10$ trials and $Y_2=4$ successes out of $m_2=5$. Fit an intercept-only model with a logit link. Compare the correctly grouped fit with the fit that treats the two proportions as equally weighted.

For the grouped-count fit, the intercept score is zero when the fitted probability equals the pooled success proportion. The total successes are $1+4=5$, and the total trials are $10+5=15$, so

\hat\mu_{\text{weighted}}=\frac{5}{15}=\frac{1}{3},\qquad \hat\beta_{\text{weighted}}=\log\left(\frac{1/3}{2/3}\right)=\log(1/2)\approx-0.6931.

The two observed proportions are $P_1=1/10=0.1$ and $P_2=4/5=0.8$. If each is given equal weight, the intercept equation sets the fitted probability to their unweighted average:

\hat\mu_{\text{unweighted}}=\frac{0.1+0.8}{2}=0.45,\qquad \hat\beta_{\text{unweighted}}=\log\left(\frac{0.45}{0.55}\right) =\log(9/11)\approx-0.2007.

These are substantially different fits. The grouped fit gives more influence to the proportion $0.1$, which is based on 10 trials rather than 5. Averaging the proportions equally discards that distinction.

This difference is not an error in arithmetic; it is a change in the model’s estimating equation. If all $m_i$ are equal, multiplying every proportion contribution by the same trial total does not change the equation’s roots. If trial totals differ, equal weighting and trial-total weighting generally lead to different coefficients. They can still coincide in special cases, such as when the unweighted equation happens to have the same root, but that is not guaranteed by the data format.

Worked Example: Comparing Logistic Score Contributions

Consider three groups with covariate values $x=(0,1,2)$, trial totals $m=(4,20,8)$, and observed proportions $P=(0.25,0.60,0.50)$. Use a logistic regression with intercept and slope. Compare the score direction at the candidate $\beta_0=\beta_1=0$, for which every fitted probability is $0.5$.

For the logit link, $\mu_i'= \mu_i(1-\mu_i)$. At the candidate fit, $\mu_i'=\mu_i(1-\mu_i)=0.25$, so these factors cancel in the quasi-score. Apart from the common $1/\phi$, the intercept and slope components for the correctly weighted proportion representation are

\begin{aligned} U_0 &\propto \sum_i m_i(P_i-0.5) =4(-0.25)+20(0.10)+8(0)=1,\\ U_1 &\propto \sum_i m_i x_i(P_i-0.5) =4(0)(-0.25)+20(1)(0.10)+8(2)(0)=2. \end{aligned}

Without trial-total weights, the corresponding components are

\begin{aligned} U_0^{\text{unweighted}} &\propto \sum_i(P_i-0.5)=-0.25+0.10+0=-0.15,\\ U_1^{\text{unweighted}} &\propto \sum_i x_i(P_i-0.5) =0(-0.25)+1(0.10)+2(0)=0.10. \end{aligned}

The intercept score even points in opposite directions in the two calculations. This candidate is not the fitted model; the calculation shows directly that the two approaches are solving different regression equations. The trial-total-weighted score is the one equivalent to the grouped-count analysis.

Interpreting the Weight and Avoiding Common Errors

A trial-total weight is not an arbitrary preference for large groups. It is the factor required by the variance relationship between a count and its proportion. Under the quasi-binomial variance, proportion precision is proportional to $m_i$ when the fitted probabilities are comparable. In a logistic model, the full working weight also depends on the fitted probability through $\mu_i(1-\mu_i)$; the trial total is one part of that weight.

The equivalence is about the response representation and its estimating equation. It does not mean that every software option called “weight” has identical meaning in every model or procedure. When entering grouped data, use the software’s documented grouped-binomial form, or use proportions with trial totals supplied in the role that reproduces the quasi-binomial score. Do not assume that merely entering a numeric proportion preserves the grouped analysis.

Mistake: treating each group proportion as one equally precise observation. This drops the $m_i$ factor and can change the fit. Use trial-total weights when proportions represent grouped trials.
Mistake: weighting a count by its trial total again. A count $Y_i$ already has mean $m_i\mu_i$ and variance proportional to $m_i$. The equivalent proportion formulation uses $m_i$ as its weight; do not apply that same adjustment twice.
Mistake: claiming that weighting changes the observed proportions. The values $P_i=Y_i/m_i$ are unchanged. Weighting changes their contribution to estimation, reflecting their different variances.

Key takeaway: Grouped counts and proportions give the same quasi-binomial fit when the proportion contribution is scaled by its trial total. The count variance grows with $m_i$, while the proportion variance shrinks as $1/m_i$; omitting the trial-total weights generally changes the model fit.

Check Your Understanding

Use the mean–variance relationship and score equivalence to check your understanding.

If $Y_i$ has mean $m_i\mu_i$ and variance $\phi m_i\mu_i(1-\mu_i)$, derive the mean and variance of $P_i=Y_i/m_i$.
For an intercept-only fit, explain why trial-total-weighted proportions produce the pooled success proportion.
Two groups have the same observed proportion but trial totals of 8 and 80. Under the quasi-binomial variance, which proportion has smaller variance, and by what factor, if their fitted means are equal?
State the residual term in the count quasi-score and show how it equals the trial-total-weighted proportion residual.
Why can treating unequal-trial-total proportions as equally weighted change a regression fit?

Grouped Proportions, Trial Totals, and Weights

What You'll Learn