Quasi-Binomial Models for Grouped Trials

Why Grouped Trial Totals Matter

A grouped binary response records how many successes occurred among a known number of trials. For example, a clinic might report that 12 of 40 patients responded to a treatment, while another clinic reports 6 of 15. Both observations are proportions, but they are based on different numbers of trials. A model for grouped data must keep track of that difference: the trial total affects the expected success count and the variability of the count and proportion.

The quasi-binomial model specifies a mean and a variance for each group without requiring a complete probability distribution for the grouped count. Its key feature is that the variance of a count is proportional to its trial total, while the variance of the corresponding proportion is inversely proportional to that total.

Definition: For group $i$, let $m_i$ be its fixed, positive number of trials, $Y_i$ its number of successes, and $\hat{p}_i=Y_i/m_i$ its observed success proportion. A quasi-binomial mean model specifies $E[Y_i]=m_i p_i$, where $0<p_i<1$. With a common dispersion parameter $\phi>0$, its variance specification is $$ \operatorname{Var}(Y_i)=\phi m_i p_i(1-p_i). $$ Equivalently, $$ E[\hat{p}_i]=p_i,\qquad \operatorname{Var}(\hat{p}_i)=\frac{\phi p_i(1-p_i)}{m_i}. $$

The expected count is $m_i p_i$, not just $p_i$: $p_i$ is the expected success fraction. A regression model can relate this fraction to predictors through a link, for example $g(p_i)=\mathbf{x}_i^\mathsf{T}\boldsymbol{\beta}$. The link describes how predictors affect the group success probability; the trial total determines how that probability translates into an expected count.

The factor $p_i(1-p_i)$ is the binomial variance function. When $\phi=1$, the specified variance agrees with the variance of a binomial count. When $\phi>1$, it is larger by the factor $\phi$, conditional on the same $m_i$ and $p_i$. The quasi-binomial specification alone does not assert that $Y_i$ has a binomial probability distribution, nor does it specify probabilities for every possible count.

Deriving the Count and Proportion Variances

The binomial variance provides the baseline for the quasi-binomial variance specification. Suppose, for the moment, that a group consists of $m_i$ independent Bernoulli trials, each with success probability $p_i$. Write $B_{ij}=1$ for a success on trial $j$ in group $i$, and $B_{ij}=0$ otherwise. Then $Y_i=\sum_{j=1}^{m_i}B_{ij}$.

Result: Under independent Bernoulli trials with common success probability $p_i$, the grouped count satisfies $E[Y_i]=m_i p_i$ and $\operatorname{Var}(Y_i)=m_i p_i(1-p_i)$. Dividing by the fixed trial total gives $E[\hat{p}_i]=p_i$ and $\operatorname{Var}(\hat{p}_i)=p_i(1-p_i)/m_i$.

To verify the count result, each indicator has $E[B_{ij}]=p_i$ and $\operatorname{Var}(B_{ij})=p_i(1-p_i)$. Linearity of expectation gives

E[Y_i]=\sum_{j=1}^{m_i}E[B_{ij}]=m_i p_i.

Independence makes every covariance between distinct trial indicators zero. Therefore

\operatorname{Var}(Y_i) =\sum_{j=1}^{m_i}\operatorname{Var}(B_{ij}) +2\sum_{1\leq j<k\leq m_i}\operatorname{Cov}(B_{ij},B_{ik}) =m_i p_i(1-p_i).

Since $m_i$ is fixed, $\hat{p}_i=Y_i/m_i$ has expectation $E[Y_i]/m_i=p_i$ and variance $\operatorname{Var}(Y_i)/m_i^2=p_i(1-p_i)/m_i$. This proves the stated baseline formulas. The quasi-binomial model retains the mean and multiplies the baseline variance by $\phi$; this is a moment specification, not a claim that the Bernoulli derivation applies to every quasi-binomial data set. $\square$

The conversion between count and proportion variance is exact whenever the denominator $m_i$ is fixed. In particular, starting from the quasi-binomial count variance,

\operatorname{Var}(\hat{p}_i) =\operatorname{Var}\left(\frac{Y_i}{m_i}\right) =\frac{1}{m_i^2}\operatorname{Var}(Y_i) =\frac{1}{m_i^2}\phi m_i p_i(1-p_i) =\frac{\phi p_i(1-p_i)}{m_i}.

Thus the count variance grows in proportion to the number of trials, but the proportion variance shrinks in proportion to its reciprocal. This distinction is essential when groups have unequal trial totals.

Worked Calculations

Worked Example: One Group With Extra-Binomial Variability

A group has $m=20$ trials. Its modelled success probability is $p=0.30$, and the quasi-binomial dispersion is $\phi=1.5$. Find the expected success count, the count variance, and the variance and standard deviation of the observed proportion.

The expected count is the number of trials multiplied by the success probability:

$$ E[Y]=mp=20(0.30)=6. $$

The count variance is

\operatorname{Var}(Y) =\phi mp(1-p) =1.5(20)(0.30)(0.70) =1.5(4.2) =6.3.

The observed proportion is $\hat{p}=Y/20$, so its expectation is $0.30$ and its variance is

\operatorname{Var}(\hat{p}) =\frac{6.3}{20^2} =\frac{6.3}{400} =0.01575.

Its standard deviation is $\sqrt{0.01575}\approx 0.1255$. As a check, the proportion formula gives the same result:

\frac{\phi p(1-p)}{m} =\frac{1.5(0.30)(0.70)}{20} =\frac{0.315}{20} =0.01575.

With $\phi=1$, the baseline binomial count variance would be $20(0.30)(0.70)=4.2$. The specified variance, $6.3$, is 1.5 times that baseline.

Worked Example: Equal Probabilities, Unequal Trial Totals

Two groups share $p=0.40$ and $\phi=2$, but their trial totals are $m_1=10$ and $m_2=40$. Compare expected counts, count variances, and proportion variances.

For group 1, the expected count and variance are

E[Y_1]=10(0.40)=4,\qquad \operatorname{Var}(Y_1)=2(10)(0.40)(0.60)=4.8.

For group 2, they are

E[Y_2]=40(0.40)=16,\qquad \operatorname{Var}(Y_2)=2(40)(0.40)(0.60)=19.2.

The expected proportions are both $0.40$. Their variances, however, are

\operatorname{Var}(\hat{p}_1)=\frac{2(0.40)(0.60)}{10}=0.048, \qquad \operatorname{Var}(\hat{p}_2)=\frac{2(0.40)(0.60)}{40}=0.012.

The second group has four times as many trials, so its count variance is four times as large and its proportion variance is one-quarter as large. The groups have the same modelled success probability, but their proportions do not have the same variability.

Worked Example: Applying a Logistic Mean Model to Groups

Suppose a logistic mean model has $p_i=0.20$ for $x_i=0$ and $p_i=0.50$ for $x_i=1$. The corresponding logit coefficients are $\beta_0=\log(0.20/0.80)=\log(0.25)$ and $\beta_1=0-\log(0.25)=\log(4)$, since $\operatorname{logit}(p_i)=\beta_0+\beta_1x_i$. Let the groups have $m_1=12$ and $m_2=25$, respectively, and let $\phi=1.25$. If the observed counts are $y_1=3$ and $y_2=14$, calculate the fitted mean counts, variances, and observed proportions.

For group 1, the fitted success probability is $0.20$. Thus

E[Y_1]=12(0.20)=2.4,\qquad \operatorname{Var}(Y_1)=1.25(12)(0.20)(0.80)=2.4.

Its observed proportion is $3/12=0.25$, which is not the fitted probability $0.20$. The fitted probability determines the modelled mean and variance; the observed proportion is the realized outcome.

For group 2, the fitted probability is $0.50$, giving

E[Y_2]=25(0.50)=12.5,\qquad \operatorname{Var}(Y_2)=1.25(25)(0.50)(0.50)=7.8125.

The observed proportion is $14/25=0.56$. The two fitted proportion variances are $2.4/12^2=0.016\overline{6}$ and $7.8125/25^2=0.0125$. Even though group 2 has a larger count variance, its proportion variance is smaller because it has more trials.

Interpreting the Dispersion and Trial Total

In the quasi-binomial variance, $\phi$ multiplies the binomial variance at a given $m_i$ and $p_i$. It is the same multiplicative adjustment for the count and proportion variances, but the trial total appears differently in those two forms:

\operatorname{Var}(Y_i)=\phi m_i p_i(1-p_i), \qquad \operatorname{Var}(\hat{p}_i)=\frac{\phi p_i(1-p_i)}{m_i}.

The model therefore does not treat all observed proportions as equally precise merely because they estimate the same type of quantity. At the same $p_i$ and $\phi$, a group with more trials has a less variable proportion. For a count response, meanwhile, a larger group has a larger expected count and a larger variance.

The common $\phi$ is a variance multiplier in this specified model; it is not automatically an explanation of the mechanism causing extra variability. For example, dependence among trials within groups can inflate a count variance, but a common within-group correlation does not in general produce a single common $\phi$ when trial totals vary. If each pair of distinct Bernoulli trials within a group has correlation $\rho$, and each has success probability $p_i$, then

\begin{aligned} \operatorname{Var}(Y_i) &=m_i p_i(1-p_i) +2\binom{m_i}{2}\rho p_i(1-p_i)\\ &=m_i p_i(1-p_i)\{1+(m_i-1)\rho\}. \end{aligned}

The multiplier $1+(m_i-1)\rho$ depends on $m_i$. Unless $\rho=0$, it is not generally constant across groups with unequal trial totals. This illustrates why a variance model should be understood as a stated moment structure, rather than assumed to identify a unique data-generating mechanism.

Common Mistake

Common mistake: Using $\phi p_i(1-p_i)$ as the variance of a grouped count, or using $\phi m_i p_i(1-p_i)$ as the variance of a proportion. Both expressions omit the necessary scaling by the trial total. For the count, multiply the binomial variance function by $m_i$. For the proportion, divide that count variance by $m_i^2$, yielding $\phi p_i(1-p_i)/m_i$.

A quick check is to compare two groups with the same $p_i$ and $\phi$: doubling the trial total must double the count variance and halve the proportion variance. If a calculation does not have those changes, the count and proportion scales have been mixed.

Check Your Understanding

Use the grouped quasi-binomial formulas to answer the following.

A group has $m=30$, $p=0.20$, and $\phi=1.4$. What are the expected count and its variance?
For the same group, what is the variance of the observed proportion? Show that it agrees with the count variance divided by $m^2$.
Two groups share $p$ and $\phi$, but the second has three times the first group’s trial total. How do their count variances and proportion variances compare?
Why is the observed proportion not necessarily the fitted probability in a particular group?
Under a common within-group correlation $\rho$, why does the variance multiplier generally vary across groups with different trial totals?