Negative-Binomial Dispersion From Masked Counts

Why Masked Count Variability Needs Care

For a count outcome, variation across participants can exceed the variation allowed by a Poisson model. A negative-binomial model represents this extra variation with a dispersion parameter. In a blinded re-estimation, treatment labels are masked, so the observed pooled variability also reflects any difference between treatment-arm means. Treating all pooled variability as overdispersion can therefore inflate the dispersion estimate; ignoring dispersion can understate the variance of a rate comparison and the enrollment needed to detect a specified difference.

The calculation below uses pooled participant-level counts and an assumed treatment rate ratio. It does not recover an unknown treatment effect from masked data. The assumed ratio is a planning input used to distinguish the expected between-arm contribution from within-arm overdispersion.

Model and Assumptions

Definition: For participant $i$ in arm $g$, let $Y_{gi}$ be a count observed over the same fixed exposure time $\tau$. Assume independent counts with $$ Y_{gi}\sim \operatorname{NB}(m_g,\kappa),\qquad E(Y_{gi})=m_g=\lambda_g\tau,\qquad \operatorname{Var}(Y_{gi})=m_g+\kappa m_g^2, $$ where $\lambda_g$ is the arm-specific event rate per unit of exposure and $\kappa\geq 0$ is a common dispersion parameter. Under this parameterization, $\kappa=0$ gives the Poisson variance. The negative-binomial model allows variance greater than the mean when $\kappa>0$.

The method-of-moments calculation requires a common exposure time, independent participants, and a common $\kappa$ across arms. It also requires the allocation fraction and an assumed rate ratio. Write $N=n_0+n_1$, $q=n_1/N$, and $R=\lambda_1/\lambda_0=m_1/m_0$. The labels remain masked, but the planned allocation and the assumed ratio are available. The pooled sample mean and sample variance, calculated across the $N$ individual counts, are denoted by $\bar{x}$ and $s^2$.

If follow-up times vary materially, raw count variability combines rate dispersion with exposure variation. The formula below should not be applied directly to such counts: exposure must first be handled in an appropriate model or through a suitable common-exposure analysis. Likewise, dependence between participants or a dispersion parameter that differs across arms violates the stated model.

Separating Dispersion from Treatment-Effect Mixing

Let $m=q m_1+(1-q)m_0$ be the pooled mean count. A randomly selected participant comes from arm 1 with probability $q$ and arm 0 with probability $1-q$. The pooled variance consists of the average within-arm variance plus the variance of the arm means:

\operatorname{Var}(Y) =q\{m_1+\kappa m_1^2\}+(1-q)\{m_0+\kappa m_0^2\} +q(1-q)(m_1-m_0)^2.

The last term is present even if both arms follow Poisson distributions. It is the extra pooled variability caused by mixing two different means, not negative-binomial overdispersion. Given $R$ and the observed pooled mean, the assumed arm means are determined:

m_0=\frac{\bar{x}}{qR+(1-q)},\qquad m_1=Rm_0.

Define $A=q m_1^2+(1-q)m_0^2$ and $B=q(1-q)(m_1-m_0)^2$. Matching the pooled variance to its model expectation gives the dispersion estimate. Because $s^2$ uses denominator $N-1$, its expectation under fixed arm sizes includes a finite-sample factor on $B$:

\widehat{\kappa}_{\rm raw} = \frac{s^2-\bar{x}-\dfrac{N}{N-1}B}{A}.

The factor $N/(N-1)$ follows because the arm means are fixed across the $N$ observations while the pooled sample variance is centered on the sample mean. For large $N$, it is close to 1; the formula shown accounts for it directly. If the assumed arm means are equal, $B=0$, and the estimate reduces to the pooled excess variance divided by the squared-mean contribution.

Worked Example: Correcting the Pooled Variance for a Rate Difference

Suppose a hypothetical blinded internal pilot has $N=100$ participants, equal allocation, pooled mean count $\bar{x}=3.00$ counts per participant, and pooled sample variance $s^2=8.00$ counts squared. The prespecified planning assumption is $R=1.50$. Estimate $\kappa$.

Here $q=0.50$. The assumed arm means compatible with the pooled mean are $$ m_0=\frac{3.00}{(0.50)(1.50)+0.50}=2.40,\qquad m_1=(1.50)(2.40)=3.60 $$ counts per participant.

The squared-mean and between-arm components are $$ A=(0.50)(3.60^2)+(0.50)(2.40^2)=9.36,\qquad B=(0.50)(0.50)(3.60-2.40)^2=0.360. $$ Thus $$ \widehat{\kappa}_{\rm raw} =\frac{8.00-3.00-\dfrac{100}{99}(0.360)}{9.36} =\frac{4.636}{9.36} =0.4953. $$

The estimated dispersion is about $0.495$, a dimensionless quantity. The pooled variance of $8.00$ counts squared is not attributed entirely to dispersion: about $0.364$ of its expectation in the sample-variance calculation comes from the assumed difference between arm means. This is a planning estimate conditional on $R=1.50$, not a blinded estimate of the actual treatment effect.

Why the Moment Formula Works

For fixed arm sizes, the pooled mean has expectation $m=q m_1+(1-q)m_0$. The expected sum of squared deviations from the pooled sample mean has two sources: the sum of within-arm variances and the fixed deviations of each arm mean from the pooled mean. Dividing by $N-1$ gives

E(s^2) = \frac{n_1(m_1+\kappa m_1^2)+n_0(m_0+\kappa m_0^2)}{N} +\frac{N}{N-1}q(1-q)(m_1-m_0)^2.

Since the first term is $\bar{m}+\kappa A$, where $\bar{m}=q m_1+(1-q)m_0$, substituting the observed pooled mean for $\bar{m}$ and solving for $\kappa$ yields the estimator above. This is a method-of-moments argument: it equates observed pooled summaries to their model expectations. It does not make the estimate exact in a particular pilot sample, and substituting estimates for unknown means adds sampling uncertainty.

A raw estimate can be negative when the observed variance is below the Poisson-plus-mixing contribution. A negative value is not a valid negative-binomial dispersion parameter. A common operational choice is to use $\max(0,\widehat{\kappa}_{\rm raw})$ for planning and report that truncation. Truncation is a practical constraint, not evidence that the data establish an exactly Poisson distribution.

From Dispersion to Rate-Comparison Variance

For a group of $n_g$ participants with equal exposure $\tau$, estimate its rate by $\widehat{\lambda}_g=\sum_iY_{gi}/(n_g\tau)$. Under the model, independence gives

\operatorname{Var}(\widehat{\lambda}_g) =\frac{m_g+\kappa m_g^2}{n_g\tau^2} =\frac{\lambda_g}{n_g\tau}+\frac{\kappa\lambda_g^2}{n_g}.

For independent arms, the variance of the estimated rate difference $\widehat{\lambda}_1-\widehat{\lambda}_0$ is the sum of the arm variances. With equal group sizes $n$, a normal-approximation calculation for a two-sided test of $H_0:\lambda_1-\lambda_0=0$, at significance level $\alpha$ and power $1-\beta$ for a planned difference $\Delta=\lambda_1-\lambda_0$, gives

n\ \approx\ \frac{(z_{1-\alpha/2}+z_{1-\beta})^2 \left\{\dfrac{\lambda_0+\lambda_1}{\tau} +\kappa(\lambda_0^2+\lambda_1^2)\right\}} {\Delta^2}.

Here $z_u$ is the $u$-quantile of the standard normal distribution, and $n$ is the approximate number of participants per arm. This expression assumes the rates and dispersion used for planning are appropriate and that a normal approximation is adequate. It is a fixed-design illustration; a group-sequential design may require design-specific critical values rather than the fixed-design $z$-quantiles.

Worked Example: Effect of Dispersion on Enrollment

Consider a hypothetical equal-exposure rate comparison with $\tau=1$ year per participant. Plan for $\lambda_0=0.300$ events per person-year and $\lambda_1=0.450$ events per person-year, so $\Delta=0.150$ events per person-year. Use two-sided $\alpha=0.05$, power $0.80$, and the estimate $\widehat{\kappa}=0.4953$ from the preceding example.

The variance component in braces is $$ \frac{0.300+0.450}{1} +(0.4953)(0.300^2+0.450^2) =0.750+(0.4953)(0.2925) =0.8949. $$ The normal quantiles are approximately $z_{0.975}=1.960$ and $z_{0.80}=0.8416$. Therefore, $$ n\approx \frac{(1.960+0.8416)^2(0.8949)}{0.150^2} =\frac{7.849(0.8949)}{0.0225} =312.2. $$

The approximation is about 312 participants per arm, or 624 in total, before any design-specific adjustment or operational rounding. For comparison, setting $\kappa=0$ gives $$ n\approx\frac{7.849(0.750)}{0.0225}=261.6 $$ participants per arm. Under these assumptions, ignoring overdispersion would materially understate the planned enrollment. The comparison isolates the dispersion contribution; the actual re-estimation must follow the trial’s prespecified design and analysis plan.

Diagnostics, Sensitivity, and Alternatives

A single pooled variance provides limited information about dispersion. Before using the estimate, check that the input summaries refer to participant-level counts on the same exposure scale, that the observed allocation fraction matches $q$, and that the assumed ratio is the one used in the calculation. Large differences between a raw and a truncated estimate, or a raw estimate near zero, signal that sampling variation may dominate the moment estimate. Sparse counts, a small internal pilot, and a few unusually large counts can also make $s^2$, and therefore $\widehat{\kappa}$, unstable.

The treatment-effect assumption matters because it determines the mixing correction. With equal allocation and the same $\bar{x}=3.00$ and $s^2=8.00$ as before, assuming $R=1$ gives $m_0=m_1=3.00$, $B=0$, $A=9.00$, and $$ \widehat{\kappa}_{\rm raw}=\frac{8.00-3.00}{9.00}=0.5556. $$ Assuming $R=2$ instead gives $m_0=2.00$, $m_1=4.00$, $B=1.00$, and $A=10.00$, so $$ \widehat{\kappa}_{\rm raw} =\frac{8.00-3.00-\dfrac{100}{99}(1.00)}{10.00} =0.3990. $$ The pooled data summaries are identical, yet the dispersion estimates differ because the assumed treatment-effect contribution differs. A useful diagnostic is therefore a prespecified sensitivity analysis over plausible rate ratios, rather than presenting one plug-in estimate as assumption-free.

The main alternatives answer different questions. Setting $\kappa=0$ uses a Poisson variance and is appropriate only if that restriction is defensible; it is not a correction for a noisy estimate. Using the uncorrected pooled moment $(s^2-\bar{x})/A$ attributes treatment-effect mixing to dispersion and may overstate it. A negative-binomial likelihood can use the individual counts and a specified mixture model, but it still needs assumptions about the arm means or their ratio when labels are masked. If those assumptions are unavailable, pooled counts alone do not identify treatment-effect mixing separately from dispersion.

Common Mistake

Common mistake: “The pooled sample variance exceeds the pooled mean, so the entire excess must be negative-binomial dispersion.” This ignores the between-arm term $B$. When the arm means differ, a pooled sample combines two distributions and gains variance even if each arm is Poisson. Correct the pooled variance for the treatment-effect contribution under an explicit assumed ratio, then estimate dispersion from what remains. If the assumed ratio is not credible, report sensitivity or use a justified alternative model rather than interpreting the pooled excess as a uniquely determined $\kappa$.

Key takeaway: A blinded negative-binomial dispersion estimate is conditional on the count model, common exposure, allocation, and assumed rate ratio. Its role is to improve the nuisance-variance input to planning, not to reveal the masked treatment effect.

Check Your Understanding

Answer each question, then compare with the solutions below.

In the stated model, what does $\kappa=0$ imply about the count variance?
Why can a pooled sample variance exceed the pooled mean even when both arms are Poisson?
With equal allocation, pooled mean $2.00$, and assumed $R=1$, what arm means are implied?
What does a negative raw moment estimate of $\kappa$ mean for negative-binomial planning?

Solutions. (1) The variance equals the mean, which is the Poisson variance. (2) If arm means differ, mixing them adds the between-arm variance $q(1-q)(m_1-m_0)^2$. (3) The implied means are $m_0=m_1=2.00$, because the assumed ratio is 1 and their pooled mean is 2.00. (4) It is not a valid negative-binomial dispersion value; it indicates that sampling variability or the assumed model and inputs make the observed variance smaller than the fitted Poisson-plus-mixing contribution. A planning procedure may truncate it at zero, while documenting the choice and checking the assumptions.