Pooled Variance Estimation With Treatment Labels Masked

Why the Masked Variance Needs Care

In an internal-pilot study, a sample-size update may use an interim estimate of outcome variability while treatment labels remain masked. For a continuous outcome, one direct calculation is the sample variance of all observed outcomes treated as one combined set. Its definition is simple, but its numerator contains more than the variation within each treatment group: it also reflects how the group means sit relative to the combined mean. That distinction matters whenever group sizes differ or the group means are not identical.

The aim here is to derive that calculation and make its components transparent. The algebraic identity itself does not require a probability model. A model is needed, however, to describe the data-generating assumptions under which the calculation can be interpreted as estimating a common outcome variance.

Setup and Definitions

Consider a hypothetical two-arm trial with $n_1$ outcomes in one group and $n_0$ in the other, for a total of $N=n_1+n_0$ outcomes. Write the observed outcomes as $Y_{1i}$, $i=1,\ldots,n_1$, and $Y_{0i}$, $i=1,\ldots,n_0$. The subscripts identify the groups for purposes of the derivation; an analyst calculating the masked variance uses the combined outcomes without assigning treatment labels to individual observations.

The group means are $\bar{Y}_1$ and $\bar{Y}_0$. The mean of all $N$ outcomes, equivalently the allocation-weighted mean of the group means, is denoted by $\bar{Y}$. The masked pooled sample variance is the ordinary sample variance of those combined outcomes, with denominator $N-1$.

Definition: The masked pooled sample variance is $$ s_{\mathrm{mask}}^2=\frac{\sum_{i=1}^{n_1}(Y_{1i}-\bar{Y})^2+\sum_{i=1}^{n_0}(Y_{0i}-\bar{Y})^2}{N-1}, \qquad \bar{Y}=\frac{n_1\bar{Y}_1+n_0\bar{Y}_0}{N}. $$ The weights in $\bar{Y}$ are the group sizes: a group with more observed outcomes contributes more to the combined mean.

For a sampling interpretation, one useful model is that observations are independent within and between groups, with $Y_{ji}=\mu_j+\varepsilon_{ji}$, where the errors have mean zero and common variance $\sigma^2$. This states a common-variance model while allowing the group means $\mu_1$ and $\mu_0$ to differ. The sum-of-squares decomposition below is an identity for the observed data and does not depend on this model. Independence and the common-variance assumption are relevant when interpreting the variance estimate; the formula does not establish that those assumptions hold.

Deriving the Numerator

To separate the sources of variation, add and subtract each group mean inside every squared deviation from the overall mean. For group 1, for example, $Y_{1i}-\bar{Y}=(Y_{1i}-\bar{Y}_1)+(\bar{Y}_1-\bar{Y})$. When the squares are summed, the cross-product vanishes because the deviations from the group mean sum to zero: $\sum_i(Y_{1i}-\bar{Y}_1)=0$. The same reasoning applies to group 0.

\begin{aligned} \sum_{i=1}^{n_1}(Y_{1i}-\bar{Y})^2 &=\sum_{i=1}^{n_1}(Y_{1i}-\bar{Y}_1)^2+n_1(\bar{Y}_1-\bar{Y})^2,\\ \sum_{i=1}^{n_0}(Y_{0i}-\bar{Y})^2 &=\sum_{i=1}^{n_0}(Y_{0i}-\bar{Y}_0)^2+n_0(\bar{Y}_0-\bar{Y})^2. \end{aligned}

Adding these equations gives the within-group sums of squares plus a between-group term. The allocation weights in that term follow from expressing each group mean’s deviation from the combined mean in terms of the difference between group means: $\bar{Y}_1-\bar{Y}=(n_0/N)(\bar{Y}_1-\bar{Y}_0)$ and $\bar{Y}_0-\bar{Y}=-(n_1/N)(\bar{Y}_1-\bar{Y}_0)$. Substitution yields the following identity.

Result: The combined sum of squares is the sum of the two within-group sums of squares and a between-group component: $$ \begin{aligned} \sum_{j\in\{0,1\}}\sum_{i=1}^{n_j}(Y_{ji}-\bar{Y})^2 &=\sum_{i=1}^{n_1}(Y_{1i}-\bar{Y}_1)^2 +\sum_{i=1}^{n_0}(Y_{0i}-\bar{Y}_0)^2\\ &\quad+n_1(\bar{Y}_1-\bar{Y})^2+n_0(\bar{Y}_0-\bar{Y})^2\\ &=\sum_{i=1}^{n_1}(Y_{1i}-\bar{Y}_1)^2 +\sum_{i=1}^{n_0}(Y_{0i}-\bar{Y}_0)^2 +\frac{n_1n_0}{N}(\bar{Y}_1-\bar{Y}_0)^2. \end{aligned} $$

Thus the masked variance divides this entire numerator by $N-1$. The factor $n_1n_0/N$ is determined by the allocation counts; it is not generally the same as either group size, and it is not obtained by giving the two group means equal weight. If the means are equal, the between-group component is zero. Otherwise, the component is nonnegative and contributes to the combined sum of squares.

Worked Calculations

Worked Example: Unequal Group Sizes and the Combined Variance

Suppose a hypothetical interim data set has group 1 outcomes $10,12,14$ units and group 0 outcomes $10,10,12,12$ units. There are $n_1=3$, $n_0=4$, and $N=7$ observations. First calculate the group means and within-group sums of squares.

For group 1, $\bar{Y}_1=(10+12+14)/3=12$, and its sum of squares is $(10-12)^2+(12-12)^2+(14-12)^2=4+0+4=8$ squared units. For group 0, $\bar{Y}_0=(10+10+12+12)/4=11$, and its sum of squares is $(10-11)^2+(10-11)^2+(12-11)^2+(12-11)^2=1+1+1+1=4$ squared units.

The combined mean is allocation-weighted: $\bar{Y}=(3(12)+4(11))/7=80/7\approx11.4286$ units. The between-group term is $(3(4)/7)(12-11)^2=12/7\approx1.7143$ squared units. Therefore the total numerator is $8+4+12/7=96/7\approx13.7143$ squared units, and $s_{\mathrm{mask}}^2=(96/7)/(7-1)=16/7\approx2.286$ squared units. The corresponding sample standard deviation is $\sqrt{16/7}\approx1.512$ units.

As a direct check, the squared deviations of the seven raw outcomes from $80/7$ sum to $96/7$. This agrees with the decomposition. The allocation-weighted mean is not the unweighted average $(12+11)/2=11.5$, because the group sizes differ.

Worked Example: Verifying the Calculation From Raw Outcomes

Suppose a second hypothetical interim sample has two outcomes $20,22$ units in group 1 and two outcomes $18,20$ units in group 0. The group means are $21$ and $19$ units, and each within-group sum of squares is $2$ squared units. Since the allocation is balanced, the combined mean is $(2(21)+2(19))/4=20$ units.

The between-group term is $(2(2)/4)(21-19)^2=2(4)=8$ squared units. The combined numerator is $2+2+8=12$ squared units, so $s_{\mathrm{mask}}^2=12/(4-1)=4$ squared units.

Check directly: the four squared deviations from 20 are $0,4,4,0$, which sum to 8, not 12. This exposes an arithmetic error in the stated decomposition if the between-group term is taken as 8. Recomputing it carefully, the coefficient is $n_1n_0/N=2(2)/4=1$, so the correct between-group term is $1(21-19)^2=4$. The numerator is $2+2+4=8$ squared units, and the masked variance is $8/3\approx2.667$ squared units, agreeing with the direct calculation.

This check illustrates why the allocation coefficient must be evaluated before multiplying by the squared difference in means. The group sample variances are each $2$ squared units, but the variance of all four outcomes is $8/3$ squared units because it is computed around one combined mean.

Worked Example: Why the Allocation-Weighted Mean Matters

Consider a third hypothetical sample: group 1 has outcomes $5,7$ units, while group 0 has outcomes $1,3,3,5$ units. The group means are $\bar{Y}_1=6$ and $\bar{Y}_0=3$. The within-group sums of squares are $2$ and $8$ squared units, respectively.

The correct overall mean is $(2(6)+4(3))/6=4$ units. The between-group term is $(2(4)/6)(6-3)^2=4/3(9)=12$ squared units. Thus the numerator is $2+8+12=22$ squared units, and $s_{\mathrm{mask}}^2=22/5=4.4$ squared units.

If the group means were incorrectly averaged without weights, the result would be $4.5$ units, which is not the mean of the six outcomes. Centering all observations at $4.5$ would therefore not calculate the sample variance of the combined data. The direct raw-data check around the correct mean $4$ gives squared deviations $1,9,9,1,1,1$, totaling $22$.

What the Calculation Does and Does Not Represent

The masked pooled variance is the ordinary sample variance of the combined observations. By contrast, a pooled within-group variance combines deviations from the separate group means and divides their sum by $N-2$: $(SS_1+SS_0)/(N-2)$. These quantities answer different computational questions. The former centers every outcome at one overall mean and uses $N-1$; the latter centers outcomes at their own group means and uses $N-2$. They need not be equal.

With treatment labels masked, the combined-data calculation can be made without identifying which outcome came from which treatment. The decomposition is useful for understanding what that calculation contains, but its group-specific pieces cannot generally be recovered from the masked values alone. In particular, one should not describe the masked sample variance as if it were automatically the same quantity as a within-group variance. The appropriate interpretation depends on the design, assumptions, and purpose of the variance update.

The algebra assumes only that the observations can be partitioned into the two groups for the derivation. For the simple common-variance model, independent observations and a common variance are substantive assumptions, not consequences of masking. If the data include duplicated records, inconsistent units, or outcomes that were not measured on the same scale, the numerical result may be computable but not a meaningful estimate for the planned analysis.

Checks and Common Pitfalls

Several quick checks catch most errors in this calculation:

Check the overall mean. It must equal the mean of all $N$ raw outcomes and also $(n_1\bar{Y}_1+n_0\bar{Y}_0)/N$.
Check the between-group term. It is nonnegative and equals $(n_1n_0/N)(\bar{Y}_1-\bar{Y}_0)^2$. It is zero when the group means match.
Check units. Means have outcome units; sums of squares and variances have squared outcome units; the standard deviation has the original outcome units.
Check the divisor. The combined sample variance uses $N-1$, not $N$ and not $N-2$.
Check the arithmetic independently. When raw data are available, the sum of squared deviations from the combined mean must match the decomposed numerator.

A common mistake is to average the group means equally even when the allocation is unequal, then calculate deviations around that incorrect value. The correction is to weight each group mean by its observed group size. Another mistake is to take only the within-group sums of squares but call the result the combined-data variance; that omits the between-group component and uses a different centering rule. Finally, a formula can be copied correctly while a squared deviation is evaluated incorrectly, so recomputing each term and checking against raw outcomes is good practice.

Key takeaway: The masked sample variance is the total sum of squares around the allocation-weighted overall mean divided by $N-1$. Its numerator separates exactly into within-group sums of squares and a between-group term weighted by $n_1n_0/N$.

Check Your Understanding

Answer the questions, then compare with the brief solutions.

For group sizes $n_1=5$ and $n_0=3$, what weights determine the overall mean of the two group means?
If the two group means are equal, what is the between-group component of the combined sum of squares?
For group sizes $n_1=2$ and $n_0=4$, calculate the coefficient on $(\bar{Y}_1-\bar{Y}_0)^2$ in the between-group term.
Why does the combined sample variance use $N-1$, rather than the $N-2$ divisor in the pooled within-group variance?
What should you check if a decomposed numerator does not match the sum of squared deviations calculated from the raw outcomes?

Solutions. (1) Use weights $5/8$ on $\bar{Y}_1$ and $3/8$ on $\bar{Y}_0$. (2) It is zero because the squared difference between the group means is zero. (3) The coefficient is $n_1n_0/N=2(4)/6=4/3$. (4) The combined sample variance centers all $N$ observations at one estimated overall mean, whereas the pooled within-group variance centers each group separately and uses $N-2$. (5) Recheck the group sizes, weighted overall mean, squared deviations, between-group coefficient, and arithmetic; the identity requires the two numerator calculations to agree.