Blocking and Stratified Randomization

Why Block Before Randomizing?

In a randomized experiment, chance can leave treatment and control groups imbalanced on characteristics that strongly predict the outcome. Blocking addresses this design problem directly: units are first grouped using prespecified pretreatment characteristics, then treatment is randomized within each group. The resulting comparison is made among units assigned under the same block-specific randomization, rather than relying on an analysis to repair an avoidable imbalance afterward.

Blocking is also called stratified randomization. It can ensure that each arm includes units from every important block and can improve precision when outcomes differ substantially across blocks. The estimator must reflect the target population: a contrast for a small block should not automatically receive the same weight as one for a much larger block.

Definition: A block is a group of experimental units formed using information measured before treatment assignment. In blocked (stratified) randomization, a fixed number of units is assigned to treatment within each block by randomization. Let \(b=1,\ldots,B\) index blocks, \(N_b\) be the number of units in block \(b\), and \(n_{1b}\) and \(n_{0b}\) the numbers assigned to treatment and control, respectively, so \(N_b=n_{1b}+n_{0b}\).

To describe the estimand and estimator, use the potential-outcomes framework. Unit \(i\) has potential outcomes \(Y_i(1)\) and \(Y_i(0)\), the outcomes that would be observed under treatment and control. Its individual treatment effect is \(Y_i(1)-Y_i(0)\). Assume the treatment and outcome are well-defined, there is no interference between units, and every unit has a positive chance of assignment to either arm within its block. The potential outcomes and block memberships are fixed for design-based analysis; the random assignment is the source of randomness. The observed outcome is the potential outcome corresponding to the unit’s assigned treatment.

Block-Specific Contrasts and Target Weights

Within block \(b\), let \(\bar{Y}_{1b}\) be the mean observed outcome among treated units and \(\bar{Y}_{0b}\) the mean among control units. Their difference estimates the average treatment effect for that block. To estimate an average effect for the full experimental population, weight the block-specific differences by the proportions of units in the blocks.

\widehat{\tau}_b=\bar{Y}_{1b}-\bar{Y}_{0b}, \qquad \widehat{\tau}_{\mathrm{block}} =\sum_{b=1}^{B} W_b\widehat{\tau}_b, \qquad W_b=\frac{N_b}{N},\quad N=\sum_{b=1}^{B}N_b.

More generally, \(W_b\) can be a prespecified share of a target population, provided the block-specific estimates identify effects for the units represented by that target population. The weights must sum to one. When the target is the population of randomized units, the natural weights are \(N_b/N\). Equal weights instead target the average of the block effects, which is generally a different estimand.

Worked Example: Combining Effects for the Experimental Population

Consider a hypothetical trial of a program intended to increase a test score, measured in points. The trial has two blocks defined before assignment. Block A contains 40 units and block B contains 60. Within each block, half the units are assigned to each arm. Suppose the observed block means are as follows:

Block	\(N_b\)	Treated mean	Control mean	Within-block contrast
A	40	14 points	10 points	4 points
B	60	8 points	7 points	1 point

For the 100 randomized units, the block weights are \(W_A=40/100=0.40\) and \(W_B=60/100=0.60\). The estimated average treatment effect is

\widehat{\tau}_{\mathrm{block}} =0.40(14-10)+0.60(8-7) =0.40(4)+0.60(1) =2.2\text{ points}.

The estimate is a weighted average of the randomized groups’ within-block contrasts. It is not the equal-weight average \((4+1)/2=2.5\) points, which would give the small and large blocks equal influence. This estimate describes an average effect for the experimental population; by itself, it does not establish an effect for people outside that population.

Why the Weighted Estimator Is Unbiased

The key design fact is that within each block the treated units are a random subset of that block, and the controls are the remaining units. Consequently, the treated mean averages the block’s fixed treatment potential outcomes without systematic selection, and the control mean does the same for the control potential outcomes.

Result: Under fixed-count random assignment within each block, \(\widehat{\tau}_b\) is design-unbiased for the average treatment effect in block \(b\). Therefore, \(\sum_b W_b\widehat{\tau}_b\) is design-unbiased for the target-weighted average of the block effects.

For the experimental-population weights, define the block effect as \(\tau_b=N_b^{-1}\sum_{i\in b}\{Y_i(1)-Y_i(0)\}\). Under complete randomization within block \(b\), every unit is equally likely to be assigned to treatment, so the expected treated mean is the block mean of \(Y_i(1)\). Likewise, the expected control mean is the block mean of \(Y_i(0)\). Subtracting these expectations gives \(\mathbb{E}(\widehat{\tau}_b)=\tau_b\). Linearity of expectation then gives \(\mathbb{E}(\sum_b W_b\widehat{\tau}_b)=\sum_b W_b\tau_b\). No assumption that treatment effects are identical across blocks is needed.

Design-Based Variance and a Conservative Standard Error

Write \(S_{1b}^2\) and \(S_{0b}^2\) for the finite-population variances of the potential outcomes \(Y_i(1)\) and \(Y_i(0)\) within block \(b\), using denominator \(N_b-1\). Write \(S_{\tau b}^2\) for the finite-population variance of the individual effects within that block. Under independent complete randomization within blocks, the exact design variance of the weighted estimator is

\operatorname{Var}(\widehat{\tau}_{\mathrm{block}}) = \sum_{b=1}^{B} W_b^2 \left( \frac{S_{1b}^2}{n_{1b}} +\frac{S_{0b}^2}{n_{0b}} -\frac{S_{\tau b}^2}{N_b} \right).

This is a design-based result: it describes variation over the assignment procedure while holding the potential outcomes fixed. The negative term involves the variation in individual treatment effects, which cannot generally be calculated because each unit reveals only one potential outcome. Dropping this nonnegative term gives a conservative variance expression. Replacing the potential-outcome variances with the observed sample variances within each arm gives the usual conservative variance estimator:

\widehat{V}_{\mathrm{cons}} = \sum_{b=1}^{B} W_b^2 \left( \frac{s_{1b}^2}{n_{1b}} +\frac{s_{0b}^2}{n_{0b}} \right), \qquad SE(\widehat{\tau}_{\mathrm{block}}) =\sqrt{\widehat{V}_{\mathrm{cons}}}.

Here \(s_{1b}^2\) and \(s_{0b}^2\) are the sample variances of observed outcomes in the treated and control arms of block \(b\). This estimate requires enough observations in each arm to calculate those variances. It is conservative in the design-based sense because it omits the nonnegative treatment-effect-variation term; it is not a claim that every realized standard error must exceed the true standard deviation.

Worked Example: Calculating a Conservative Standard Error

Use the hypothetical trial above. Suppose the observed within-arm sample variances, in points squared, are \(s_{1A}^2=16\), \(s_{0A}^2=25\), \(s_{1B}^2=9\), and \(s_{0B}^2=16\). The allocation counts are \(n_{1A}=n_{0A}=20\) and \(n_{1B}=n_{0B}=30\). Substituting the block weights and arm counts gives

\begin{aligned} \widehat{V}_{\mathrm{cons}} &=0.40^2\left(\frac{16}{20}+\frac{25}{20}\right) +0.60^2\left(\frac{9}{30}+\frac{16}{30}\right)\\ &=0.16(2.05)+0.36(0.8333)\\ &=0.328+0.300=0.628\text{ points}^2,\\ SE(\widehat{\tau}_{\mathrm{block}}) &=\sqrt{0.628}=0.7925\text{ points}. \end{aligned}

The conservative standard error is about 0.793 points. It summarizes randomization uncertainty in the weighted effect estimate under the stated blocked design, using the observed within-arm variability. It is not the standard deviation of individual outcomes, nor does it measure uncertainty from selecting a different target population.

How Blocking Can Improve Precision

Blocking can improve precision when the block variables predict outcomes: randomization then compares treatment and control units within groups with similar outcome levels, rather than allowing chance differences in the composition of the arms to contribute to the contrast. This is a design advantage, not an automatic guarantee. The relevant comparison is the randomization variance under the chosen design, and its size depends on the potential outcomes and allocation.

Worked Example: A Prognostic Block Can Reduce Randomization Variance

Consider a hypothetical finite population of eight units divided into two blocks of four. The control potential outcomes in block A are \(0,2,4,6\) points, and those in block B are \(20,22,24,26\) points. Suppose every unit’s treatment potential outcome is exactly 2 points higher than its control potential outcome. In both designs below, four units receive treatment and four receive control.

Within either block, the finite-population variance of the control outcomes is \((9+1+1+9)/3=20/3\) points squared. Treatment outcomes differ by a constant shift of 2, so their within-block variance is also \(20/3\); the individual treatment effect is constant, so \(S_{\tau b}^2=0\). Under blocked randomization, two units are treated in each block. Each block-specific contrast therefore has variance \((20/3)/2+(20/3)/2=20/3\). Since each block has weight \(1/2\), the variance of the weighted estimate is

\operatorname{Var}(\widehat{\tau}_{\mathrm{block}}) =\left(\frac12\right)^2\frac{20}{3} +\left(\frac12\right)^2\frac{20}{3} =\frac{10}{3}\approx 3.333\text{ points}^2.

For comparison, under complete randomization across all eight units, the treatment and control potential outcomes each have finite-population variance \(120\) points squared: for example, the eight control outcomes have mean 13, and their squared deviations sum to 840, giving \(840/7=120\). The individual effect is still constant, so the complete-randomization variance of the difference in means is

\operatorname{Var}(\widehat{\tau}_{\mathrm{complete}}) =\frac{120}{4}+\frac{120}{4}-\frac{0}{8} =60\text{ points}^2.

In this example, the blocks separate units with very different baseline outcome levels, and blocking reduces the randomization variance from 60 to about 3.333 points squared. The improvement comes from preventing chance imbalance between the low-outcome and high-outcome blocks. It should not be generalized to every choice of blocks: poorly chosen blocks may add little precision, and small or numerous blocks can make it harder to maintain workable allocations.

Respect the Assignment Design

A blocked experiment is analyzed using the randomization actually carried out. The block-specific contrasts use only treated and control units from the same block; the final estimate then combines those contrasts with weights matched to the estimand. Ignoring the blocks and reporting a pooled contrast can discard the intended within-block comparisons, while an unweighted average of block contrasts can silently change the target.

Blocks should be fixed using pretreatment information. Defining blocks after inspecting outcomes, or using a variable affected by treatment, does not preserve the original design logic and can distort comparisons. Blocking is also distinct from a model-based adjustment: it determines how assignment occurs. The subsequent tutorial on covariate adjustment treats adjustment as an analysis strategy, rather than as the design procedure described here.

Common Mistakes

Giving every block equal weight. This targets the average of block effects, not necessarily the average effect for the randomized population. Use \(N_b/N\) when targeting that population.
Assuming blocking guarantees greater precision. Blocking helps when it creates useful within-block comparisons, but the variance depends on the design and potential outcomes. It is not guaranteed to fall for every blocking scheme.
Forming blocks after assignment or with post-treatment variables. Blocks should be prespecified from pretreatment information so the assignment mechanism and analysis remain aligned.
Pooling outcomes without regard to the design. A pooled difference may fail to reflect the planned within-block contrasts and target weights.
Calling the conservative variance exact. The exact variance contains the unobserved within-block treatment-effect variance term. The usual conservative estimator omits that term.

Key takeaway: Prespecify blocks, randomize within each block, estimate a treated-control contrast within each block, and combine those contrasts using weights that represent the target population. Blocking can improve precision when it prevents chance imbalance on prognostic characteristics, but the design and estimand determine the appropriate calculation.

Check Your Understanding

Work through these questions before moving on.

A trial has 30 units in block A and 70 in block B. What weights estimate the average effect for all randomized units, and why?
In one block, the treated mean is 12 points and the control mean is 9 points. What is the block-specific estimate, and what population does it describe?
Why is the weighted blocked estimator unbiased even when treatment effects differ between blocks?
What term is omitted from the conservative design-based variance estimator, and why can it not generally be calculated from the observed data?
Give one reason blocking may improve precision and one reason it may fail to do so.