Why the Level of Assignment Matters
In a cluster-randomized experiment, treatment is assigned to groups such as clinics, schools, or villages, while outcomes are measured on individuals within those groups. The assignment mechanism therefore randomizes clusters, not the people inside them. This matters for two distinct reasons: uncertainty must reflect the number of randomized clusters, and the treatment effect can be averaged across people or across clusters. Those averages need not agree when cluster sizes differ.
For example, a trial may ask about the average effect on a randomly selected person in the study population. That gives a larger cluster more influence because it contains more people. A different question is the average effect for a randomly selected clinic, giving each clinic equal influence regardless of its size. Choosing between these questions is part of defining the causal estimand, not merely a choice of standard error.
Potential Outcomes and Cluster Assignment
Suppose there are \(G\) clusters, indexed by \(g=1,\ldots,G\). Cluster \(g\) contains \(N_g\) individuals, indexed by \(i=1,\ldots,N_g\), and the total number of individuals is \(N=\sum_{g=1}^{G}N_g\). Let \(Z_g\) be the cluster’s assignment, with \(Z_g=1\) for treatment and \(Z_g=0\) for control. Every individual in that cluster receives the same assigned treatment.
Write \(Y_{ig}(1)\) and \(Y_{ig}(0)\) for individual \(i\)’s potential outcomes if their cluster is assigned treatment or control. The observed outcome is \(Y_{ig}=Z_gY_{ig}(1)+(1-Z_g)Y_{ig}(0)\). This setup assumes treatment versions are well-defined and that an individual’s outcome can depend on the assignment of their own cluster, but not on assignments in other clusters. Effects within a cluster, including spillovers among its members, can be incorporated into these cluster-assignment potential outcomes.
Define the potential mean outcome in cluster \(g\) as \(\bar Y_g(z)=N_g^{-1}\sum_{i=1}^{N_g}Y_{ig}(z)\), and the potential total as \(T_g(z)=\sum_{i=1}^{N_g}Y_{ig}(z)\). A cluster’s average treatment effect is \(\tau_g=\bar Y_g(1)-\bar Y_g(0)\). The two population averages below answer different questions.
Equivalently, \(\tau_I=\sum_g(N_g/N)\tau_g\), whereas \(\tau_C=\sum_g(1/G)\tau_g\). If all clusters have the same size, the weights are equal and the estimands coincide. They can also coincide with unequal sizes if cluster size happens to have no effect on the weighted average of treatment effects. In general, however, one should not use the two labels interchangeably.
Worked Example: Two Averages of the Same Cluster Effects
Consider two hypothetical clusters. Cluster A has 2 individuals, each with a control potential outcome of 10 and a treatment potential outcome of 14. Cluster B has 8 individuals, each with a control potential outcome of 20 and a treatment potential outcome of 20. These values imply effects of 4 in A and 0 in B.
The cluster-weighted effect gives the two clusters equal weight:
The individual-weighted effect gives the 2 people in A and the 8 people in B their respective shares of the population:
Both calculations are correct for their stated estimands. The cluster-weighted effect is 2 outcome units per cluster-average individual; the individual-weighted effect is 0.8000 outcome units per individual in the combined population. The difference arises because the larger cluster has no treatment effect and receives more weight in \(\tau_I\).
Estimating the Two Estimands
A common design assigns exactly \(G_1\) of the \(G\) clusters to treatment, with the remaining \(G_0=G-G_1\) assigned to control. Under complete randomization, each cluster has treatment probability \(G_1/G\), and every allocation with \(G_1\) treated clusters is equally likely. The cluster-level difference in means is
where \(\bar Y_g=N_g^{-1}\sum_iY_{ig}\) is the observed mean for cluster \(g\). Under this design, each assigned group is a random sample of clusters from the fixed set of \(G\) clusters. Thus the treated-group mean has expectation \(G^{-1}\sum_g\bar Y_g(1)\), and the control-group mean has expectation \(G^{-1}\sum_g\bar Y_g(0)\). Subtracting gives \(E(\hat\tau_C)=\tau_C\).
To estimate \(\tau_I\), use cluster totals rather than cluster means. More generally, let \(\pi_g=P(Z_g=1)\), with \(0<\pi_g<1\) known from the assignment design. An individual-weighted Horvitz–Thompson estimator is
where only the total for the assigned condition is observed: \(Z_gT_g(1)=Z_g\sum_iY_{ig}\) and \((1-Z_g)T_g(0)=(1-Z_g)\sum_iY_{ig}\). The inverse-probability factors adjust for the chance that each cluster was assigned to the relevant condition.
Unequal Cluster Sizes: Three Calculations
Worked Example: Comparing Estimates Under Unequal Cluster Sizes
Suppose a hypothetical experiment has four clusters with sizes 2, 8, 2, and 4. Exactly two are randomly assigned to treatment. In one realized allocation, the first two clusters are treated and the last two are controls. Their observed cluster means are 14, 10, 8, and 6, respectively. The corresponding observed totals are \(2(14)=28\), \(8(10)=80\), \(2(8)=16\), and \(4(6)=24\). Here \(N=16\) and each cluster’s assignment probability is \(\pi_g=2/4=0.5\).
The cluster-level difference in means is
The pooled individual-level difference in means weights the observed clusters by their numbers of individuals:
The Horvitz–Thompson estimate of the individual-weighted effect instead uses the known assignment probabilities:
These are three distinct realized estimates, not three arithmetic routes to the same answer. The first targets \(\tau_C\); the Horvitz–Thompson estimator is design-unbiased for \(\tau_I\). The pooled contrast is a ratio-based comparison of observed individual means. Its denominators—the numbers of individuals assigned to each arm—can vary with the cluster allocation, so it is not generally design-unbiased for either finite-population estimand. An estimate from one allocation can differ substantially from the others.
Why the Analysis Must Respect Cluster Assignment
Individuals in the same cluster share an assignment. Their outcomes may also be correlated because of shared settings, providers, or local conditions. Consequently, a calculation that treats all \(N\) individuals as independently assigned can report far more independent information than the experiment contains. The number of randomized units is \(G\), not \(N\). This does not mean individual outcomes are irrelevant: they determine the cluster means or totals. It means uncertainty must be calculated in a way that respects the cluster-level randomization.
For the cluster-weighted estimand under complete randomization, a standard conservative Neyman variance estimator uses the sample variances of the observed cluster means within each assignment arm. If \(s_1^2\) and \(s_0^2\) are those sample variances, then
The expression is conservative for the randomization variance because it does not subtract the unobserved variance of cluster-specific treatment effects. It uses variation across clusters—not variation among individual outcomes within a cluster—as the basis for the two arm-specific contributions. For the individual-weighted estimand, the analogous design-based uncertainty calculation must use cluster totals and the assignment probabilities. The appropriate estimator and variance calculation both depend on the target estimand.
Worked Example: A Standard Error Based on Cluster Means
In a hypothetical trial, three treated clusters have observed means 12, 10, and 9, while three control clusters have observed means 8, 7, and 5. The estimated cluster-weighted effect is
For the treated clusters, the mean is 10.333. The sample variance is
For the controls, the mean is 6.667 and
Thus the estimated standard error is
This standard error describes uncertainty for the cluster-weighted contrast under the stated cluster randomization. It is not calculated by pretending that the six cluster means represent six independent individual assignments; they are the six randomized units. The small number of clusters also means that any additional inferential approximation should be considered carefully.
Choosing and Reporting the Estimand
The right weighting depends on the scientific question. If the target is the average effect on an individual in a defined population, use an individual-weighted estimand. If the target is the average effect for a clinic, school, or other cluster, use a cluster-weighted estimand. State the target explicitly, especially when cluster sizes vary. Reporting only “the average treatment effect” can hide an important change in whose outcomes receive the most weight.
Unequal cluster sizes also affect precision. Large clusters may provide more precise information about their own mean outcomes, but they do not create additional randomized treatment assignments. A cluster-level analysis that weights every cluster equally is directly aligned with \(\tau_C\); an individual-weighted analysis must account for cluster sizes and the randomization probabilities to align with \(\tau_I\). Neither weighting scheme is universally preferable: each is appropriate for a different target.
Common Mistakes
- Assuming both averages are the same. They agree when cluster sizes are equal, but unequal sizes generally give the two estimands different weights.
- Using a cluster mean when the target is an individual average. Equal weighting of cluster means targets the average cluster effect; it does not automatically target the average effect across individuals.
- Pooling individuals and treating the result as automatically unbiased. A pooled difference divides by the realized number of individuals in each arm. With unequal cluster sizes those denominators can vary across allocations. Use an estimator suited to the stated estimand and design.
- Counting people as independent assignments. Treatment was randomized across clusters, so an individual-level standard error that ignores clustering can understate uncertainty.
Check Your Understanding
Answer each question by referring to the assignment level and the target population.
- In a trial with clinics as clusters, what does the cluster-weighted average treatment effect give equal weight to?
- Two clusters have sizes 3 and 9 and effects of 6 and 0 outcome units. Calculate both the cluster-weighted and individual-weighted effects.
- Why does a cluster-level difference in means target the cluster-weighted effect under complete randomization?
- What information is needed to form the individual-weighted Horvitz–Thompson estimator when assignment probabilities differ across clusters?
- Why can a standard error that treats every person as independently randomized be misleading in a cluster-randomized experiment?