Design Effects and Effective Sample Size in Group Sequential Trials

Why Clustered Enrollment Changes Information

In a trial with clustered observations, participants from the same cluster tend to have more similar outcomes than participants from different clusters. Examples include patients treated at the same clinic or students attending the same school. Counting every participant as an independent observation therefore overstates how much information the trial contains when within-cluster outcomes are positively correlated.

For equally sized clusters, the effect can be summarized by a design effect: the factor by which the variance of an estimate exceeds the corresponding variance under independent observations. Dividing the actual sample size by this factor gives an effective sample size, the number of independent observations that would provide the same variance under the stated model. This is an information-equivalent count, not a claim that participants have been removed from the trial.

Definition: Suppose there are $K$ independent clusters, each containing $m$ participants, and the within-cluster intraclass correlation is $\rho$. Under equal marginal outcome variances and a common exchangeable within-cluster correlation, the equal-cluster design effect is $$ D=1+(m-1)\rho. $$ For $N=Km$ participants, the corresponding effective sample size is $$ N_{\mathrm{eff}}=\frac{N}{D}=\frac{Km}{1+(m-1)\rho}. $$

Model and Derivation

The formula applies to a quantitative outcome when each participant has marginal variance $\sigma^2$, the correlation between any two different participants in the same cluster is $\rho$, and outcomes from different clusters are independent. Cluster sizes are all $m$, and the estimate is the participant-weighted overall mean. These assumptions make the covariance structure exchangeable within each cluster. In particular, the calculation does not allow the within-cluster correlation to differ across participant pairs.

To derive the result, let $Y_{ij}$ denote the outcome for participant $i$ in cluster $j$, and let $\bar{Y}_j$ be the mean outcome in cluster $j$. The variance of a cluster mean includes $m$ individual variances and $m(m-1)$ ordered covariance terms:

\operatorname{Var}(\bar{Y}_j) =\frac{1}{m^2}\left[m\sigma^2+m(m-1)\rho\sigma^2\right] =\frac{\sigma^2}{m}\left[1+(m-1)\rho\right].

Because the $K$ clusters are independent and equally sized, the overall participant mean is also the average of the $K$ cluster means. Its variance is therefore:

\operatorname{Var}(\bar{Y}) =\frac{\operatorname{Var}(\bar{Y}_j)}{K} =\frac{\sigma^2}{Km}\left[1+(m-1)\rho\right] =\frac{\sigma^2D}{N}.

If all $N$ observations were independent with the same marginal variance, the variance would be $\sigma^2/N$. The ratio of the clustered variance to that independent-observation variance is $D$, which gives the design effect. Equating $\sigma^2/N_{\mathrm{eff}}$ to $\sigma^2D/N$ yields $N_{\mathrm{eff}}=N/D$.

When $\rho=0$, the formula gives $D=1$, so the effective and actual sample sizes agree. With positive $\rho$, $D$ increases with cluster size: adding participants within existing clusters contributes less information than adding the same number of participants in new, independent clusters. A negative correlation is possible mathematically, but the usual planning use of this formula assumes a nonnegative ICC.

Applying the Design Effect to a Treatment Contrast

Consider a two-arm trial in which clusters, rather than individual participants, are assigned to treatment. Assume equal cluster size $m$, independent clusters, the same marginal variance $\sigma^2$ and ICC $\rho$ in both arms, and a participant-mean estimator in each arm. Let $n_T$ and $n_C$ be the numbers of participants in the treatment and control arms. The variance of the difference in arm means is:

\operatorname{Var}(\bar{Y}_T-\bar{Y}_C) =\sigma^2D\left(\frac{1}{n_T}+\frac{1}{n_C}\right).

Under independent observations with the same marginal variance and allocation, the corresponding variance would be $\sigma^2(1/n_T+1/n_C)$. Thus the treatment-contrast information, defined here as the reciprocal of its variance, is $1/D$ times the independent-observation information. If allocation and the design effect stay constant across analyses, the factor $1/D$ cancels in an information fraction. This cancellation has conditions; it is not a general rule that enrollment fraction always equals information fraction.

Worked Example: Effective Sample Size for a Balanced Trial

Suppose a hypothetical trial plans 20 clinics, with 10 patients per clinic, and assumes an ICC of $0.05$. Ten clinics are assigned to each arm. Find the design effect, the total effective sample size, and the effective sample size per arm.

The assumptions for the calculation are equal clinic sizes, independent clinics, a common within-clinic ICC of $0.05$, and a common marginal outcome variance. The total enrollment is $N=20(10)=200$, with $100$ participants in each arm.

First calculate the design effect:

D=1+(m-1)\rho =1+(10-1)(0.05) =1+0.45 =1.45.

The total effective sample size is:

N_{\mathrm{eff}}=\frac{200}{1.45}=137.9.

Each arm has 10 clinics and 100 participants, so its effective sample size is $100/1.45=68.97$. The interpretation is that, for this variance calculation, the 200 clustered observations provide about as much information as 137.9 independent observations overall, or about 68.97 independent observations per arm. The actual enrollment remains 200.

Information at Sequential Looks

In a group sequential trial, each planned analysis uses the information accumulated by that look. If the same cluster size, ICC, outcome variance, and treatment allocation apply at every look, then the design effect is constant. For example, with balanced arms, the treatment-contrast information at look $k$ is proportional to the number of participants accrued per arm divided by $D$. The same proportionality applies at the final analysis, so the information fraction is the ratio of the look-specific effective sample size to the final effective sample size.

When the same design effect applies at both looks, this ratio simplifies to the ratio of participant counts (or cluster counts, if cluster size is fixed). The effective sample size is still useful: it expresses the clustered information on the scale of independent observations, making clear how much information the planned enrollment represents. It does not replace the actual information used to calculate the sequential test statistic.

Worked Example: Effective Sample Size at an Interim Look

A hypothetical trial plans 30 clinics, with 8 participants per clinic, and an ICC of $0.10$. It has 15 clinics per arm at the final analysis. An interim analysis occurs after 12 clinics have completed follow-up, with 6 clinics per arm. Calculate the design effect, the total effective sample sizes at both analyses, and the information fraction at the interim analysis.

Assume the clinics are independent, all have 8 participants with observed outcomes, the ICC and marginal variance remain constant, and the arm allocation is balanced at both analyses. The design effect is:

$$ D=1+(8-1)(0.10)=1.70. $$

At the interim analysis there are $12(8)=96$ participants, so:

N_{\mathrm{eff},k}=\frac{96}{1.70}=56.47.

At the final analysis there are $30(8)=240$ participants, giving:

N_{\mathrm{eff},K}=\frac{240}{1.70}=141.2.

Since the design effect is unchanged, the information fraction is:

t_k=\frac{I_k}{I_K} =\frac{N_{\mathrm{eff},k}}{N_{\mathrm{eff},K}} =\frac{56.47}{141.2} =0.400.

Thus the interim analysis has about 40.0% of the planned final information under these assumptions. This matches the ratio of completed clinics, $12/30=0.400$, because cluster size and the design effect are constant. Each arm’s effective sample size is $6(8)/1.70=28.24$ at interim and $15(8)/1.70=70.59$ at final.

Why the Number of Clusters Matters

At a fixed total enrollment, larger clusters generally mean fewer independent clusters and a larger design effect when $\rho>0$. This is why a plan that recruits more participants within existing sites can provide less information than a plan that spreads the same enrollment across more sites. The calculation makes this trade-off explicit.

Worked Example: Same Enrollment, Different Cluster Sizes

Compare two hypothetical balanced designs with 240 participants in total and an ICC of $0.08$. Design A has 40 clusters of 6 participants, with 20 clusters per arm. Design B has 20 clusters of 12 participants, with 10 clusters per arm. Assume independent clusters, equal marginal variances, and the same ICC in both designs. Find each design effect and effective sample size. If the marginal standard deviation is 10 outcome units, compare the standard errors of the treatment-mean difference.

For Design A:

D_A=1+(6-1)(0.08)=1.40, \qquad N_{\mathrm{eff},A}=\frac{240}{1.40}=171.4.

Each arm has 120 participants, so the variance of the treatment-mean difference is:

\operatorname{Var}(\bar{Y}_T-\bar{Y}_C) =10^2(1.40)\left(\frac{1}{120}+\frac{1}{120}\right) =2.333, \qquad SE=\sqrt{2.333}=1.528\text{ outcome units}.

For Design B:

D_B=1+(12-1)(0.08)=1.88, \qquad N_{\mathrm{eff},B}=\frac{240}{1.88}=127.7.

Its treatment-contrast variance and standard error are:

\operatorname{Var}(\bar{Y}_T-\bar{Y}_C) =10^2(1.88)\left(\frac{1}{120}+\frac{1}{120}\right) =3.133, \qquad SE=\sqrt{3.133}=1.770\text{ outcome units}.

Under these assumptions, Design A has a smaller standard error and a larger effective sample size, despite the same total enrollment. The difference arises from the number and size of clusters, not from a change in the number of participants or the outcome’s marginal standard deviation.

Checks, Alternatives, and Limitations

Before using a design effect to interpret a planned information target, check that the assumptions match the trial. In particular, confirm that cluster sizes are equal, the assumed ICC is suitable for the outcome and population, clusters can reasonably be treated as independent, and the same design effect applies at the looks being compared. A sensitivity analysis over plausible ICC values is often useful because the effective sample size can change substantially when $m$ is large.

The simple conversion is a planning and interpretation device, not a substitute for a cluster-aware analysis. A naïve analysis that treats correlated participants as independent uses too small a standard error under positive ICC. A direct analysis that accounts for the cluster structure is the appropriate basis for inference; the design-effect calculation provides a compact way to compare its variance with an independent-observation benchmark. For unequal cluster sizes, the calculation requires additional care; see the earlier material on information with unequal cluster sizes for that case.

In sequential monitoring, use the information appropriate to the planned analysis rather than mechanically applying an enrollment fraction. If cluster size, allocation, ICC, or the set of observed outcomes changes over time, the simple cancellation that gave $t_k=N_{\mathrm{eff},k}/N_{\mathrm{eff},K}$ may no longer hold. Reassess the information under the actual design assumptions before interpreting a look as having reached its target. The design effect itself does not change the trial’s type I error rate or define its stopping boundaries.

Common mistake: “There are 200 participants, so the trial has the information of 200 independent observations.” This ignores the shared outcomes within clusters. With 10 participants per cluster and $\rho=0.05$, $D=1.45$, so the independent-sample equivalent is $200/1.45=137.9$, under the stated model. Conversely, $N_{\mathrm{eff}}$ is not the number actually recruited and should not be reported as the trial’s enrollment.

Check Your Understanding

Answer each question using the equal-cluster assumptions stated in this tutorial. Brief solutions follow the questions.

A study has clusters of size 5 and an ICC of $0.10$. What is its design effect?
There are 18 clusters of 8 participants each, with $\rho=0.05$. Calculate the total effective sample size.
Why does the information fraction equal the enrollment fraction when the cluster size and design effect are constant across looks?
At a fixed total enrollment, why can recruiting additional participants within existing clusters add less information than recruiting participants in new clusters?

Solutions. (1) $D=1+(5-1)(0.10)=1.40$. (2) The total enrollment is $18(8)=144$, and $D=1+(8-1)(0.05)=1.35$, so $N_{\mathrm{eff}}=144/1.35=106.7$. (3) The information at each look is proportional to its effective sample size; if $D$ is constant, dividing both the interim and final sample sizes by the same $D$ leaves their ratio unchanged. (4) Participants in one cluster share correlated outcomes, so they contribute overlapping information; increasing the number of independent clusters reduces that dependence penalty.