Intraclass Correlation Estimation In Masked Cluster Trials

Why Masking Complicates Cluster-Correlation Estimation

In a cluster trial, participants in the same clinic, school, or other randomized unit may have more similar outcomes than participants in different clusters. The intraclass correlation coefficient (ICC) describes that similarity. It is a nuisance parameter that can matter for planning, but estimating it from treatment-masked outcomes requires care: treatment assignment varies between clusters, so a treatment effect can look like extra between-cluster variation.

Masking means the analyst estimating the nuisance parameter does not see which treatment each cluster received. It does not mean that treatment effects are absent from the pooled data. The method here makes that distinction explicit for a balanced design, and shows why a masked estimate can depend on an assumed treatment effect.

Model and Definition

Consider a continuous outcome measured on $G$ clusters, with $m$ participants in every cluster, so the total number of participants is $n=Gm$. Of the clusters, $G_1$ are assigned to treatment and $G_0$ to control, where $G=G_0+G_1$. Let $T_i=1$ for a treatment cluster and $T_i=0$ for a control cluster. A random-intercept model is

Y_{ij}=\mu+\tau T_i+b_i+\varepsilon_{ij}, \qquad i=1,\ldots,G,\quad j=1,\ldots,m.

Here $\mu$ is the control-arm mean, $\tau$ is the treatment effect in outcome units, $b_i$ is the random effect for cluster $i$, and $\varepsilon_{ij}$ is the individual-level error. Assume $b_i$ are independent with mean zero and variance $\sigma_b^2$; the $\varepsilon_{ij}$ are independent with mean zero and variance $\sigma_e^2$; and the two sets of terms are mutually independent. For the usual normal-theory ANOVA distributions, additionally assume these terms are normally distributed. Treatment is assigned at the cluster level, and the allocation counts $G_0,G_1$ are fixed for the calculation.

Definition: Under this model, two different participants in the same cluster have covariance $\sigma_b^2$ and marginal variance $\sigma_b^2+\sigma_e^2$. Their intraclass correlation is $$ \rho=\frac{\sigma_b^2}{\sigma_b^2+\sigma_e^2}. $$ It is the correlation between outcomes for two distinct participants in the same cluster, not the correlation of a participant with their own outcome.

The model assigns a shared source of variation, $b_i$, to all participants in a cluster, while $\varepsilon_{ij}$ varies between participants. The treatment effect $\tau T_i$ is also shared by every participant in a cluster, but it is a fixed arm difference, not part of $\sigma_b^2$. Confusing these sources is the central estimation pitfall.

What the Masked ANOVA Mean Squares Contain

Calculate the mean of each cluster, $\bar Y_i$, and the overall participant mean, $\bar Y$. The within-cluster mean square and the between-cluster mean square are

MS_W=\frac{\sum_{i=1}^{G}\sum_{j=1}^{m}(Y_{ij}-\bar Y_i)^2}{G(m-1)}, \qquad MS_B=\frac{m\sum_{i=1}^{G}(\bar Y_i-\bar Y)^2}{G-1}.

Because treatment is constant within a cluster, subtracting the cluster mean removes both the cluster intercept and the treatment term. Thus $MS_W$ estimates individual-level variance, with $E(MS_W)=\sigma_e^2$ under the model. The between-cluster mean square contains both random cluster variation and any differences between treatment and control cluster means.

For balanced cluster sizes, the expected between-cluster mean square is

E(MS_B)=\sigma_e^2+m\sigma_b^2+ \frac{m\tau^2G_0G_1}{G(G-1)}.

The final term is the contribution of the fixed treatment difference to pooled between-cluster variation. The expression uses the actual numbers of clusters allocated to each arm; it does not assume that the pooled treatment effect is zero. If the assignment labels are hidden, $\tau$ is also not estimated from the masked pooled outcomes by this calculation. A prespecified value $\delta_0$ for the treatment effect can instead be used to adjust the mean square.

Within-cluster variation.
The within-cluster residuals are $Y_{ij}-\bar Y_i=b_i-b_i+\varepsilon_{ij}-\bar\varepsilon_i$. Their expected mean square is $\sigma_e^2$.

Cluster-mean variation.
The cluster means are $\bar Y_i=\mu+\tau T_i+b_i+\bar\varepsilon_i$. Their random component has variance $\sigma_b^2+\sigma_e^2/m$, which contributes $m\sigma_b^2+\sigma_e^2$ to $MS_B$.

Fixed treatment contribution.
The sum of squared deviations of the fixed arm means from their overall mean, on the cluster-mean scale, is $\tau^2G_0G_1/G$. Multiplying by $m$ for the between-cluster sum of squares and dividing by $G-1$ gives $m\tau^2G_0G_1/[G(G-1)]$.

These expected mean squares give the masked, design-adjusted method-of-moments estimate

\widehat{\sigma}_e^2=MS_W,\qquad \widehat{\sigma}_b^2(\delta_0)= \frac{MS_B-MS_W-\dfrac{m\delta_0^2G_0G_1}{G(G-1)}}{m}, \qquad \hat\rho(\delta_0)= \frac{\widehat{\sigma}_b^2(\delta_0)} {\widehat{\sigma}_b^2(\delta_0)+MS_W}.

The assumed effect $\delta_0$ is expressed in the outcome's units. The correction uses its square, so the adjustment is the same for assumptions of equal magnitude in opposite directions. The estimate is unbiased for the random-intercept variance in expectation if $\delta_0^2=\tau^2$ and the model and balanced-design conditions hold. If that assumption is wrong, the estimated cluster variance—and therefore the ICC—can be too large or too small.

Worked Examples

Worked Example: Adjusting for a Prespecified Treatment Effect

Suppose a hypothetical balanced trial has $G=20$ clusters, with $G_1=G_0=10$, and $m=8$ participants per cluster. The masked ANOVA summaries are $MS_B=28$ and $MS_W=12$ outcome-units squared. The prespecified treatment-effect assumption is $\delta_0=2$ outcome units. Estimate the ICC.

First calculate the treatment contribution to $MS_B$:

\frac{m\delta_0^2G_0G_1}{G(G-1)} = \frac{8(2^2)(10)(10)}{20(19)} = \frac{3200}{380} =8.421.

The individual-level variance estimate is $12$ outcome-units squared. The adjusted cluster variance estimate is

\widehat{\sigma}_b^2 = \frac{28-12-8.421}{8} =0.9474\text{ outcome-units}^2.

Therefore,

\hat\rho = \frac{0.9474}{0.9474+12} =0.07318.

Under this model and effect assumption, the estimated correlation between two different participants in the same cluster is about $0.0732$. This is not the proportion of participants who are similar; it is a correlation on the outcome scale.

Worked Example: Sensitivity to the Effect Assumption

Consider another hypothetical balanced trial with $G=24$, $G_0=G_1=12$, $m=6$, $MS_B=26$, and $MS_W=10$, where the mean squares are in outcome-units squared. Compare assumptions of $\delta_0=1$ and $\delta_0=2$ outcome units.

The coefficient multiplying $\delta_0^2$ in the correction is

\frac{mG_0G_1}{G(G-1)} = \frac{6(12)(12)}{24(23)} =1.565.

For $\delta_0=1$, the adjusted cluster variance and ICC are

\widehat{\sigma}_b^2 = \frac{26-10-1.565(1^2)}{6} =2.406\text{ outcome-units}^2, \qquad \hat\rho=\frac{2.406}{2.406+10}=0.1939.

For $\delta_0=2$, the treatment correction is $1.565(2^2)=6.261$ outcome-units squared. Hence

\widehat{\sigma}_b^2 = \frac{26-10-6.261}{6} =1.623\text{ outcome-units}^2, \qquad \hat\rho=\frac{1.623}{1.623+10}=0.1397.

Both calculations use the same masked summaries, yet the ICC estimates differ because the treatment contribution cannot be separated from cluster variation using those summaries alone. Reporting the effect assumption alongside the estimate is therefore essential.

Worked Example: A Negative Adjusted Variance Estimate

Suppose a hypothetical trial has $G=16$ clusters, $G_0=G_1=8$, $m=5$, $MS_B=13$, and $MS_W=10$, in outcome-units squared. Use $\delta_0=2$ outcome units.

The correction is

\frac{m\delta_0^2G_0G_1}{G(G-1)} = \frac{5(4)(8)(8)}{16(15)} =5.333\text{ outcome-units}^2.

Consequently,

\widehat{\sigma}_b^2 = \frac{13-10-5.333}{5} =-0.4667\text{ outcome-units}^2.

A variance cannot be negative, so this is not a valid variance-component estimate and must not be interpreted as a negative ICC. It indicates that the observed between-cluster mean square is too small to support the assumed treatment correction plus a nonnegative cluster variance. A constrained report may set the variance estimate, and hence the ICC, to zero, but should flag the boundary result rather than treating zero as proof that clusters are independent.

Diagnostics, Alternatives, and Limitations

The estimator is most defensible when clusters have the same size, allocation counts are known, and a scientifically justified effect assumption is available. In a blinded analysis, the assumed effect should be set without searching over values until the desired ICC or sample-size implication appears. Sensitivity calculations across plausible effect magnitudes make the dependence on that assumption visible.

Check the design. The displayed ANOVA formula relies on equal cluster sizes and the stated allocation counts. With unequal cluster sizes, ordinary one-way mean squares do not have these expected values; a method that accounts for cluster sizes is needed.
Check the components. $MS_W$ should be positive, and the adjusted cluster-variance estimate should be nonnegative for an ordinary variance-component interpretation. A negative result is a warning about sampling variability, an incompatible effect assumption, or model/design mismatch.
Check masking and assumptions. The calculation uses masked outcomes and a prespecified $\delta_0$, not an effect estimated by comparing the hidden arms. Its interpretation also relies on a common treatment effect and the random-intercept model; substantial cluster-specific treatment effects or other variance structures are not represented.
Check information content. With few clusters, the between-cluster mean square can be imprecise even when there are many participants within each cluster. A precise-looking ICC from a plug-in calculation should not be taken as evidence of precise estimation.

The main alternative is to use treatment labels in a model that includes treatment as a fixed effect and cluster as a random effect. That separates the arm mean difference from residual cluster variation, but it is not a masked analysis. A simpler masked calculation that sets $\delta_0=0$ is equivalent to making no treatment correction: it is appropriate only if a zero treatment effect is the intended assumption. Otherwise it can attribute treatment-related variation to the cluster component and inflate the ICC. The choice is therefore not between an automatically correct masked estimate and an automatically correct unmasked estimate; it depends on the masking requirement, model assumptions, and information available.

This method estimates an ICC, not an enrollment target. It also does not remove uncertainty from the ICC estimate: the formulas are method-of-moments estimates, and a single point estimate can be unstable. The useful practice is to retain the component estimates, state the effect assumption and design conditions, and investigate whether plausible changes in that assumption materially change the result.

Key takeaway: In a balanced masked cluster trial, within-cluster variation estimates individual-level variance, while pooled between-cluster variation contains both random cluster variation and treatment-related variation. Subtract the design-based treatment contribution under a stated effect assumption before forming the ICC.

Check Your Understanding

Answer each question using the model and balanced-design estimator above. Brief solutions follow.

What does the ICC $\rho$ describe in the random-intercept model?
Why can $MS_B-MS_W$ overstate cluster variance when treatment labels are masked?
A balanced hypothetical trial has $G=12$, $G_0=G_1=6$, $m=4$, $MS_B=15$, $MS_W=9$, and $\delta_0=1$ outcome unit. Calculate the adjusted cluster variance and ICC.
What should be concluded if the adjusted method-of-moments variance estimate is negative?
Why is the equal-cluster-size formula not automatically appropriate when cluster sizes differ?

Solutions. (1) It is the correlation between outcomes for two distinct participants in the same cluster. (2) The treatment effect is shared by all participants within an assigned cluster, so an arm difference contributes to pooled between-cluster variation. (3) The correction is $4(1^2)(6)(6)/[12(11)]=1.091$ outcome-units squared. Thus $\widehat{\sigma}_b^2=(15-9-1.091)/4=1.227$ outcome-units squared, and $\hat\rho=1.227/(1.227+9)=0.1200$. (4) It is not a valid variance or ICC; report the boundary issue and investigate sampling variability, the assumed effect, and model fit rather than interpreting a negative correlation. (5) Unequal sizes change the expected mean squares and their weighting, so a method accounting for the actual cluster sizes is required.