Cluster Randomized and Multi-Level Designs
Sample size planning for a two-arm multisite randomized trial with a continuous outcome when the treatment effect can vary from site to site. Uses the standardized hierarchical linear model of Raudenbush & Liu (2000) to account explicitly for treatment-by-site variance.
This calculator implements the balanced two-treatment multisite model described by Stephen W. Raudenbush and Xiaofeng Liu in Statistical Power and Optimal Design for Multisite Randomized Trials, published in Psychological Methods in 2000. Their framework treats participants as nested within sites and allows the treatment effect to vary randomly across sites.
The calculation is for the average treatment effect. The site-to-site variability relevant to this calculation is the treatment-by-site variance, denoted τ11. It is distinct from the variance of site means, τ00.
With the within-treatment, within-site residual variance standardized to 1, Raudenbush and Liu give the noncentrality parameter for testing the average treatment effect as:
where n is the total number of participants per site, J is the number of sites, γ10 is the standardized average treatment effect, and τ11 is the standardized treatment-by-site variance.
The test statistic follows a noncentral F distribution with 1 and J − 1 degrees of freedom. For a two-sided test at significance level α, the critical value is the upper α point of the corresponding central F distribution, and power is the upper-tail probability of the noncentral F distribution evaluated at that critical value.
When treatment effects vary more substantially across sites, the denominator of the noncentrality parameter becomes larger. Consequently, holding the effect size and participants per site constant, additional sites are needed to achieve the same power. This is one of the central design considerations in multisite trials.
The original paper emphasizes that increasing the number of sites can be particularly important when treatment effects vary across sites. It also notes that the appropriate balance between participants within sites and the number of sites depends on the purpose of the study and the relative costs of sampling at the two levels.
Raudenbush & Liu's Table 1 uses a standardized treatment effect of 0.20, treatment-by-site variance of 0.15, 8 participants per site, and 50 sites. Their reported power for the treatment main effect is approximately 0.405.
This implementation gives 0.4052 power for those inputs. With 49 sites, it gives 0.3982; therefore 50 sites is the first integer site count meeting a target power of 0.405 in this validation setup.
Raudenbush, S. W., & Liu, X. (2000). Statistical power and optimal design for multisite randomized trials. Psychological Methods, 5(2), 199–213. DOI: 10.1037/1082-989X.5.2.199.
this method is a sample-size and power-analysis software system whose documentation provides procedure-specific formulas, examples, and validation material. The present implementation uses the published Raudenbush–Liu multisite randomized-trial model as the mathematical basis for the calculation.