Comparing Two ROC Curves (Independent) Sample Size

Methodology

This calculator compares two ROC AUCs when the diagnostic tests are evaluated in independent participant samples. The null hypothesis is H₀: AUC₁ = AUC₂, and the alternative hypothesis is that the two AUCs differ. Because the participant samples are independent, the covariance between the two AUC estimates is zero.

The AUC variance approximation follows Hanley and McNeil's method. For an AUC denoted by θ, with n₊ disease-positive subjects and n₋ disease-negative subjects:

For independent ROC curves, the variance of the difference is the sum of the two individual AUC variances:

The sample-size calculation uses a normal approximation with the variance under the null hypothesis and under the specified alternative. For the null variance, the common AUC is taken as the midpoint of the two anticipated AUCs. For a two-sided test:

Here V₀ and V_A are the asymptotic variance components per disease-positive participant after accounting for the specified controls-to-cases ratio. The calculation is then rounded upward to the next whole disease-positive participant. The corresponding number of disease-negative participants is obtained from the allocation ratio.

Independent versus paired ROC curves

This calculator is specifically for two independent participant samples. For example, Test 1 might be evaluated in one cohort and Test 2 in a separate cohort. It should not be used when both tests are measured on the same patients. In the paired setting, the covariance between the two AUC estimates can materially affect the required sample size and a paired ROC method should be used instead.

PASS documentation for its two-ROC sample-size procedure describes the general AUC-comparison framework and gives the Obuchowski–McClish methodology for discrete ratings and Hanley–McNeil methodology for continuous criterion values. The PASS two-ROC procedure itself is designed around tests obtained from the same patients, so this calculator removes the covariance term for the independent-groups design.

Worked validation example

A worked numerical check uses Test 1 AUC = 0.80, Test 2 AUC = 0.90, α = 0.05, 80% power, a two-sided test, and a 1:1 control-to-case ratio within each independent test arm. Applying the Hanley–McNeil variance components gives a required sample size of 118 disease-positive and 118 disease-negative participants per test arm, or 236 evaluable participants per arm and 472 evaluable participants overall. With these values, the normal-approximation calculation reproduces the target sample-size result.

Assumptions and limitations

• The two ROC curves come from independent participant samples.
• The endpoint being compared is the full AUC, not a partial AUC.
• The Hanley–McNeil large-sample variance approximation is used.
• The two diagnostic tests have the specified anticipated AUCs.
• The case/control allocation ratio is the same in both independent test arms.
• The normal approximation may be less reliable for very small samples or AUCs close to 0.5 or 1.0.
• This calculator does not account for clustering, multiple readers, repeated lesions, paired measurements, or complex multi-reader multi-case designs.
• Dropout inflation is a simple enrollment adjustment and assumes the anticipated unusable-data percentage applies equally to the required evaluable sample.

References

Hanley, J. A., & McNeil, B. J. (1982). The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology, 143(1), 29–36.

Obuchowski, N. A. (1998). Sample size calculations in studies of test accuracy. Statistical Methods in Medical Research, 7(4), 371–392.

NCSS / PASS. Tests for Two ROC Curves. PASS Sample Size Software documentation. The documentation describes the AUC-comparison framework and the Obuchowski–McClish and Hanley–McNeil variance approaches.