Comparing Two ROC Curves (Paired) Sample Size Calculator

Methodology

This calculator implements the paired discrete (rating) data branch of the PASS Tests for Two ROC Curves procedure. PASS states that this branch uses the sample-size methodology of Obuchowski and McClish (1997), while the continuous-data branch uses a different Hanley-McNeil formulation. The paired design accounts for the correlation between the two diagnostic-test scores within the same positive and negative subjects.

Hypothesis and sample-size equation

Let Δ denote the difference between the two anticipated AUCs. PASS uses the large-sample normal approximation with separate variance expressions under the null and alternative hypotheses. For the number of positive (diseased) subjects:

For a two-sided test, z_α is the standard-normal quantile at 1 − α/2. For a one-sided test, it is the quantile at 1 − α. The power term is z_β = Φ⁻¹(power).

Paired variance calculation

PASS defines the alternative variance as the sum of the two AUC variances minus twice their covariance:

The covariance incorporates the correlation between the two tests among positive subjects (r+) and negative subjects (r−). The calculation therefore reflects the efficiency gained from evaluating both tests on the same individuals rather than treating the two AUC estimates as independent.

Binormal parameterization used by PASS

For the full ROC range and the conservative variance-ratio specification B₁ = B₂ = 1 used by the PASS examples, the binormal separation parameter corresponding to an AUC is:

The implementation evaluates the Obuchowski-McClish variance and covariance expressions directly in JavaScript and then rounds the required group sizes upward, because a fraction of a subject cannot be enrolled.

Important scope

• This calculator is for a paired design: the same subjects receive both diagnostic tests.
• It uses the full AUC, from false-positive rate 0 to 1.
• It implements the PASS discrete/rating-data formulation with B₁ = B₂ = 1.
• For continuous marker data, PASS documents a different Hanley-McNeil variance formulation; that method is not silently substituted here.
• The allocation ratio R is defined as N−/N+.

Validation example

The PASS documentation provides a sample-size example with AUC₁ = 0.800, AUC₂ = 0.825, two-sided α = 0.05, power = 0.90, R = 2, and correlations of 0.60 in both the positive and negative groups. PASS reports N+ = 1,937 and N− = 3,874, for a total of 5,811 subjects.

References

Obuchowski, N. A. & McClish, D. K. (1997). Sample size determination for diagnostic accuracy studies involving binormal ROC curve indices. Statistics in Medicine, 16(13), 1529–1542.

Hanley, J. A. & McNeil, B. J. (1983). A method of comparing the areas under receiver operating characteristic curves derived from the same cases. Radiology, 148, 839–843.

PASS Sample Size Software, Tests for Two ROC Curves, NCSS documentation. The PASS procedure explicitly identifies the Obuchowski-McClish methodology for discrete rating data and supplies the validation example reproduced above.