Sample Size for Diagnostic Accuracy Studies

Introduction

Diagnostic accuracy studies evaluate how well a diagnostic test identifies a target condition compared with an appropriate reference standard. The central quantities are usually sensitivity and specificity, although positive predictive value, negative predictive value, likelihood ratios, and area under the ROC curve may also be important.

Sample size planning for diagnostic accuracy studies is different from sample size planning for a conventional parallel-group clinical trial. The investigator is usually not asking how many patients are needed to detect a treatment difference. Instead, the objective is often to estimate a diagnostic accuracy parameter with a prespecified level of precision.

For example, the study might require the lower bound of a 95% confidence interval for sensitivity to be sufficiently close to the expected sensitivity. That requirement determines how many subjects with the target condition are needed.

Key idea: Sensitivity is estimated from subjects who truly have the condition, while specificity is estimated from subjects who do not have the condition. Therefore, diagnostic accuracy sample size is fundamentally driven by the number of diseased and non-diseased subjects, not simply by the total sample size.

The Diagnostic Accuracy 2 × 2 Table

Suppose a binary diagnostic test is compared with a reference standard. The resulting data can be summarized as:

	Reference Standard Positive	Reference Standard Negative
Test Positive	True Positive (TP)	False Positive (FP)
Test Negative	False Negative (FN)	True Negative (TN)

Sensitivity is:

Se=P(T^+\mid D^+) = \frac{TP}{TP+FN}

Specificity is:

Sp=P(T^-\mid D^-) = \frac{TN}{TN+FP}

The denominators are important. Sensitivity depends on the number of diseased subjects. Specificity depends on the number of non-diseased subjects.

Why Total Sample Size Alone Is Misleading

Consider a diagnostic test for a disease whose prevalence is only 10%. If a study enrolls 500 subjects, the expected number with disease is only:

$$ 500(0.10)=50 $$

The study therefore has approximately 50 subjects contributing information to the sensitivity estimate, while approximately 450 subjects contribute information to specificity.

If sensitivity is the more demanding parameter, increasing the total sample size may be necessary simply to obtain enough diseased subjects.

Diagnostic accuracy planning is asymmetric: a study can have hundreds or thousands of participants and still provide poor precision for sensitivity if the target condition is uncommon.

Precision-Based Sample Size for Sensitivity

Suppose the anticipated sensitivity is $$Se$$ , and the investigator wants a two-sided confidence interval with half-width $$d$$ . Using the normal approximation to a binomial proportion, the required number of diseased subjects is approximately:

n_D = \frac{Z_{1-\alpha/2}^{\,2}Se(1-Se)} {d^2}

where:

$$n_D$$ = required number of diseased subjects
$$Se$$ = anticipated sensitivity
$$d$$ = desired half-width of the confidence interval
$Z_{1-\alpha/2}$ = standard normal critical value

For a 95% confidence interval:

Z_{0.975}=1.96

Precision-Based Sample Size for Specificity

The corresponding approximation for specificity is:

n_{ND} = \frac{Z_{1-\alpha/2}^{\,2}Sp(1-Sp)} {d^2}

where $n_{ND}$ is the required number of non-diseased subjects.

The formulas for sensitivity and specificity have exactly the same mathematical structure because both are binomial proportions. The important difference is the population from which each proportion is calculated.

A Complete Worked Example

Suppose a new diagnostic blood test is being developed to identify a clinically important disease. The investigators expect:

Parameter	Planning Assumption
Expected sensitivity	90%
Expected specificity	95%
Confidence level	95%
Desired half-width	5 percentage points
Expected prevalence	20%

The desired precision means that the investigator wants the approximate 95% confidence interval to extend no more than 0.05 on either side of the estimated sensitivity and specificity.

Step 1: Calculate the Number of Diseased Subjects

Using the sensitivity formula:

n_D = \frac{1.96^2(0.90)(1-0.90)} {0.05^2}

Therefore:

n_D = \frac{3.8416(0.90)(0.10)} {0.0025}

$$ n_D = 138.30 $$

Because sample size must be an integer, we round upward:

\boxed{n_D=139}

Approximately 139 diseased subjects are therefore required to estimate a sensitivity of 90% with approximately 5 percentage points of precision using this normal-approximation calculation.

Step 2: Calculate the Number of Non-Diseased Subjects

For specificity:

n_{ND} = \frac{1.96^2(0.95)(1-0.95)} {0.05^2}

Therefore:

n_{ND} = \frac{3.8416(0.95)(0.05)} {0.0025}

n_{ND} = 73.00

Thus:

\boxed{n_{ND}=73}

Only about 73 non-diseased subjects are required to achieve the specified approximate precision for specificity under these assumptions.

Step 3: Convert Diseased Subjects Into Total Sample Size

The expected disease prevalence is 20%. Therefore:

$$ P(D^+)=0.20 $$

To obtain 139 diseased subjects, the total number of enrolled participants must be:

N_D= \frac{139}{0.20} = 695

Thus, approximately 695 total participants are needed to obtain 139 diseased subjects if the disease prevalence is approximately 20%.

At this total sample size, the expected number of non-diseased subjects is:

$$ 695(1-0.20)=556 $$

The specificity requirement needs only 73 non-diseased subjects, so the sensitivity requirement is clearly the limiting component.

Result: Under the planning assumptions above, the sensitivity requirement drives the sample size. Approximately 695 total subjects are needed to obtain about 139 diseased subjects at a 20% prevalence.

Why You Must Check Both Requirements

The total sample size should satisfy both the sensitivity and specificity precision requirements. A convenient way to express this is:

N_{Se} = \frac{n_D}{P(D^+)}

and:

N_{Sp} = \frac{n_{ND}}{1-P(D^+)}

The final total sample size is approximately:

N=\max(N_{Se},N_{Sp})

For the worked example:

Requirement	Required Subjects	Prevalence Component	Approximate Total N
Sensitivity	139 diseased	20%	695
Specificity	73 non-diseased	80%	92
Final requirement	Take the larger value		695

This calculation illustrates an important principle: the total sample size is not determined by simply adding the sensitivity and specificity sample sizes. They refer to different subsets of participants.

The Role of Disease Prevalence

Prevalence can have a dramatic effect on total sample size. Suppose the study still requires 139 diseased subjects, but the expected prevalence falls from 20% to 5%. The total sample size becomes:

N= \frac{139}{0.05} = 2780

The number of diseased subjects has not changed. The difference is that substantially more people must be screened or enrolled to obtain the required number of diseased participants.

Expected Prevalence	Required Diseased Subjects	Approximate Total N
50%	139	278
25%	139	556
20%	139	695
10%	139	1,390
5%	139	2,780

This is why diagnostic accuracy studies for uncommon diseases can require very large screening populations even when the number of required diseased subjects itself is relatively modest.

Prevalence is a planning assumption, not merely a descriptive statistic. If the actual prevalence is substantially lower than expected, the study may fail to obtain enough diseased participants and therefore fail to achieve its intended precision for sensitivity.

How Confidence Interval Width Changes Sample Size

The desired precision has an especially strong effect on sample size because the half-width appears in the denominator squared:

n\propto\frac{1}{d^2}

Therefore, reducing the desired half-width from 5 percentage points to 2.5 percentage points approximately quadruples the required sample size.

For sensitivity of 90%:

Desired Half-Width	Approximate Diseased Subjects
10 percentage points	35
7.5 percentage points	62
5 percentage points	139
2.5 percentage points	554

The relationship is not perfectly intuitive at first. A study designed to estimate sensitivity within ±2.5 percentage points is asking for substantially more information than one designed for ±5 percentage points.

Confidence Level Also Matters

The normal approximation contains the critical value:

Z_{1-\alpha/2}

Increasing the confidence level therefore increases the required sample size. For example:

Confidence Level	Critical Value
90%	1.645
95%	1.960
99%	2.576

A 99% confidence interval requires more information than a 95% confidence interval if the desired precision is held constant.

Why the Expected Sensitivity or Specificity Matters

The binomial variance is:

$$ p(1-p) $$

This quantity is largest when $$p=0.5$$ . Consequently, if the anticipated sensitivity is uncertain, the required sample size can be conservative when planning around a value near 50%.

For example:

Expected Sensitivity	Approximate n for ±5%
70%	323
80%	246
90%	139
95%	73
99%	15

The sample size becomes smaller as the anticipated sensitivity approaches 100%. However, this does not mean that a study expecting 99% sensitivity automatically requires only 15 diseased subjects. The normal approximation can be inadequate with very high proportions or small sample sizes, and confidence interval methodology becomes important.

Normal Approximation vs. Exact and Score-Based Methods

The simple formula used above is based on the normal approximation to a binomial proportion. It is useful for understanding the basic sample size relationship and is often adequate for planning calculations when the sample size is sufficiently large.

However, diagnostic accuracy studies frequently involve sensitivities or specificities close to 0 or 1. In these situations, the normal approximation can perform poorly.

Alternative approaches include:

Wilson score confidence intervals
Exact binomial confidence intervals
Clopper–Pearson methods
Other score-based or likelihood-based approaches

The sample size method should generally match the confidence interval method that will be used for the primary analysis.

Practical point: Do not automatically use the simple Wald formula when the expected sensitivity or specificity is very high, the sample size is small, or the anticipated number of false negatives or false positives is very low. A more exact planning method may be preferable.

Planning Around the Lower Confidence Bound

Some diagnostic studies are designed around a more stringent criterion than simply requiring a confidence interval of a certain width. For example, the study might require the lower confidence bound for sensitivity to exceed 85%.

Suppose the expected sensitivity is 95% and the objective is to demonstrate that the lower confidence bound is above 90%. This is a different statistical problem from simply estimating sensitivity with a ±5% confidence interval.

The sample size calculation must reflect the specific hypothesis or confidence bound being evaluated.

H_0:Se\le Se_0 \qquad\text{vs.}\qquad H_A:Se>Se_0

where $$Se_0$$ is the minimum acceptable sensitivity.

This distinction is important because diagnostic accuracy sample size calculations can be either:

Precision-based: estimate sensitivity or specificity with a desired confidence interval width.
Hypothesis-based: demonstrate that sensitivity or specificity exceeds a prespecified threshold.

Precision-Based vs. Hypothesis-Based Designs

Feature	Precision-Based	Hypothesis-Based
Primary objective	Estimate accuracy precisely	Demonstrate accuracy exceeds a threshold
Typical requirement	Confidence interval width	Power and significance level
Key input	Expected accuracy and desired precision	Null and alternative accuracy
Example	95% CI within ±5%	Show sensitivity >90%
Common context	Estimation and validation	Regulatory or performance claims

Before calculating sample size, the statistical objective should therefore be explicitly stated. A calculation can be mathematically correct but inappropriate if it answers the wrong design question.

Verification of the Reference Standard

Diagnostic accuracy studies depend critically on knowing the true disease status. That status is typically established using a reference standard.

If every participant receives the reference standard, sensitivity and specificity can be estimated directly. But in some studies, the reference standard may be invasive, expensive, or otherwise difficult to administer.

This can create a verification problem. For example, investigators might perform the definitive reference procedure only for subjects with a positive index test.

That design can introduce partial verification bias because the reference-standard result is not observed equally across test-positive and test-negative participants.

Sample size cannot repair verification bias. A very large diagnostic accuracy study can still produce biased sensitivity and specificity estimates if the reference-standard strategy is systematically different across groups.

Accounting for Uninterpretable or Missing Reference Standards

Suppose the calculation requires 695 evaluable participants, but the investigators expect 8% of enrolled participants to have missing or unusable reference-standard results. The inflation factor is:

N_{enroll} = \frac{N_{evaluable}}{1-r}

where $$r$$ is the expected proportion of non-evaluable participants.

For an 8% non-evaluable rate:

N_{enroll} = \frac{695}{1-0.08} = 755.43

Rounding upward gives:

\boxed{N_{enroll}=756}

Thus, approximately 756 participants should be enrolled to obtain about 695 evaluable subjects if 8% are expected to be unusable.

Why Simple "Dropout Inflation" Can Be Inadequate

In a conventional randomized trial, dropout is often considered as a percentage of the overall sample. Diagnostic accuracy studies can be more complicated because missingness may affect diseased and non-diseased subjects differently.

For example, if subjects with positive index-test results are much more likely to complete the reference standard than subjects with negative results, the resulting verification pattern can introduce bias.

Therefore, sample size planning should consider:

Overall non-evaluable rate
Expected disease prevalence
Reference-standard completion
Index-test failure rate
Indeterminate test results
Whether missingness differs by disease status

When Prevalence Is Very Low

Suppose the target disease has a prevalence of only 2%. If 200 diseased subjects are required:

N= \frac{200}{0.02} = 10{,}000

A prospective cohort study would therefore require approximately 10,000 participants to obtain 200 diseased subjects if the prevalence estimate is correct.

This may be impractical. One possible alternative is an enriched or case-control-type design in which diseased and non-diseased subjects are sampled separately.

However, this changes what can be estimated directly. In particular, predictive values depend on prevalence and generally cannot be interpreted from an artificially enriched sample as though the sample reflected the target clinical population.

Important distinction: Sensitivity and specificity are generally not dependent on disease prevalence, whereas positive and negative predictive values are strongly prevalence-dependent. The sampling design therefore matters greatly depending on which diagnostic performance measure is the primary objective.

Sensitivity, Specificity, and Predictive Values

Positive predictive value is:

PPV = P(D^+\mid T^+) = \frac{TP}{TP+FP}

Negative predictive value is:

NPV = P(D^-\mid T^-) = \frac{TN}{TN+FN}

Unlike sensitivity and specificity, these quantities depend heavily on the prevalence of disease.

For example, a test can have excellent sensitivity and specificity but a relatively low PPV when the target disease is very uncommon.

This is one reason diagnostic accuracy studies should clearly distinguish between intrinsic test characteristics such as sensitivity and specificity and population-dependent predictive values.

ROC Curves and Sample Size

Many diagnostic tests produce a continuous measurement rather than a simple positive/negative result. A threshold is then selected to classify patients.

Different thresholds produce different pairs of sensitivity and specificity, which can be represented using a receiver operating characteristic (ROC) curve.

The area under the ROC curve (AUC) is another commonly used summary measure:

$$ AUC=P(T^+>T^-) $$

where the expression can be interpreted as the probability that a randomly chosen diseased subject has a higher test measurement than a randomly chosen non-diseased subject, assuming higher values indicate greater evidence of disease.

Sample size planning for AUC is different from planning for a single sensitivity or specificity estimate. It depends on the anticipated AUC, the null or alternative AUC, the allocation between diseased and non-diseased subjects, and the desired precision or power.

Paired Diagnostic Tests

Sometimes the goal is not merely to estimate one diagnostic test's accuracy. Instead, investigators want to compare two diagnostic tests in the same subjects.

For example, a new biomarker may be compared with an existing diagnostic assay, with both tests evaluated against the same reference standard.

This creates a paired design because each participant contributes results for both tests. The correlation between tests must be accounted for when planning the study.

The sample size calculation for comparing sensitivities or specificities is therefore not simply the same as calculating the sample size for one proportion and doubling it. The joint distribution of the paired test results matters.

Design principle: When the scientific question is a comparison between diagnostic tests, the sample size should be based on the planned comparison rather than on the precision of each test considered independently.

A Useful Planning Table

For a study expecting 90% sensitivity and 95% specificity with a 95% confidence level and ±5 percentage-point precision, the approximate required numbers of subjects can be summarized as follows:

Expected Prevalence	Diseased Needed	Non-Diseased Needed	Total N Driven by Sensitivity
50%	139	73	278
25%	139	73	556
20%	139	73	695
10%	139	73	1,390
5%	139	73	2,780

This table demonstrates why prevalence should be treated as a major design parameter. The same diagnostic test can require radically different enrollment targets depending on the population in which the study is conducted.

Choosing the Primary Accuracy Endpoint

A diagnostic study should identify the primary accuracy endpoint before the sample size is finalized. Potential primary endpoints include:

Sensitivity
Specificity
Sensitivity and specificity jointly
Positive predictive value
Negative predictive value
Area under the ROC curve
Likelihood ratios
Diagnostic agreement

If both sensitivity and specificity are important, both should be incorporated into the sample size calculation. It is not sufficient to calculate the sample size using whichever parameter happens to produce the smaller number.

Sample Size Planning in R

The basic precision calculation can be implemented directly in R. For the worked example:

alpha <- 0.05
z <- qnorm(1 - alpha / 2)

Se <- 0.90
Sp <- 0.95
d  <- 0.05

n.diseased <- ceiling(
  z^2 * Se * (1 - Se) / d^2
)

n.nondiseased <- ceiling(
  z^2 * Sp * (1 - Sp) / d^2
)

n.diseased
n.nondiseased

The resulting values are approximately:

n.diseased
# 139

n.nondiseased
# 73

The prevalence can then be incorporated:

prevalence <- 0.20

N.sensitivity <- ceiling(
  n.diseased / prevalence
)

N.specificity <- ceiling(
  n.nondiseased / (1 - prevalence)
)

N <- max(
  N.sensitivity,
  N.specificity
)

N

This gives:

N.sensitivity
# 695

N.specificity
# 92

N
# 695

If an 8% non-evaluable rate is anticipated:

non_evaluable <- 0.08

N.enroll <- ceiling(
  N / (1 - non_evaluable)
)

N.enroll
# 756

Planning With Expected Disease Counts Instead of Prevalence

Not every diagnostic accuracy study uses a simple prevalence-based cohort design. In some settings, the study may deliberately recruit a target number of diseased and non-diseased subjects.

For example, a protocol might specify:

Population	Target Number
Diseased subjects	200
Non-diseased subjects	200
Total evaluable subjects	400

This can be useful when the disease is uncommon and a prevalence-based cohort would be inefficient. The design, however, must remain aligned with the intended estimand and target population.

If predictive values or population-level classification performance are primary objectives, artificially changing the disease prevalence in the study sample requires additional care.

Verification Bias and Spectrum Bias

Two important sources of bias deserve particular attention.

Verification Bias

Verification bias occurs when disease status is not established using the same reference-standard strategy for all relevant participants. Partial verification can bias sensitivity and specificity estimates.

Spectrum Bias

Diagnostic performance can also depend on the clinical spectrum of patients. A study consisting entirely of obvious severe cases and exceptionally healthy controls may produce a different sensitivity and specificity than a study conducted in the intended clinical population.

Therefore, sample size alone does not determine whether a diagnostic accuracy study is clinically informative. The enrolled population must represent the population in which the test will actually be used.

Common Sample Size Mistakes

Calculating total N without considering prevalence. Sensitivity precision depends on the number of diseased subjects.
Using sensitivity sample size to justify specificity precision. The two parameters are estimated from different subsets of participants.
Ignoring the reference standard. A diagnostic accuracy estimate is only as credible as the method used to establish true disease status.
Using a Wald formula automatically for small samples. Exact or score-based confidence intervals may be more appropriate.
Ignoring non-evaluable participants. Missing reference-standard results can reduce the effective sample size.
Assuming prevalence will match the planning value. A lower-than-expected prevalence can substantially reduce the number of diseased subjects obtained.
Confusing diagnostic accuracy with predictive value. PPV and NPV depend on disease prevalence.
Ignoring the paired structure when comparing two diagnostic tests. Comparative diagnostic accuracy studies require methods that account for the within-subject pairing.
Choosing a sample size simply because it looks large. A large sample is not necessarily informative if the study population or reference-standard strategy is inappropriate.

A Practical Diagnostic Accuracy Sample Size Workflow

A useful workflow is:

Define the target population and clinical use case.
Define the reference standard for disease status.
Specify the primary diagnostic accuracy endpoint.
Specify the anticipated sensitivity and/or specificity.
Specify the confidence level.
Specify the desired confidence interval precision or hypothesis-testing objective.
Calculate the required number of diseased subjects.
Calculate the required number of non-diseased subjects.
Incorporate expected disease prevalence if using a prospective cohort design.
Inflate for non-evaluable subjects and reference-standard failures.
Assess whether verification or spectrum bias could compromise the study.
Prespecify the primary analysis and confidence interval method.

What a Protocol Should State

A diagnostic accuracy protocol should make the sample size assumptions transparent. At minimum, the statistical section should identify:

Primary diagnostic accuracy endpoint
Expected sensitivity and/or specificity
Target confidence level
Desired confidence interval width or precision
Expected disease prevalence, if applicable
Required number of diseased subjects
Required number of non-diseased subjects
Total evaluable sample size
Expected proportion of non-evaluable subjects
Total planned enrollment
Reference-standard strategy
Primary confidence interval methodology

This allows reviewers to reconstruct the sample size calculation and determine whether the study is adequately powered or precise for its stated objective.

How Sample Size Changes With the Target Precision

The most important conceptual relationship is:

n \approx \frac{Z^2p(1-p)}{d^2}

There are three major drivers:

Confidence level: higher confidence requires more subjects.
Expected accuracy: values near 50% produce the greatest binomial variance.
Precision: narrower confidence intervals require substantially more subjects.

For diagnostic studies, a fourth quantity becomes especially important: prevalence determines how many total participants are needed to obtain the required number of diseased subjects.

The central planning equation: first determine how many diseased and non-diseased subjects are required for the desired precision, and only then determine how many participants must be enrolled to obtain those subjects.

Summary

Sample size planning for diagnostic accuracy studies is fundamentally different from sample size planning for a conventional randomized clinical trial. The primary objective is often to estimate sensitivity and specificity with adequate precision rather than to detect a treatment difference.

Sensitivity is calculated among subjects who truly have the disease, while specificity is calculated among subjects who do not. Consequently, the sample size calculation should first determine the required numbers of diseased and non-diseased subjects.

Using the normal approximation, the required number of diseased subjects for sensitivity is approximately:

n_D = \frac{Z_{1-\alpha/2}^{\,2}Se(1-Se)} {d^2}

and the required number of non-diseased subjects for specificity is:

n_{ND} = \frac{Z_{1-\alpha/2}^{\,2}Sp(1-Sp)} {d^2}

If the study uses a prospective cohort design, disease prevalence then converts these requirements into a total enrollment target.

In the worked example, an expected sensitivity of 90%, specificity of 95%, a 95% confidence level, and ±5 percentage-point precision required approximately 139 diseased and 73 non-diseased subjects. At a 20% disease prevalence, the sensitivity requirement translated into approximately 695 evaluable participants. After allowing for an 8% non-evaluable rate, approximately 756 participants would need to be enrolled.

The most important practical lesson is that diagnostic accuracy sample size is not simply about obtaining a large total number of subjects. The study must contain enough information in both the diseased and non-diseased populations, use an appropriate reference standard, and preserve the clinical spectrum in which the test will ultimately be used.

Bottom line: For diagnostic accuracy studies, determine the required number of diseased and non-diseased subjects from the desired precision or hypothesis-testing objective, then translate those requirements into total enrollment using the expected prevalence and appropriate inflation for non-evaluable participants. For small samples or sensitivities and specificities near 0 or 1, use an appropriate exact or score-based confidence interval method rather than relying blindly on the simple normal approximation.

References

Buderer, N.M.F. (1996). Statistical methodology: I. Incorporating the prevalence of disease into the sample size calculation for sensitivity and specificity. Academic Emergency Medicine, 3(9), 895–900.
Flahault, A., Cadilhac, M. & Thomas, G. (2005). Sample size calculation should be performed for design accuracy in diagnostic test studies. Journal of Clinical Epidemiology, 58(8), 859–862.
Hanley, J.A. & McNeil, B.J. (1982). The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology, 143(1), 29–36.
Obuchowski, N.A. (1998). Sample size calculations in studies of test accuracy. Statistical Methods in Medical Research, 7(4), 371–392.
Pepe, M.S. (2003). The Statistical Evaluation of Medical Tests for Classification and Prediction. Oxford University Press.
Zhou, X.-H., Obuchowski, N.A. & McClish, D.K. (2011). Statistical Methods in Diagnostic Medicine, 2nd ed. Wiley.