Youden’s J as a Threshold Criterion

What Youden’s J Measures

A classification threshold determines which cases receive a positive prediction. Raising the threshold for a score where higher values indicate stronger evidence of the positive class usually reduces false positives, but may also miss more actual positives. Youden’s J summarizes the resulting trade-off using two class-specific rates: sensitivity and specificity.

Unlike a measure that counts correct classifications across the whole sample, J gives sensitivity and specificity equal weight. That makes it useful for comparing candidate operating points when the goal is to balance the two rates. It does not, by itself, establish that this balance matches the costs, prevalence, or practical needs of a particular decision.

Assume throughout that higher scores indicate a greater likelihood of the positive class and that a case is classified as positive when its score is at least the threshold \(c\). For actual positives \(P\) and actual negatives \(N\), let TP, FN, TN, and FP denote the true positives, false negatives, true negatives, and false positives. When both actual classes are present,

The formulas use different denominators: sensitivity is calculated among actual positives, and specificity among actual negatives. They are not the proportions of predictions that are correct. The value of J ranges from \(-1\) to \(1\). A threshold that classifies every case positive has sensitivity \(1\), specificity \(0\), and \(J=0\); a threshold that classifies every case negative also has \(J=0\).

Why Maximizing J Is a Balanced-Accuracy Criterion

Balanced accuracy is the arithmetic mean of sensitivity and specificity. The relation between it and J follows directly from the definitions:

Therefore, maximizing J across candidate thresholds gives exactly the same maximizing threshold as maximizing balanced accuracy. The transformation from J to balanced accuracy is increasing: if one threshold has a larger J than another, it also has a larger balanced accuracy.

There is also a geometric interpretation. On an ROC plot, the horizontal coordinate is FPR and the vertical coordinate is TPR. At a threshold,

A line from the ROC origin to the point \((\text{FPR},\text{TPR})\) has slope \(\text{TPR}/\text{FPR}\), when FPR is nonzero, but J is not that slope. Instead, J is the point’s vertical distance above the diagonal line \(\text{TPR}=\text{FPR}\): at a given FPR, the diagonal’s height is FPR, so the vertical difference is TPR minus FPR. Maximizing J selects the ROC point with the greatest such separation from the diagonal.

Calculating J Across Candidate Thresholds

To select a threshold empirically, calculate sensitivity and specificity at each candidate threshold using the same validation data and the same positive-class definition. Then calculate J for each row and identify the largest value. If thresholds tie for the maximum, J alone does not distinguish them; another decision principle is needed to choose between their operating points.

What the Criterion Does Not Account For

A maximum J is not a universal definition of the best threshold. It gives equal weight to sensitivity and specificity, but it does not encode the consequences of errors in a particular application. If a false negative is substantially more costly than a false positive, a threshold that emphasizes sensitivity may be preferred even if its J is lower. Conversely, a review process with limited resources may not be able to act on every positive prediction. J does not impose a limit on the number of cases flagged.

J is built from class-conditional rates, not from the proportion of cases that are positive in the target population. Consequently, its formula does not explicitly include prevalence. However, prevalence can affect precision and the number of positive predictions in use; a threshold chosen by maximizing J does not guarantee a high positive predictive value. Nor does J evaluate whether predicted probabilities are calibrated. It evaluates a classification operating point, not every property of a predictive model.

The maximizing threshold is also estimated from data. If candidate thresholds produce similar J values, sampling variation may change their order in another sample. Report the operating rates and the validation context rather than presenting the selected threshold as exact or certain. A threshold selected on one data set may not maintain the same sensitivity and specificity in a different population.

A Common Calculation Trap

Specificity is not the false-positive rate. They are complements: \(\text{FPR}=1-\text{specificity}\). For the threshold 0.75 in the claim-review example, sensitivity is 0.70, specificity is 0.80, and FPR is 0.20. The correct calculation is

A wrong calculation would add sensitivity to FPR and subtract one: \(0.70+0.20-1=-0.10\). That substitutes the error rate for the correct-classification rate. If using ROC coordinates, subtract FPR from TPR; if using sensitivity and specificity, add those two rates and subtract one.

When comparing close thresholds, calculate with the underlying fractions or retain sufficient precision until the comparison is complete. Rounding sensitivity and specificity too early can create or conceal a tie. If two candidate thresholds have exactly the same maximum J, state that the criterion leaves a tie rather than arbitrarily claiming a unique optimum.