Why Tuning Changes the Evaluation Problem
Cross-validation can estimate how a prespecified model performs on new observations. Hyperparameter tuning changes the task: rather than evaluating one prespecified setting, the analyst compares several settings and chooses the one with the most favorable validation result. If that same result is then reported as the performance estimate, the evaluation data have helped select the model being evaluated. The reported score is therefore generally optimistic.
This issue arises whether the hyperparameter is a penalty strength, a number of selected genes, a tree depth, or another choice made using validation performance. It also arises when many candidate models are compared. Fold-wise scaling, feature selection, imputation, and other learned pipeline steps must remain within each training fold, as established by leakage-safe analysis pipelines. That prevents one kind of leakage, but it does not remove the selection optimism from choosing among candidates using the same cross-validation results that are reported.
The distinction is about information use, not merely about having two sets of fold labels. Within an outer split, the outer held-out observations must not influence which hyperparameter is chosen, how the pipeline is designed, or any other decision whose performance is being estimated. The outer data are reserved for evaluation after those decisions have been made using the corresponding outer training data.
Why the Best Cross-Validation Score Is Biased
Suppose a candidate setting \(j\) has true predictive performance \(S_j\), where larger values are better, and its cross-validation estimate is \(\hat S_j=S_j+\varepsilon_j\). The error \(\varepsilon_j\) represents the difference between the observed estimate and the underlying performance. Even if each candidate's estimate is centered correctly before selection, choosing the largest estimate tends to choose a candidate with a positive error.
The inequality follows because the maximum is a convex function: the expected maximum is at least the maximum of the expectations. Equality is possible in special cases, such as when there is only one candidate or when the estimates have no selection-relevant variation. In typical tuning, however, several noisy estimates compete. The more opportunities there are to select a favorable fluctuation, the more important the distinction between the winning score and the selected model's performance.
This argument does not say that cross-validation is inherently invalid, or that every selected score must be higher than the truth in every particular data set. It says that, over repeated applications under these conditions, the maximum observed score is not a neutral estimate of the selected model's performance. For a loss where smaller values are better, the same reasoning applies with the minimum observed loss: selection tends to favor a negative estimation error.
Worked Example: Selecting the Highest Accuracy
Suppose three penalty settings are compared using the same three inner folds. Their hypothetical fold accuracies are as follows.
| Penalty setting | Fold 1 | Fold 2 | Fold 3 | Mean accuracy |
|---|---|---|---|---|
| A | 0.72 | 0.76 | 0.72 | 0.7333 |
| B | 0.76 | 0.80 | 0.76 | 0.7733 |
| C | 0.72 | 0.76 | 0.76 | 0.7467 |
For setting B, the arithmetic is \((0.76+0.80+0.76)/3=2.32/3=0.7733\). The largest mean is 0.7733, so B is selected. If 0.7733 is then reported as an unbiased estimate of B's future accuracy, the reasoning is incomplete: the same fold results both chose B and supplied the reported score. B may truly be best, but its favorable observed score may also partly reflect favorable validation noise. These figures alone do not reveal the size of any optimism; that requires an evaluation not reused for selection.
What the Outer Loop Estimates
In nested cross-validation, each outer split sets aside observations for evaluation. The remaining observations form the outer training set. Hyperparameters are selected using cross-validation performed only within that outer training set; the full pipeline is then fitted on the outer training set using the selected setting, and evaluated on the outer held-out observations. The detailed sequence of operations is the subject of the algorithm treatment; the key principle here is that the outer observations have no role in choosing the setting they evaluate.
The resulting outer-fold scores estimate the performance of a procedure: given a training sample, search the candidate settings by inner cross-validation, select one, fit the pipeline, and predict new observations. They do not simply estimate the performance of a single hyperparameter chosen once on all available data. In ordinary \(K\)-fold outer validation, each fitted model uses roughly \((K-1)/K\) of the available observations. Thus the estimate is most directly relevant to that training size and to the specified tuning procedure.
This justification depends on the validation design matching the intended prediction task. Observations treated as independent must actually be independent at the relevant sampling unit; if people are related, repeated samples are present, or cohorts or sites define the intended generalization, the split must respect that structure. Outer and inner splits should represent compatible prediction targets. Otherwise the estimate may be honest about the wrong task. The full analysis pipeline, including preprocessing and any feature selection, must also be learned using only the training portion relevant to each evaluation.
Worked Example: Outer Evaluation After Inner Selection
Consider a hypothetical binary genomic prediction task with three outer folds. In each outer training set, inner cross-validation compares penalty settings \(\lambda=0.1, 1,\) and \(10\), and selects the setting with the highest inner accuracy. Suppose it selects \(1\), \(10\), and \(1\) in the respective outer folds. Each outer test set contains 20 individuals, and the selected pipeline makes 4, 5, and 3 errors, respectively.
The outer error rates are \(4/20=0.20\), \(5/20=0.25\), and \(3/20=0.15\). Since the test sets have equal sizes, their average error rate is
Equivalently, there are \(4+5+3=12\) errors among \(20+20+20=60\) outer-test predictions, giving \(12/60=0.20\). The result is an estimated 20% classification error for the procedure that uses inner cross-validation to choose among these settings and then fits the selected pipeline. It is not an estimate that the setting \(\lambda=1\) has 20% error in all cases: the selected setting varied, and each outer result evaluates the selection process on observations not used for that split's selection.
This interpretation assumes that the outer test individuals are independent of the corresponding training and selection decisions and represent the target population. If those individuals were used to choose the candidate grid, change preprocessing, or revise the model after seeing outer results, the separation would no longer hold for those decisions.
A Small Derivation of Winner's-Score Optimism
A simple model quantifies the selection effect. Assume two candidate settings have identical true loss \(L\), and their observed validation losses are \(L+\varepsilon_1\) and \(L+\varepsilon_2\), where the errors are independent \(N(0,\sigma^2)\). Selecting the smaller observed loss gives \(L+\min(\varepsilon_1,\varepsilon_2)\). Using \(\min(a,b)=(a+b-|a-b|)/2\),
Because \(\varepsilon_1-\varepsilon_2\sim N(0,2\sigma^2)\), the mean absolute value of this centered normal variable is \(2\sigma/\sqrt{\pi}\). Therefore
So, under these assumptions, the selected validation loss is lower than the common true loss by \(\sigma/\sqrt{\pi}\) on average. This calculation illustrates the mechanism, not a general correction formula: real candidate estimates can be correlated, have different true risks, and have non-normal errors. An independent outer evaluation avoids using the winning inner score as the estimate; it does not make the outer estimate noiseless.
When the Separation Fails
Nested validation is useful only if the outer results remain reserved for evaluation. After examining outer scores, repeatedly changing the feature set, hyperparameter grid, outcome definition, or analysis plan in response to them uses those scores as tuning information. A newly computed average over the same outer folds is no longer a fully independent assessment of all the choices made. A final model may still be built after development, but a fresh external evaluation is needed if a genuinely untouched assessment of those later decisions is required.
Other failure modes concern the split or the pipeline rather than the basic selection argument:
- Incompatible split units. A random split may put related individuals or repeated samples on both sides, making test performance too favorable for a new-family or new-person prediction target.
- Information crossing fold boundaries. If feature screening, imputation, scaling, or dimension reduction is learned using held-out observations, those observations have influenced the fitted pipeline. Keep these operations within the appropriate training data.
- Outer results drive later choices. Treating outer scores as feedback for repeated redesign turns an evaluation set into development data.
- Too few or unstable folds. Outer estimates can vary substantially with the particular partition, especially in small genomic cohorts. A single estimate should not be presented as exact or universal.
Diagnostics include auditing which observations informed each operation, checking that split units match the intended deployment setting, and examining whether the outer-fold results vary sharply. A large difference between the selected inner score and the outer estimate can be consistent with selection optimism, but it can also reflect different training sizes, sampling variation, or a shift in what the folds represent. It is not, by itself, proof of a particular cause.
How Nested Validation Compares With Alternatives
| Approach | Role of validation data | Interpretation and limitation |
|---|---|---|
| Tune and report the same cross-validation score | The folds both choose the setting and supply the reported score | Simple, but generally optimistic after selection; not a neutral estimate of the selected procedure's performance. |
| Nested cross-validation | Inner folds tune; outer folds evaluate | Estimates the tuning-and-fitting procedure under the chosen split design; uses data for both roles across nested splits and can be variable. |
| Independent external validation | A separate data set evaluates a model after development and tuning | Can assess performance in a distinct sample or setting, but requires appropriate external data and does not automatically represent every future population. |
Nested cross-validation is especially useful when data are limited and an external evaluation set is unavailable. It is not a substitute for careful target definition or an appropriate validation design, and its outer scores should not be conflated with the final model's exact future performance. After evaluation, the chosen development strategy can be applied to all available development data to produce a model for use; the nested estimate describes the strategy under the outer training sizes, rather than certifying that particular refitted model.
Check Your Understanding
Answer each question, then compare with the solutions.
- Two settings have observed cross-validation accuracies of 0.78 and 0.74. Which setting is selected if accuracy is the sole tuning criterion, and why is 0.78 not automatically an unbiased estimate of its accuracy?
- In a nested design, may the outer test observations be used to choose the penalty setting for that outer fold? Explain.
- Suppose two settings have the same true loss and independent normal validation errors with standard deviation \(\sigma=0.06\). What is the expected amount by which the lower of their two observed losses falls below the common true loss?
- Why can an honest outer-loop estimate still be unsuitable for predicting performance in a new hospital?
Solutions. (1) Select the setting with accuracy 0.78, because it has the larger observed accuracy. That same score participated in the selection, so favorable validation noise may have helped it win; a separate evaluation is needed to estimate performance without that reuse. (2) No. The outer test observations must not inform the setting choice; inner validation within the outer training data makes that choice. (3) The expected amount is \(\sigma/\sqrt{\pi}=0.06/\sqrt{\pi}\approx0.0339\) loss units, by the two-candidate derivation above. (4) The outer split may estimate performance for individuals drawn like the study participants, not for a different hospital population. A change in site or population can alter the prediction task, so the split design and target population must match the intended use.