Designing a Nested Validation Study for a Learning Pipeline

Why the Whole Pipeline Must Be Evaluated

A learning pipeline is more than a fitted prediction rule. It can include imputation, scaling, feature construction, model-class choice, hyperparameter tuning, and a final refit. If any of these steps use information from an outer test fold, that fold is no longer an independent assessment of the procedure. A defensible validation plan therefore specifies the data split and the complete sequence of operations before examining outer-fold performance.

The target is usually the performance of a learning procedure: the rule that takes a training sample and returns a fitted predictor. Nested validation evaluates that procedure, including its data-driven choices, at a particular training-set size and under a particular split design. It does not directly give an unbiased evaluation of the single final model later fitted to all available data.

Definition: Let $Z=(X,Y)$ denote a future observation and let $P$ be its target distribution. A learning procedure $\mathcal A_m$ takes a training sample of size $m$, performs all specified preprocessing, model selection, and tuning, and returns a predictor. For a loss function $L$, its expected risk at training size $m$ is $$ R_m(\mathcal A)=E_{D_m,U}\left[E_{Z\sim P}\left\{L\bigl(Y,\mathcal A_m(D_m,U)(X)\bigr)\mid D_m,U\right\}\right], $$ where $D_m$ is a training sample from the target sampling scheme and $U$ represents any randomness used by the procedure, such as algorithm initialization or randomized search.

This definition makes the target explicit. The sampling scheme determines what a new observation means: it might be a new patient from a familiar population, a patient from a new clinic, or a future time block. The loss and its aggregation also matter. For an ordinary per-observation loss, risk is an expected loss for a future observation. For a nondecomposable criterion, the target may instead be a functional of a whole test sample; the scoring and aggregation plan must then be stated in those terms.

The Nested Plan

In an outer split, hold out data solely for assessment. Use the outer training set to run the complete selection procedure. Inside that training set, inner splits compare candidate pipelines; after selecting one, refit it using all of the outer training data. Only then score it on the untouched outer test set.

Specify the target.
Name the deployment population, prediction unit, loss or performance criterion, and the training size the assessment is intended to represent.

Choose outer splits.
Make the held-out units reflect the target: observations, whole groups, or future time blocks, as appropriate. Keep all information from each test fold out of fitting and selection.

Select within each outer training set.
For every inner split, fit preprocessing using only the inner training portion, then fit and score each candidate pipeline on its inner validation portion. Apply the pre-specified selection rule to the resulting scores.

Refit and assess.
Refit the selected pipeline, including its preprocessing, using the entire outer training set. Score its predictions on the outer test set and retain the score and selected configuration.

Summarize and deploy.
Aggregate outer-fold results according to the estimand and report the split design and training sizes. After assessment, run the same selection procedure on all development data to obtain the deployment model.

A candidate pipeline is a complete specification, not just a model name. It includes the preprocessing rule, model class, and allowed hyperparameter values. Any step that learns from data—including imputation values, scaling constants, feature screening, or a data-adaptive representation—must be fitted anew using only the training portion relevant to that step. Transformations fixed in advance without reference to the study data can be applied consistently, but this distinction should be documented.

Theorem: Suppose $Z_1,\ldots,Z_n$ are independent and identically distributed from $P$, the loss is integrable, and an outer partition is generated independently of the observations with $K$ equal-sized folds. Suppose the full procedure run in each outer training set uses no information from its test fold, and each fitted procedure is evaluated on every observation in that fold. If $m=n(K-1)/K$, the expected mean outer-fold loss equals $R_m(\mathcal A)$.

Proof. Write $I_k$ for outer fold $k$, and let $\hat f_{-k}$ be the predictor returned by the complete selection-and-refit procedure using observations outside $I_k$. The mean outer loss is $$ \widehat R_{\mathrm{outer}}=\frac{1}{n}\sum_{k=1}^{K}\sum_{i\in I_k}L(Y_i,\hat f_{-k}(X_i)). $$ For any fixed $k$ and $i\in I_k$, the training observations outside $I_k$ and any procedure randomness are independent of $Z_i$. Their joint distribution is that of a training sample of size $m$, together with the procedure randomness. Therefore $$ E\left[L(Y_i,\hat f_{-k}(X_i))\right]=R_m(\mathcal A). $$ Taking expectations of the finite sum and using that the folds contain $n$ observations in total gives $$ E[\widehat R_{\mathrm{outer}}]=\frac{1}{n}\sum_{k=1}^{K}\sum_{i\in I_k}R_m(\mathcal A)=R_m(\mathcal A). $$ The fold scores need not be independent for this expectation identity. The identity does not, by itself, establish an uncertainty interval or extend to dependent observations, unequal fold training sizes, or arbitrary nondecomposable criteria. $\square$

Worked Example: Patient-Level Nested Validation

A study has 120 patients sampled independently from the population where a binary risk score will be used. The planned procedure compares elastic-net logistic regression and a random forest, with a pre-specified grid of tuning values for each. It uses median imputation and, where required, standardization. The target is expected patient-level misclassification risk for a procedure trained on approximately 96 patients.

Use five outer folds of 24 patients each. In one outer iteration, 96 patients form the outer training set and 24 are reserved for assessment. Within those 96 patients, use four inner folds: each candidate is fitted on 72 patients and scored on the remaining 24, repeating until every outer-training patient has an inner validation prediction. Fit the imputation rule and standardization constants using the 72 inner-training patients each time; applying preprocessing fitted once to all 96 would leak inner validation information into selection.

Suppose the pooled inner misclassification rates for the best-tuned candidates are 18/96 for elastic-net logistic regression and 20/96 for the random forest. These are $0.1875$ and approximately $0.2083$, respectively, so the pre-specified rule selects elastic-net logistic regression for this outer iteration. Refit its imputation, scaling, and selected logistic model on all 96 outer-training patients. If it misclassifies four of the 24 outer-test patients, its outer score is $4/24=0.1667$. The test fold played no part in selecting that configuration.

If the five outer folds produce 4, 3, 5, 2, and 4 errors, the pooled outer estimate is $18/120=0.15$. This estimates the expected risk of the specified procedure trained on 96 patients, under the stated sampling and loss assumptions. The selected model may differ across outer iterations. After the assessment is complete, run the selection procedure on all 120 patients to choose and fit the deployment pipeline; the outer estimate is not a direct evaluation of that final 120-patient fit.

Selection Scores Are Not Performance Scores

Inner validation chooses among candidates. Its best observed score is consequently optimistic as a measure of future performance: the selection rule favors candidates that score well, including those that benefited from random variation in the inner splits. Outer assessment addresses this selection by evaluating the selected procedure on data not used in its choice.

Result: If $S_1,\ldots,S_J$ are candidate validation losses and the selected candidate minimizes them, then $$ E\left[\min_{1\leq j\leq J}S_j\right]\leq \min_{1\leq j\leq J}E[S_j]. $$ Thus, the minimum observed validation loss cannot be treated as an unbiased estimate of the selected candidate’s future loss merely because each candidate was scored on validation data.

Derivation. For each fixed candidate $j$, the pointwise inequality $\min_l S_l\leq S_j$ holds. Taking expectations gives $E[\min_l S_l]\leq E[S_j]$ for every $j$. Taking the minimum of the right-hand side over $j$ proves the result. No independence between candidate scores is required. The inequality concerns the selected minimum versus the best candidate’s expected validation score; it does not claim a particular amount of bias or that the inequality is strict in every setting.

Worked Example: Grouped Assessment for New Clinics

Suppose a model will be used at clinics not represented in its training data. There are 30 clinics, with multiple patients per clinic, and the target criterion is the average clinic-specific error rate for a new clinic. A random patient split would place patients from the same clinic in training and test sets, so it would not assess the stated target. Instead, assign whole clinics to five outer folds, six clinics per fold, and use grouped inner folds within each outer training set.

For each inner split, fit every data-dependent preprocessing step using only patients from the inner-training clinics. Select the model class and tuning values using the pre-specified average of the validation clinics’ error rates, giving each clinic equal weight. Then refit the selected pipeline on all patients in the outer-training clinics and predict patients in the six held-out clinics.

Suppose the five outer test folds yield equal-weighted mean clinic error rates of $0.12, 0.15, 0.10, 0.18,$ and $0.14$. The outer summary is $$ \frac{0.12+0.15+0.10+0.18+0.14}{5}=\frac{0.69}{5}=0.138. $$ This estimates average error for clinics under the study’s clinic-sampling scheme and the procedure’s outer-training size. Pooling all patient errors instead would weight large clinics more heavily and answer a different question. The choice of split unit and score aggregation must therefore match the estimand together.

Worked Example: Rolling Validation for Future Weeks

A forecasting procedure will be applied using the most recent 96 weeks of data to predict the following eight weeks. There are 120 ordered weeks. Use three outer test blocks: weeks 97–104, 105–112, and 113–120. For each block, train on the immediately preceding 96 weeks: weeks 1–96, 9–104, and 17–112, respectively. No future week may contribute to a training transformation or a tuning decision.

Within the first outer training set, compare candidates using rolling inner splits: fit on weeks 1–64 and validate on 65–72; fit on 1–72 and validate on 73–80; fit on 1–80 and validate on 81–88; and fit on 1–88 and validate on 89–96. Repeat the same temporal rule within each outer training set. Any scaling or imputation is fitted using only the observations available at each inner origin.

Suppose the average inner mean absolute errors for the best-tuned versions of three candidate model classes are 4.8, 5.1, and 4.6 units. The third candidate is selected because $4.6$ is the smallest score, then refitted on weeks 1–96. If its outer-block mean absolute errors are 4.9, 5.4, and 5.0, their mean is $(4.9+5.4+5.0)/3=5.1$ units. This summary describes the specified rolling-origin procedure and these forecast horizons. Unlike the independent-observation theorem above, its interpretation depends on the temporal sampling scheme and any assumptions used to generalize across forecast origins.

Common Pitfalls and Reporting

A nested design is only as sound as its target and information boundaries. Before running it, record the prediction unit, intended deployment population, outer and inner split rules, candidate pipelines, selection criterion, tie-breaking rule, randomization policy, and aggregation of outer scores. Report outer training sizes as well as the number of folds; an estimate based on smaller training sets can differ from the performance of a final procedure trained on all observations.

Using the outer results to revise the pipeline. If the analyst changes candidates after inspecting outer scores and then reports those same scores as final assessment, the outer data have influenced the procedure. A new untouched assessment would be needed for that revised procedure.
Preprocessing before splitting. Even an apparently routine operation can transmit information across a boundary if it estimates a quantity from the full data. Fit data-adaptive preprocessing within each training portion.
Splitting at the wrong unit. Patient-level random folds do not assess performance at new clinics when clinic membership is shared. The split unit should represent what is new at deployment.
Calling the inner minimum the final performance estimate. Inner scores select a pipeline; outer scores assess the selection procedure. These serve different purposes.
Reporting only one chosen model. Outer iterations may select different classes or hyperparameters. This variability is useful evidence about the stability of the selection process, even though outer folds are not independent replications.

Key takeaway: Nested validation evaluates the entire data-driven procedure by separating selection from assessment. State exactly what is new at deployment, keep every learned operation inside its training split, and interpret the outer score at its actual training size and under its actual split and scoring design.

Check Your Understanding

Use the target population and the information boundaries in each answer.

A pipeline estimates missing-value replacements and selects features before cross-validation. Which information boundary has been violated, and how should the plan change?
Why does a nested outer score generally estimate risk for a smaller training set than the final model fitted on all development data?
A study targets performance at new hospitals. What should be kept together in the outer folds, and how might the score be aggregated if hospitals differ in size?
Explain why the minimum inner validation loss is not a final performance estimate, even when each candidate was scored on held-out inner folds.
For a time-ordered forecasting target, why can a random outer split answer the wrong question?

Designing a Nested Validation Study for a Learning Pipeline

What You'll Learn