Why Nest the Folds?
Cross-validation can serve two different purposes: choosing model settings and estimating predictive performance. If the same validation results do both jobs, the reported performance can be optimistic: among the settings considered, the one with the most favorable validation result may have benefited from random variation. Nested cross-validation separates these jobs. Inner folds choose the settings; outer folds assess the procedure that chooses and fits them.
For an elastic-net model, the settings commonly include the mixing parameter \(\alpha\) and penalty parameter \(\lambda\). The inner procedure compares candidate pairs using a selected loss function, such as mean squared error (MSE). The outer procedure then measures predictions from a model whose settings were selected without using that outer fold. Nested cross-validation is aligned with the SOA Exam SRM (Statistics for Risk Modeling) syllabus.
The Nested Procedure
Let the observed data be \(D=\{(x_i,y_i):i=1,\ldots,n\}\), and let \(I_k\) denote the observations in outer fold \(k\). The outer-training data for that iteration are \(D_{-k}=D\setminus I_k\). Suppose the candidate elastic-net settings form a grid \(\mathcal{G}\) of \((\alpha,\lambda)\) pairs. For each pair, calculate its inner cross-validation loss using only \(D_{-k}\), and select the pair with the smallest loss, or apply another selection rule specified in advance.
For squared-error loss, let \(\widehat f_{-k,\widehat\theta_{-k}}\) be the prediction rule fitted on all of \(D_{-k}\) using the settings \(\widehat\theta_{-k}\) selected by inner cross-validation. The nested estimate is the mean loss on the outer observations:
The outer folds partition the observations, so each observation contributes one outer prediction error. If outer folds have equal sizes, this pooled MSE equals the average of the outer-fold MSEs. If fold sizes differ, the pooled calculation weights each fold in proportion to its number of observations; an unweighted average of fold means generally does not.
Choose the outer split plan before inspecting outer-fold performance. Each iteration will use one fold only for evaluation.
For each candidate \((\alpha,\lambda)\), calculate inner-fold validation loss using only the current outer-training data. Apply the preselected tuning rule to these inner results.
Fit the selected settings on all of the current outer-training data, then calculate predictions and losses for the untouched outer fold.
Pool the outer prediction losses, or average fold means with weights proportional to fold sizes. Report the metric on its original scale or transform it as required.
The tuning rule itself belongs inside the inner loop. This includes the candidate grid, a rule for breaking ties, and any use of the one-standard-error rule. If preprocessing is part of the modeling pipeline, its learned quantities must also be estimated using the training portion of each split. For example, a standardization mean and standard deviation are calculated on an inner-training fold before transforming its inner-validation fold. After settings are selected, preprocessing is estimated anew on the full outer-training set before the outer model is fitted. The outer fold must not influence either calculation.
Why the Outer Score Assesses the Tuning Procedure
The nested estimate is useful because the outer observation did not participate in the selection or fitting of the model used to predict it. The following justification assumes independent, identically distributed observations, a loss defined for each prediction, and folds that leave out observations without allowing their outcomes into training or tuning.
Here \(R\) is the expected loss for a new observation from the same population, conditional on the fitted prediction rule. To see why the equality holds, first condition on \(D_{-k}\). The inner folds, selected settings, and outer-training fit are then determined by that data. Each outer observation is independent of \(D_{-k}\) and has the same distribution as a new observation. Thus each conditional expected loss in the sum is the risk of that fitted rule. Averaging those expected losses gives the displayed equality.
This argument explains the separation, not a claim that every finite-sample nested estimate is exact or that the outer-fold losses are independent. Outer training sets overlap, so their losses can be dependent. The estimate primarily describes the tuning-and-fitting procedure trained on the outer-training sample size, approximately \(n(K-1)/K\) for equal-sized \(K\)-fold outer validation. It need not equal the risk of a model trained on all \(n\) observations.
Worked Example: Planning Nested Folds and Selecting Settings
A hypothetical insurer has 24 independent policy-level records and uses squared prediction error to compare elastic-net models for a continuous outcome. The analyst plans four outer folds and three inner folds. The candidate grid contains 12 pairs of \(\alpha\) and \(\lambda\). Determine the fold sizes, select settings for one outer iteration from the supplied inner losses, and count the candidate fits required for the complete nested procedure. Assume equal fold sizes and that all preprocessing is refitted within each training split.
Method. Apply three-fold cross-validation only within each outer-training set. For each candidate, average its three inner validation MSEs; choose the candidate with the smallest average.
Fold sizes. Each of the four outer folds contains \(24/4=6\) records. In one iteration, \(24-6=18\) records form the outer-training set. Dividing those into three inner folds gives six inner-validation records per fold and \(18-6=12\) inner-training records per fold.
For one outer iteration, the inner validation MSEs are:
| Candidate \((\alpha,\lambda)\) | Inner fold 1 | Inner fold 2 | Inner fold 3 | Mean inner MSE |
|---|---|---|---|---|
| \((0.2,0.8)\) | 12 | 10 | 14 | \((12+10+14)/3=12\) |
| \((0.8,0.8)\) | 9 | 13 | 11 | \((9+13+11)/3=11\) |
| \((0.5,0.2)\) | 10 | 8 | 12 | \((10+8+12)/3=10\) |
The third candidate has the smallest mean inner MSE, \(10\), so this outer iteration selects \((\widehat\alpha,\widehat\lambda)=(0.5,0.2)\). Refit that setting on all 18 outer-training records, including the records that served as inner-validation data. Then evaluate it on the six outer records, which have not been used for tuning or fitting.
Across the full nested procedure, the inner comparisons require \(4\times3\times12=144\) candidate-fold fits. There are also four refits, one for each selected setting on its complete outer-training set. This count excludes implementation efficiencies such as fitting a regularization path in one computation.
Check. Each of the 144 candidate fits uses 12 inner-training records, not the outer fold. The final outer evaluation for each iteration uses six records. Common mistake avoided: selecting the candidate from an outer-fold result would let evaluation data influence the choice.
Worked Example: Calculating a Pooled Nested MSE
A hypothetical fleet-risk team uses three outer folds of equal size, with two policies in each fold. In each outer-training set, two elastic-net candidates, \(A=(0.2,1.0)\) and \(B=(0.7,0.4)\), are compared by two-fold inner MSE. The inner fold losses and the residuals from the resulting outer predictions are shown below. Calculate the selected settings by outer iteration, the pooled nested MSE, and the corresponding RMSE.
Method. Select the lower mean inner MSE separately within each outer-training set. Then square each outer residual, sum all six squared residuals, divide by six, and take the square root for RMSE.
| Outer iteration | Candidate A inner losses | Candidate B inner losses | Outer residuals from selected model |
|---|---|---|---|
| 1 | 4, 6 | 3, 9 | 1, -2 |
| 2 | 7, 5 | 4, 4 | 2, 0 |
| 3 | 2, 8 | 5, 3 | -1, 3 |
In iteration 1, candidate A has mean inner loss \((4+6)/2=5\); candidate B has \((3+9)/2=6\). Select A. In iteration 2, A has \((7+5)/2=6\), while B has \((4+4)/2=4\). Select B. In iteration 3, A has \((2+8)/2=5\), while B has \((5+3)/2=4\). Select B.
Now calculate outer squared errors. Iteration 1 contributes \(1^2=1\) and \((-2)^2=4\), total \(5\). Iteration 2 contributes \(2^2=4\) and \(0^2=0\), total \(4\). Iteration 3 contributes \((-1)^2=1\) and \(3^2=9\), total \(10\). The total is \(5+4+10=19\) over six observations.
The MSE is \(3.1667\) squared outcome-units, and the RMSE is \(1.7795\) outcome-units. Check. The three outer-fold MSEs are \(5/2=2.5\), \(4/2=2\), and \(10/2=5\); their equal-weight average is also \(19/6\). Common mistake avoided: do not average the inner losses and report that result as performance. Inner losses selected the settings; only the outer residuals estimate performance here.
Worked Example: Weighting Unequal Outer Folds
A hypothetical agricultural insurer has nine independent farm records. Three outer folds contain two, three, and four records, respectively. Within each outer-training set, inner cross-validation has already selected the settings for that iteration. The squared outer residuals are \(1,9\) in fold 1; \(0,4,4\) in fold 2; and \(1,1,4,0\) in fold 3. Calculate the pooled nested MSE and compare it with the unweighted average of the three fold MSEs.
Method. Since the target is mean squared error per record, add all squared errors and divide by the total number of records. An unweighted fold average gives a two-record fold the same weight as a four-record fold, so it is not the pooled record-level MSE.
Fold 1 has total squared error \(1+9=10\) and MSE \(10/2=5\). Fold 2 has total \(0+4+4=8\) and MSE \(8/3=2.6667\). Fold 3 has total \(1+1+4+0=6\) and MSE \(6/4=1.5\). The pooled total is \(10+8+6=24\), and the number of records is \(2+3+4=9\).
The unweighted average of fold means is \((5+2.6667+1.5)/3=9.1667/3=3.0556\), which differs from the pooled estimate. The correct weighted calculation is \((2(5)+3(2.6667)+4(1.5))/9=24/9=2.6667\). Check. The weights are the fold sizes, and their sum is nine. Common mistake avoided: treating unequal-sized fold means as equally precise contributions to the record-level average.
Common Traps and Interpretation
- Using outer results to choose settings. Suppose a candidate grid produces outer error totals of 20 and 14 over the same nine observations. Selecting the second candidate because \(14/9\) is smaller and reporting \(14/9\) as its performance reuses the outer results for selection. The correction is to select settings using inner folds separately within each outer-training set, then score those selected procedures on the untouched outer folds.
- Letting preprocessing cross a fold boundary. Scaling predictors once on the complete data allows held-out predictor information into training. Fit scaling, imputation, and any data-dependent feature processing inside each training split, then apply the learned transformation to its validation observations.
- Changing the tuning rule after seeing outer scores. Choosing a grid, tie-breaker, or one-standard-error rule after examining outer performance turns the outer folds into another tuning set. Specify the rule before evaluating the outer folds.
- Misstating what the estimate represents. The outer models are trained on less than the full data set. The nested score estimates the performance of the specified tuning procedure at approximately that outer-training size; it does not directly report the exact risk of a future model trained on all available observations.
Nested cross-validation is most informative when the full modeling workflow is repeated faithfully in every outer iteration. If observations are dependent, such as multiple records from one policyholder, the split must respect the dependence structure; otherwise closely related observations can appear in both training and evaluation data. For independent observations, the central discipline is straightforward: inner folds may choose, outer folds may assess, and outer information must not flow back into the choice.
Check Your Understanding
For each problem, use the stated loss and keep tuning decisions inside the inner folds.
- A data set has 80 independent records and uses five equal outer folds. Within each outer-training set, use four equal inner folds. How many records are in an outer test fold, an outer-training set, an inner validation fold, and an inner training fold? (Answer: 16, 64, 16, and 48 records, respectively.)
- In one outer iteration, three candidate pairs have inner-fold MSEs \(A:(6,8,10)\), \(B:(7,7,7)\), and \(C:(5,9,8)\). Which pair is selected by minimum mean inner MSE, and what is that mean? (Answer: Candidate B, with mean MSE 7.)
- Two equal-sized outer folds have squared residuals \((1,4)\) and \((0,9)\). Calculate the pooled nested MSE and RMSE. (Answer: MSE \(14/4=3.5\); RMSE \(\sqrt{3.5}=1.8708\), rounded to four decimals.)
- Three unequal outer folds have sizes 2, 3, and 5, with fold MSEs 4, 2, and 1. What is the pooled record-level MSE? (Answer: \((2(4)+3(2)+5(1))/10=19/10=1.9\).)
- An analyst standardizes all predictors using all 100 observations, then performs nested cross-validation and claims that outer-fold errors assess the full pipeline. Is the claim justified, and what is the correction? (Answer: No. Estimate standardization quantities using only the training portion at each inner split, then refit them on the full outer-training set before outer evaluation.)