From Cross-Validation to One Final Model
Cross-validation answers a tuning question: which values of the elastic-net mixing parameter \(\alpha\) and penalty parameter \(\lambda\) perform best under the chosen validation procedure? It does not, by itself, produce the single model that should be used for future predictions. Each fold-specific model was fitted using only the training portion of that fold. After selecting the tuning parameters, the usual next step is to fit the model again using all available development observations.
The distinction matters in actuarial modeling. A hypothetical insurer might use historical policy-level data to predict claim cost. Fold-specific fits help compare candidate tuning choices, but each leaves out some development observations. The final refit uses all development observations, while any reserved test set remains untouched until the final assessment.
The Fit Being Recomputed
For a response \(y_i\) and \(p\) predictors, one common elastic-net objective for a training sample of size \(n\) is
Here \(\beta_0\) is an unpenalized intercept, \(\boldsymbol{\beta}\) is the coefficient vector, \(0\leq\alpha\leq1\), and \(\lambda\geq0\). With this convention, \(\alpha=1\) gives the lasso penalty and \(\alpha=0\) gives the ridge penalty. The penalty scaling shown is one standard convention; software can use different scaling or standardization conventions. A refit must use the same objective convention used during tuning, or the selected numerical value of \(\lambda\) may no longer represent the same strength of penalty.
Cross-validation evaluates candidate settings by fitting on each fold's training portion and calculating loss on its validation portion. Once the selected pair \((\alpha,\lambda)\) is fixed, refitting replaces the fold training sample with the complete development sample. The model is optimized again; coefficients are not merely copied from a fold.
Preprocessing is part of this distinction. If predictors were centered, scaled, or otherwise transformed, the fold procedure must estimate those transformations from each fold's training portion. For the final refit, estimate the transformations from all development observations, apply them to those observations, and retain the resulting quantities for future predictions. If a test set was reserved, it supplies no information for estimating these transformations.
Why the Coefficients Change: A Short Derivation
A one-predictor case shows why refitting is an optimization on the selected data set, rather than an average of existing fits. Suppose the predictor and response are centered, the predictor has mean squared value \(1\), and the objective uses the convention above. Write \(c=\sum_i x_i y_i/n\). Ignoring terms that do not depend on \(\beta\), the objective is
For a positive coefficient, differentiating gives \(\{1+\lambda(1-\alpha)\}\beta-c+\lambda\alpha=0\). A negative coefficient gives the corresponding equation with the opposite sign for the absolute-value penalty. At zero, the absolute-value penalty can hold the optimum at zero when \(|c|\leq\lambda\alpha\). Combining those cases gives the minimizer
The result depends on \(c\), which depends on the observations used in the fit. A fold-specific \(c\) and the full-development-data \(c\) need not be equal. Therefore the same selected tuning parameters can yield different coefficients in fold fits and in the final refit. This derivation assumes a centered, unit-mean-square predictor and response, an unpenalized intercept handled by centering, and the stated objective scaling; do not apply this closed-form expression directly to unstandardized predictors or a different penalty convention.
Worked Example: Refit After a Two-Fold Tuning Exercise
A hypothetical insurer models a centered claim-cost response using one centered, standardized predictor. A two-fold cross-validation comparison, using mean squared validation error, reports these average losses:
| \(\alpha\) | \(\lambda\) | Mean validation loss |
|---|---|---|
| 0.25 | 0.20 | 0.184 |
| 0.50 | 0.20 | 0.162 |
| 0.75 | 0.20 | 0.170 |
| 0.50 | 0.40 | 0.175 |
The fold-specific training fits at the winning setting \((\alpha,\lambda)=(0.50,0.20)\) have \(c=0.05\) in one fold and \(c=0.80\) in the other. The full development data, after applying the full-development standardization, have \(c=0.50\). Find the selected setting, the fold coefficients, and the final refitted coefficient.
Method and selection. The smallest listed validation loss is \(0.162\), so select \(\alpha=0.50\) and \(\lambda=0.20\). Keep these values fixed for the refit; do not run a new tuning search at this stage.
Fold fits. The threshold in the one-predictor formula is \(\lambda\alpha=(0.20)(0.50)=0.10\), and the denominator is \(1+\lambda(1-\alpha)=1+(0.20)(0.50)=1.10\). For the first fold, \(|c|=0.05\leq0.10\), so its coefficient is \(0\). For the second fold, the coefficient is \((0.80-0.10)/1.10=0.70/1.10=0.6364\), rounded to four decimal places.
Final refit. Use the full-development value \(c=0.50\). The coefficient is \((0.50-0.10)/1.10=0.40/1.10=0.3636\), rounded to four decimal places. The final model's coefficient is therefore \(0.3636\).
Check and common mistake. The full-data coefficient is positive because \(c\) exceeds the threshold \(0.10\), and it is smaller than the second fold's coefficient because the full-data \(c=0.50\) is smaller than \(0.80\). Averaging the two fold coefficients gives \((0+0.6364)/2=0.3182\), not \(0.3636\). That average is not the refitted coefficient: it optimizes neither the stated objective on all development data nor necessarily any other clearly defined fitting objective.
Preprocessing and the Coefficient Scale
The refit often changes more than the coefficient estimates. If standardization was used, the final transformation is estimated using all development observations. For a predictor \(x\) with development mean \(\bar{x}\) and scale \(s\), define \(z=(x-\bar{x})/s\). A fitted coefficient on \(z\) is not yet the coefficient per original unit of \(x\). For a one-predictor model with response mean \(\bar{y}\), if the fitted equation is \(\widehat{y}=\bar{y}+\widehat{\gamma}z\), its original-scale slope is \(\widehat{\gamma}/s\), and its intercept is \(\bar{y}-\widehat{\gamma}\bar{x}/s\). These transformations apply to this centering and scaling setup; other preprocessing, such as transformations of the response, requires the corresponding inverse transformation.
Worked Example: Convert a Full-Data Standardized Fit
For a hypothetical portfolio model, cross-validation selects \(\alpha=0.50\) and \(\lambda=0.20\). The final refit uses all development observations. Their predictor has mean \(\bar{x}=10\) and scale \(s=2\); the response mean is \(\bar{y}=7\). For the standardized predictor \(z=(x-10)/2\), the centered cross-product is \(c=0.98\), with mean squared predictor equal to \(1\). Find the penalized coefficient and the equation on the original predictor scale.
Method. Apply the one-predictor elastic-net solution to the full-development standardized data. The penalty threshold is \((0.20)(0.50)=0.10\), and the denominator is \(1+(0.20)(1-0.50)=1.10\). Since \(c=0.98>0.10\), the standardized coefficient is \(\widehat{\gamma}=(0.98-0.10)/1.10=0.88/1.10=0.80\).
Return to original units. The slope per original predictor unit is \(0.80/2=0.40\). The intercept is \(7-(0.40)(10)=3.00\). Thus the final equation is \(\widehat{y}=3.00+0.40x\).
Check and common mistake. At \(x=10\), the standardized predictor is zero and the equation predicts \(3.00+(0.40)(10)=7.00\), the development response mean, as required by the centered fit. Reporting \(0.80\) as the slope per original unit would be wrong: it is the coefficient per one unit of the standardized predictor. The scale \(s=2\) must be accounted for.
Keep the Test Set Out of the Refit
When a final test set has been reserved, “all available development data” means all observations assigned to development, not every observation in the project. The test observations must not be used to choose \(\alpha\) or \(\lambda\), fit coefficients, estimate preprocessing quantities, or decide between models. The final refit uses the selected tuning parameters on development data only. Afterward, the test set can be used to assess predictions from that fixed model.
A test-set result is an assessment, not a new tuning signal. If the analyst changes the model after inspecting test performance, the test set has influenced model development and no longer serves as an untouched assessment set. Likewise, the cross-validation error used to choose tuning parameters is not the same quantity as a test-set error from the final refit: the former compares fold validation predictions, while the latter evaluates predictions from one model fitted on all development data.
Worked Example: Evaluate Without Refitting on Test Observations
A hypothetical claim-frequency project has 60 development observations and 20 reserved test observations. Cross-validation on the 60 development observations selects \(\alpha=0.25\) and \(\lambda=0.10\). The final model is refitted on all 60 development observations. On four of the reserved test observations, actual values are \(110,115,90,125\), while the final model predicts \(100,120,90,110\). Calculate the mean absolute error and root mean squared error for these four cases, and state whether the model should now be retuned.
Method. The prediction errors, defined as prediction minus actual value, are \(100-110=-10\), \(120-115=5\), \(90-90=0\), and \(110-125=-15\). Their absolute values are \(10,5,0,15\), with sum \(30\). Therefore the mean absolute error is \(30/4=7.50\).
The squared errors are \(100,25,0,225\), with sum \(350\). The mean squared error is \(350/4=87.5\), so the root mean squared error is \(\sqrt{87.5}=9.35\), rounded to two decimal places.
Decision and check. These metrics summarize performance on the four stated test cases; their units are the same as the response for mean absolute error and root mean squared error. The tuning parameters remain \((0.25,0.10)\), and the model is not retuned using these test outcomes. The calculation uses all four errors, including the zero error, in both denominators.
Practical Sequence and Common Traps
A disciplined sequence keeps the roles of tuning, fitting, and assessment clear:
Use the prescribed cross-validation procedure and loss function to select \(\alpha\) and \(\lambda\). Record the selection rule used, including any tie-breaking rule if relevant.
Estimate centering, scaling, and any other learned transformations from all development observations. Do not estimate them using a reserved test set.
Optimize the elastic-net objective on the complete development data with the selected \(\alpha\), \(\lambda\), and the same objective convention used during tuning.
Apply the development-data transformations to new predictor values before prediction. If there is a reserved test set, use it only to assess the fixed model.
The most frequent conceptual error is to treat the collection of fold models as the final model. Those fits served a validation purpose and were trained on different subsets. Averaging their coefficients generally does not solve the elastic-net optimization problem on the full development data. Another error is to retune after refitting simply because the sample size changed: refitting changes the observations used for estimation, not the selected tuning decision. A new tuning search is a different modeling procedure and should not be confused with the required final fit.
Check Your Understanding
For each problem, distinguish the cross-validation fits from the final refit and keep any reserved test data separate.
- A two-fold procedure selects \(\alpha=0.60\) and \(\lambda=0.15\). The development set contains 80 observations and a test set contains 20. How many observations are used for the final coefficient fit, and which tuning values are used? (Answer: Fit on all 80 development observations using \(\alpha=0.60\) and \(\lambda=0.15\); do not include the 20 test observations.)
- For a centered, standardized one-predictor fit, \(c=0.30\), \(\alpha=0.50\), and \(\lambda=0.40\). Under the objective defined in this tutorial, find the coefficient. (Answer: \(0.10/1.20=0.0833\), to four decimal places.)
- A full-development standardized coefficient is \(0.90\), and the predictor's development scale is \(s=3\). For a centered response, what is the original-unit slope? (Answer: \(0.90/3=0.30\) per original predictor unit.)
- Two fold-specific coefficients are \(0.20\) and \(0.80\). The full-development data have \(c=0.55\), with \(\alpha=0.50\) and \(\lambda=0.20\), under the one-predictor assumptions used in the derivation. Find the refitted coefficient and compare it with the fold-coefficient average. (Answer: The refit is \((0.55-0.10)/1.10=0.4091\); the fold average is \(0.50\), so it is not the refitted coefficient.)
- A final model has already been fitted on the development set. Its test-set error is larger than expected. Should the analyst change \(\lambda\) using that test error and still describe the same test set as untouched? (Answer: No. Changing \(\lambda\) based on test results uses the test set in model development, so it is no longer an untouched assessment set.)