Selecting A Validation Loss Function

Why the Validation Loss Must Match the Target

Elastic net fitting balances data fit against a penalty on the coefficients. Validation asks a different question: how costly are the predictions on observations that were not used to fit the model? The answer depends on what errors matter for the intended use. A large miss may deserve extra weight, or systematic underprediction may be more costly than overprediction. The validation loss makes that priority explicit.

For each candidate tuning choice, such as a particular pair of elastic net parameters, fit the model using the training observations in each fold. Predict the held-out observations, calculate a loss for each prediction, and aggregate those losses. The candidate with the smaller score is preferred under that chosen criterion. Keep the same folds and observations when comparing candidates so the comparison reflects differences in their predictions rather than differences in the data being scored.

Definition: For a held-out response \(y_i\) and its prediction \(\hat{y}_i\), define the prediction error as \(e_i=y_i-\hat{y}_i\). A validation loss is a nonnegative function of the response and prediction that assigns a cost to the prediction error. The validation score is an aggregation of these losses over held-out observations. Smaller scores indicate better performance for the loss being used.

Common losses for a quantitative response are squared error, absolute error, and asymmetric quantile loss. Squared error penalizes large errors strongly; absolute error grows in direct proportion to error size; asymmetric loss assigns different costs to underprediction and overprediction. These choices are not interchangeable: two candidates can have different rankings under different losses.

Squared-Error and Absolute-Error Scores

For \(n\) held-out observations, mean squared error (MSE), root mean squared error (RMSE), and mean absolute error (MAE) are

\operatorname{MSE}=\frac{1}{n}\sum_{i=1}^{n}(y_i-\hat{y}_i)^2,\qquad \operatorname{RMSE}=\sqrt{\operatorname{MSE}},\qquad \operatorname{MAE}=\frac{1}{n}\sum_{i=1}^{n}|y_i-\hat{y}_i|.

MSE has squared response units, while RMSE and MAE have the same units as the response. RMSE is often easier to interpret than MSE because it is on the response scale, but RMSE and MSE always rank candidates identically: the square-root function is increasing. MAE is less sensitive to a single very large error than squared error is.

If the validation loss is squared error, the prediction target is the conditional mean: the mean response given the predictors. If the loss is absolute error, a minimizing prediction is a conditional median. These interpretations describe the target rewarded by the loss; they do not mean that every fitted elastic net prediction exactly equals that population quantity.

Worked Example: A Large Miss Versus Consistent Smaller Misses

A hypothetical insurer compares two elastic net tuning choices using four held-out claim amounts, recorded in thousands of dollars. The observed amounts are \(10,10,10,10\). Candidate A predicts \(10,10,10,7\), and candidate B predicts \(9,9,9,9\). Calculate each candidate's MSE and MAE, then identify which one each loss prefers.

Method. Calculate the error \(e_i=y_i-\hat{y}_i\) for each observation, then apply each loss to the same errors. The error units are thousands of dollars.

For A, the errors are \(0,0,0,3\). The sum of squared errors is \(0^2+0^2+0^2+3^2=9\), so \(\operatorname{MSE}_A=9/4=2.25\) thousand-dollars squared. The absolute errors sum to \(0+0+0+3=3\), so \(\operatorname{MAE}_A=3/4=0.75\) thousand dollars.

For B, the errors are \(1,1,1,1\). The sum of squared errors is \(1+1+1+1=4\), so \(\operatorname{MSE}_B=4/4=1\) thousand-dollars squared. The absolute errors also sum to \(4\), so \(\operatorname{MAE}_B=4/4=1\) thousand dollars.

Squared error prefers B because \(1<2.25\). Absolute error prefers A because \(0.75<1\). The check is intuitive: B avoids the one large miss, while A has a smaller total absolute error. The common mistake is to assume that “smaller typical error” has one definition. Here the choice of loss changes the preferred tuning choice.

Aggregating Losses Across Folds

Suppose fold \(k\) contains \(n_k\) validation observations and has average loss \(L_k\). Two common summaries are the average of the fold scores and the pooled observation-level score:

L_{\text{fold}}=\frac{1}{K}\sum_{k=1}^{K}L_k,\qquad L_{\text{pooled}}=\frac{\sum_{k=1}^{K}n_kL_k}{\sum_{k=1}^{K}n_k}.

The first gives each fold equal weight. The second gives each validation observation equal weight. When all folds have the same size, they are equal. When fold sizes differ, they can disagree, so the aggregation rule should be specified and applied consistently across candidate models.

Worked Example: Unequal Fold Sizes Change the Aggregate Score

Two candidates are evaluated with squared error on two validation folds. Fold 1 has two observations and fold 2 has four. Candidate A has errors \(0,0\) in fold 1 and \(4,4,0,0\) in fold 2. Candidate B has errors \(4,0\) in fold 1 and \(2,2,0,0\) in fold 2. Calculate each candidate's average fold MSE and pooled MSE, and compare the rankings.

Method. First calculate each fold's sum of squared errors and divide by that fold's size. Then calculate both aggregate scores.

For A, fold 1 has squared-error sum \(0\), so its MSE is \(0/2=0\). Fold 2 has squared-error sum \(4^2+4^2+0^2+0^2=32\), so its MSE is \(32/4=8\). The equally weighted average of fold MSEs is \((0+8)/2=4\). The pooled MSE is \((0+32)/(2+4)=32/6=5.333\ldots\).

For B, fold 1 has squared-error sum \(4^2+0^2=16\), giving MSE \(16/2=8\). Fold 2 has squared-error sum \(2^2+2^2+0^2+0^2=8\), giving MSE \(8/4=2\). The equally weighted average is \((8+2)/2=5\). The pooled MSE is \((16+8)/6=24/6=4\).

The average-fold score prefers A, since \(4<5\). The pooled score prefers B, since \(4<5.333\ldots\). To three decimals, the pooled scores are 5.333 for A and 4.000 for B. The arithmetic checks against the fold sizes: the pooled result gives the larger second fold more weight. The common mistake is to average fold scores when the intended target is average loss per observation, or to pool observations when the intended target gives each fold equal importance.

Asymmetric Loss for Unequal Error Costs

In some applications, underpredicting a large response is more costly than overpredicting it by the same amount. Quantile loss, also called pinball loss, expresses this asymmetry. For a chosen quantile level \(\tau\), where \(0<\tau<1\), define

\rho_{\tau}(e)= \begin{cases} \tau e, & e\geq 0,\\ (1-\tau)|e|, & e<0. \end{cases}

Because \(e=y-\hat{y}\), a positive error means the prediction is too low. When \(\tau=0.8\), for example, underprediction receives weight \(0.8\), while overprediction receives weight \(0.2\). Minimizing expected quantile loss targets the conditional \(\tau\)-quantile of the response, rather than its conditional mean or median. This is appropriate only when that quantile or error asymmetry matches the decision being evaluated.

Worked Example: Scoring Underprediction More Heavily

A hypothetical reserve review gives equal importance to three held-out loss amounts: \(10,10,20\), in thousands of dollars. Candidate A predicts \(14,14,19\), and candidate B predicts \(8,8,22\). Compare their MAE scores with their mean quantile loss at \(\tau=0.8\).

Method. Calculate errors as observed minus predicted. For \(\tau=0.8\), multiply positive errors by \(0.8\) and the absolute values of negative errors by \(0.2\).

Candidate A's errors are \(-4,-4,1\). Its absolute errors sum to \(4+4+1=9\), so \(\operatorname{MAE}_A=9/3=3\) thousand dollars. Its quantile losses are \(0.2(4)=0.8\), \(0.2(4)=0.8\), and \(0.8(1)=0.8\). Their mean is \(2.4/3=0.8\) thousand dollars.

Candidate B's errors are \(2,2,-2\). Its absolute errors sum to \(2+2+2=6\), so \(\operatorname{MAE}_B=6/3=2\) thousand dollars. Its quantile losses are \(0.8(2)=1.6\), \(0.8(2)=1.6\), and \(0.2(2)=0.4\). Their mean is \(3.6/3=1.2\) thousand dollars.

MAE prefers B, since \(2<3\); the \(\tau=0.8\) loss prefers A, since \(0.8<1.2\). This is consistent with the stated asymmetry: B underpredicts twice, and those misses carry the larger weight. The common mistake is to reverse the sign interpretation: with \(e=y-\hat{y}\), positive errors are underpredictions.

Why the Loss Selects a Particular Prediction Target

The relationship between loss and target follows by minimizing expected loss for a fixed predictor value \(x\). Write \(a\) for a proposed prediction and consider the conditional distribution of \(Y\) given \(X=x\).

Squared error targets the mean.
If the conditional second moment is finite, then \(E[(Y-a)^2\mid X=x]=\operatorname{Var}(Y\mid X=x)+(E[Y\mid X=x]-a)^2\). The variance term does not depend on \(a\); the square is minimized at \(a=E[Y\mid X=x]\).

Absolute error targets a median.
The absolute-error risk is minimized where at least half the conditional probability is at or below \(a\), and at least half is at or above \(a\). Such a value is a conditional median. If there is a range of medians, each value in that range minimizes the risk.

Quantile loss targets a quantile.
For a continuous conditional distribution, the derivative of expected \(\rho_\tau(Y-a)\) with respect to \(a\) is \(F_{Y\mid X}(a\mid x)-\tau\). A minimum occurs where this is zero, so \(a\) is a conditional \(\tau\)-quantile. For distributions with point masses, the minimum is characterized by the corresponding quantile condition rather than a derivative equal to zero.

These facts explain why a loss choice is part of the modeling objective, not just a reporting detail. Squared error emphasizes the mean response and gives large residuals disproportionate influence. Absolute error targets a median and limits that influence. Quantile loss targets a specified part of the conditional response distribution. A different target can therefore favor a different elastic net tuning choice even when the fitted predictions are unchanged.

Practical Choices and Common Traps

The standard elastic net training objective uses squared-error fit together with a coefficient penalty. Validation need not use the same loss as training: it can score the predictions according to the intended use. The penalty is used to fit the coefficients; it is not added to the held-out prediction loss. Adding it would mix a coefficient-complexity term with the question of how well predictions match held-out outcomes.

Use the response scale and weights that represent the target. If predictions and outcomes are scored after a transformation, such as taking logarithms, the resulting errors answer a different question from errors scored on the original scale. Likewise, if observations represent different exposure amounts or have different decision costs, an unweighted average may not reflect the intended target. Define any observation weights and their normalization before comparing candidates.

Huber loss is another option when squared error's sensitivity to large errors is undesirable but a smooth loss near zero is useful. For a chosen threshold \(c>0\), it is \(\frac{1}{2}e^2\) when \(|e|\leq c\), and \(c(|e|-\frac{1}{2}c)\) when \(|e|>c\). The threshold determines where the loss changes from quadratic to linear. It must be specified on the response scale being scored; it is not a universal setting.

Key takeaway: Choose a validation loss before comparing tuning choices. Squared error, absolute error, and asymmetric loss reward different prediction targets and can select different models. State the loss, its units, its aggregation across folds, and any weights so the score has a clear interpretation.

Check Your Understanding

For each problem, identify the requested loss and show the key arithmetic before checking the answer.

A hypothetical disability model is scored on two held-out monthly benefit amounts, in thousands of dollars: observed values \(6,12\), predictions \(8,11\). Calculate MSE and MAE. (Answer: MSE \(=2.5\) thousand-dollars squared; MAE \(=1.5\) thousand dollars.)
Two candidates have held-out errors \(0,0,4\) and \(2,2,2\), respectively. Calculate each candidate's MSE and MAE, then state which candidate each loss prefers. (Answer: First candidate MSE \(=16/3\), MAE \(=4/3\); second candidate MSE \(=4\), MAE \(=2\). MSE prefers the second; MAE prefers the first.)
For \(\tau=0.75\), a model has held-out errors \(-4,2\), where error is observed minus predicted. Calculate its mean quantile loss and identify which direction of error receives the larger weight. (Answer: Losses are \(1\) and \(1.5\), so the mean is \(1.25\); underprediction receives the larger weight.)
Two folds have sizes 1 and 3. Candidate A has squared errors \(16\) in the first fold and \(0,0,0\) in the second. Candidate B has squared errors \(4\) in the first fold and \(4,4,4\) in the second. Calculate the average-fold and pooled MSE for both candidates. (Answer: A has average-fold MSE \(8\) and pooled MSE \(4\); B has average-fold MSE \(4\) and pooled MSE \(4\). The average-fold score prefers B; pooled scoring gives a tie.)
A pricing team wants a prediction at the 90th percentile because underpredicting high costs is especially consequential. Which loss is aligned with that target, and what weighting does it apply to positive versus negative errors when \(e=y-\hat{y}\)? (Answer: Quantile loss with \(\tau=0.9\); positive errors receive weight \(0.9\), and negative errors receive weight \(0.1\).)

What You'll Learn