Why the Tuning Loss Matters
Cross-validation compares candidate penalty values by measuring their prediction errors on held-out observations. But an error is not a single universal quantity: a prediction can be judged by how close a numeric value is, how well a probability is estimated, or whether a final class label is correct. The loss selected for tuning determines which of these goals the comparison emphasizes.
For penalized regression, cv.glmnet provides loss choices that depend on the response family. For a Gaussian response, mean squared error is the central prediction loss. For a binomial response, one can use deviance, classification error, or squared error on predicted probabilities. These measures can rank models differently because they retain different amounts of information about the predictions.
Losses for a Gaussian Response
For a Gaussian model, let \(y_i\) be the observed response and let \(\hat{y}_i\) be its held-out prediction. The squared error for observation \(i\) is \((y_i-\hat{y}_i)^2\), and the mean squared error over \(n\) observations is
This loss gives more influence to larger errors because the residual is squared. Its units are the square of the response units: if the response is measured in dollars, MSE is measured in dollars squared. The root mean squared error, \(\sqrt{\operatorname{MSE}}\), returns to the response's units, but it is not the same quantity as the MSE used as a tuning loss.
In cv.glmnet, Gaussian deviance and mean squared error correspond to the same prediction criterion, up to the implementation's common scaling convention. Thus, choosing Gaussian deviance does not create a different ranking from choosing MSE. When interpreting reported values, check the measure and scale rather than assuming every displayed error is in the original response units.
Worked Example: Comparing Gaussian Predictions
A hypothetical model is being tuned to predict a repair duration in hours. On four held-out cases, the observed durations are \(10,12,14,16\). Candidate A predicts \(9,13,13,17\), and candidate B predicts \(10,10,14,18\). Use MSE to compare them.
Method. Calculate each held-out residual, square it, and average the four squared residuals.
For candidate A, observed minus predicted residuals are \(10-9=1\), \(12-13=-1\), \(14-13=1\), and \(16-17=-1\). The squared residuals are \(1,1,1,1\), so the sum is \(4\) and
For candidate B, the residuals are \(10-10=0\), \(12-10=2\), \(14-14=0\), and \(16-18=-2\). The squared residuals are \(0,4,0,4\), so the sum is \(8\) and
Candidate A has the lower held-out MSE and is preferred if squared prediction error is the objective. The result is checked by noting that all four of A's predictions miss by one hour, whereas two of B's miss by two hours. The common mistake is to compare the sum of absolute errors instead of the specified squared errors; that would answer a different question.
Losses for a Binomial Response
For a binary response, write \(y_i=1\) for an event and \(y_i=0\) for a non-event. Let \(p_i\) be the model's predicted probability of the event for held-out observation \(i\). Three useful measures have distinct interpretations.
For the standard binary classification rule, predict class 1 when \(p_i\geq 0.5\) and class 0 otherwise. The cutoff matters: changing it can change classification error without changing the fitted probabilities. In cv.glmnet, type.measure = "class" evaluates classification error, while "mse" evaluates squared error on the probabilities. For binomial models, "deviance" evaluates binomial deviance. The default is deviance.
Why These Formulas Measure Different Things
Squared error penalizes the distance between the observed binary value and the probability. For example, an event with predicted probability \(0.8\) contributes \((1-0.8)^2=0.04\), while a non-event with that same prediction contributes \((0-0.8)^2=0.64\). It therefore distinguishes probabilities even when they produce the same class.
Deviance follows from the Bernoulli likelihood. Under the binomial model, an observation with event probability \(p_i\) has likelihood \(p_i^{y_i}(1-p_i)^{1-y_i}\). The saturated model assigns probability 1 to the outcome actually observed, so its log-likelihood contribution is \(\ln(1)=0\). The deviance is twice the fitted model's log-likelihood loss relative to that saturated fit:
This gives the stated binary deviance contribution. A confident but incorrect probability is penalized sharply: as \(p_i\) approaches 0 for an event, \(-2\ln(p_i)\) grows without bound. By contrast, classification error records only whether the cutoff rule got the label right. It does not distinguish a correct probability of \(0.51\) from a correct probability of \(0.99\).
The squared-error formula also has a useful justification. If, conditional on predictors, the outcome has mean \(m\), the expected squared error from predicting \(a\) is
To verify this, write \(Y-a=(Y-m)+(m-a)\), square, and take expectations. The cross-term is zero because \(E[Y-m]=0\); the remaining terms are \(\operatorname{Var}(Y)\) and \((m-a)^2\). Since the variance does not depend on \(a\), this expectation is minimized at \(a=m=E[Y\mid X]\). For a binary response this conditional mean is the event probability, which explains why probability squared error is a sensible objective when probability accuracy matters.
The derivation assumes a finite conditional second moment. For a Bernoulli response this condition holds automatically. The deviance expression assumes probabilities strictly between 0 and 1 for finite contributions; a zero probability assigned to an outcome that occurs produces infinite deviance. This is one reason a near-perfect classification result does not guarantee good probability predictions.
Worked Example: Computing Binomial Deviance and Squared Error
A hypothetical event-monitoring model produces held-out event probabilities \(0.7,0.8,0.7,0.7\). All four observed outcomes are events. Compute mean binomial deviance, classification error at cutoff \(0.5\), and the Brier score.
Method. For each event, use deviance contribution \(-2\ln(p_i)\), squared-error contribution \((1-p_i)^2\), and classify as an event if \(p_i\geq0.5\).
The deviance contributions are \(-2\ln(0.7)=0.713350\), \(-2\ln(0.8)=0.446287\), \(-2\ln(0.7)=0.713350\), and \(-2\ln(0.7)=0.713350\). Their total is \(2.586337\), which rounds to \(2.5863\). Therefore, mean deviance is \(2.586337/4=0.646584\ldots\), or \(0.6466\) to four decimal places.
All four probabilities exceed \(0.5\), so all four predicted classes are events. Every observed outcome is also an event, giving zero classification errors and classification error \(0/4=0\).
The squared-error contributions are \((1-0.7)^2=0.09\), \((1-0.8)^2=0.04\), \(0.09\), and \(0.09\). Their total is \(0.31\), so the Brier score is \(0.31/4=0.0775\). The check is that the model's class decisions are all correct even though its probabilities are not all 1; hence classification error is zero while both probability-based losses are positive.
Common mistake and correction. Rounding each deviance contribution too early can give a slightly inaccurate total. Keep full precision through the sum: \(0.7133499+0.4462871+0.7133499+0.7133499=2.5863368\ldots\), which rounds to \(2.5863\), not \(2.5862\).
Worked Example: When Classification Error Hides a Difference
A hypothetical screening model is evaluated on four held-out cases with outcomes \(1,0,1,0\). Candidate A predicts probabilities \(0.51,0.49,0.51,0.49\); candidate B predicts \(0.90,0.10,0.60,0.40\). Compare classification error and Brier score, using a cutoff of \(0.5\).
Method. First convert each probability to a class. Then compute the proportion of wrong classes and average squared probability errors.
Both candidates predict classes \(1,0,1,0\), exactly matching the observed outcomes. Each has classification error \(0/4=0\).
For A, each squared error is \(0.49^2=0.2401\): for example, the first is \((1-0.51)^2=0.2401\), and the second is \((0-0.49)^2=0.2401\). Thus, \(\operatorname{MSE}_A=(4\times0.2401)/4=0.2401\).
For B, the four squared errors are \((1-0.90)^2=0.01\), \((0-0.10)^2=0.01\), \((1-0.60)^2=0.16\), and \((0-0.40)^2=0.16\). Their sum is \(0.34\), so \(\operatorname{MSE}_B=0.34/4=0.0850\). B has the lower Brier score even though the classification errors tie. This is checked by observing that B's probabilities are closer overall to the observed zero-one outcomes.
The common mistake is to conclude that the two models are equivalent because both classify all four cases correctly. They are equivalent only under this particular classification-error measure on these cases; their probability predictions differ substantially.
Choosing a Measure for the Prediction Objective
| Prediction objective | Useful cross-validation measure | What the measure emphasizes |
|---|---|---|
| Predict a numeric response | Gaussian deviance or MSE | Squared distance between held-out values and predictions |
| Estimate event probabilities | Binomial deviance or binomial MSE | Quality of the probabilities, not only their resulting labels |
| Assign binary classes at a specified cutoff | Classification error | Whether each predicted class matches its observed class |
The appropriate choice depends on how predictions will be used. If a probability will feed into a pricing, reserving, or resource-allocation calculation, a probability-based loss is generally more informative than classification error alone. If the task is specifically to produce class labels under a defined decision rule, classification error can address that task directly. A decision rule with costs that differ between false positives and false negatives may require a cutoff other than \(0.5\); classification error at the standard cutoff would then not represent the stated objective.
In R, the relevant choice is explicit:
For binomial outcomes, use family = "binomial" and select type.measure = "deviance", "class", or "mse" according to the goal. For Gaussian outcomes, "mse" and "deviance" correspond to the same squared-error criterion. Cross-validation still compares held-out predictions across folds; the key choice here is what error is calculated for those predictions.
Common Exam Traps
- Reading classification error as probability quality. It discards the distance of probabilities from the observed outcomes once they fall on the same side of the cutoff.
- Confusing deviance with a class error rate. Deviance is a probability-based likelihood loss and can be large even when every case is assigned the correct class.
- Comparing measures on different scales as if values were directly comparable. MSE, deviance, and classification error have different definitions and scales; compare candidate models using the same measure.
- Forgetting that the cutoff affects classification. A probability prediction alone is not a class decision; the cutoff must be specified.
- Calling a lower cross-validation loss universally better. It is better for the objective represented by that loss, not necessarily for every downstream use.
Check Your Understanding
For each problem, identify the measure being used and calculate or interpret it before checking the answer.
- A Gaussian model has three held-out responses \(4,6,8\) and predictions \(5,6,10\). What is its MSE? (Answer: \(5/3=1.6667\), to four decimal places.)
- For a binary outcome \(y=0\), a model predicts \(p=0.2\). What are its squared-error contribution and deviance contribution? (Answer: \(0.0400\) and \(-2\ln(0.8)=0.4463\), respectively, to four decimal places.)
- Four binary outcomes are \(1,0,1,0\), and the predicted probabilities are \(0.60,0.40,0.70,0.20\). Using cutoff \(0.5\), what is classification error? (Answer: \(0/4=0\).)
- Two binomial models make the same correct class predictions on every held-out case, but one has probabilities closer to the observed zero-one outcomes. Which measure can distinguish their probability accuracy when classification error cannot? (Answer: Binomial deviance or binomial squared error/Brier score.)
- A team wants to rank cases by event probability for a decision that uses the probabilities directly, not just labels. Which of classification error, binomial deviance, and Brier score retains probability information? (Answer: Binomial deviance and Brier score; classification error after thresholding does not retain the full probability information.)