Why the Choice of Loss Matters
A prediction is not good or poor in the abstract. Its quality depends on how prediction errors are measured. For a numerical response, a difference of 5 might count as 25 under squared error and 5 under absolute error. For a class label, the question may instead be whether the predicted class is right or wrong. These choices can rank the same models differently.
Generalization error is the expected prediction loss for new observations from the target population. An empirical average of losses on observed cases estimates that quantity when those cases represent the target population. But the estimate answers the question posed by the chosen loss, not every possible question about prediction quality. A loss that does not reflect the task can make a model appear preferable for the wrong reason.
The empirical average is meaningful only after the loss has been chosen. Squared and absolute errors are common for numerical responses; misclassification rate is common for categorical responses. The first two require, respectively, finite expected squared error and finite expected absolute error for the corresponding population risks to be finite. For classification, the loss is defined for a discrete class label and a predicted class.
Three Evaluation Targets
For numerical predictions, write the prediction for case \(i\) as \(\hat y_i\) and the observed response as \(y_i\). Mean squared error (MSE) averages squared residuals; mean absolute error (MAE) averages their absolute values. Root mean squared error (RMSE) is the square root of MSE. Squared error is in squared response units, while MAE and RMSE are in the original response units.
For a classification problem with \(K\) possible classes, let \(y_i\) be the true class and \(\hat c_i\) the predicted class. The zero-one loss is 1 when the prediction is wrong and 0 when it is right. Its empirical average is the misclassification rate.
Here \(I(A)\) equals 1 when statement \(A\) is true and 0 otherwise. The rate lies between 0 and 1; multiplying by 100 expresses it as a percentage. It counts errors equally: an incorrect prediction with a probability just below the decision threshold counts the same as one made with high confidence.
Worked Example: Two Numerical Models, Two Rankings
Worked Example: Two Numerical Models, Two Rankings
A hypothetical severity-monitoring team evaluates two models on four new cases. The observed responses are \(0,0,0,8\). Model A predicts \(2\) for every case; Model B predicts \(0\) for every case. Calculate MSE and MAE for both models, then identify which model each measure favors.
Method. Calculate each case’s residual \(y_i-\hat y_i\), square it for MSE, and take its absolute value for MAE. Divide each sum by \(n=4\).
Model A. Its residuals are \(0-2=-2\), \(-2\), \(-2\), and \(8-2=6\). The squared residuals are \(4,4,4,36\), with sum \(48\), so \(\operatorname{MSE}_A=48/4=12\). The absolute residuals are \(2,2,2,6\), with sum \(12\), so \(\operatorname{MAE}_A=12/4=3\).
Model B. Its residuals are \(0,0,0,8\). The squared residuals sum to \(64\), so \(\operatorname{MSE}_B=64/4=16\). The absolute residuals sum to \(8\), so \(\operatorname{MAE}_B=8/4=2\).
Conclusion and check. MSE favors Model A because \(12<16\); MAE favors Model B because \(2<3\). The large error of 8 receives a squared penalty of 64, which weighs heavily under MSE. The averages use the same denominator, four cases, and both models’ component losses sum to the stated totals.
Common mistake avoided. It would be wrong to say that one model is simply “better” without naming the loss. The two measures give opposite rankings, so the preferred model depends on whether large errors should receive a disproportionate penalty.
Why Squared Error and Absolute Error Have Different Targets
The loss also determines the best prediction when the predictors \(Z=z\) are fixed. Assume the conditional response has a finite second moment for squared error, and a finite first moment for absolute error. Under squared-error loss, the conditional mean minimizes expected loss. To see why, let \(a\) be any proposed numerical prediction and write \(\mu(z)=E[Y\mid Z=z]\). Expanding around that mean gives:
The cross term in the expansion is zero because \(E[Y-\mu(z)\mid Z=z]=0\). The first term does not depend on \(a\); the second is nonnegative and is zero at \(a=\mu(z)\). Thus the conditional mean minimizes squared-error risk. This result explains why a model evaluated by squared error is assessed in a way that rewards accurate conditional-mean predictions.
Under absolute-error loss, any conditional median minimizes expected loss. A median \(m(z)\) is a value with at least half the conditional probability at or below it and at least half at or above it. One way to justify the result is to examine how expected absolute loss changes with a proposed prediction \(a\): moving \(a\) to the right decreases the loss for observations to its right and increases it for observations to its left. At a minimum, neither side can contain more than half the probability. This is exactly the median condition. If the distribution has multiple medians, more than one prediction can minimize the loss.
Worked Example: Mean Versus Median for a Skewed Set of Responses
Suppose five hypothetical claim-review times, in minutes, are \(0,0,0,9,9\). Compare the constant predictions given by the sample mean, \(3.6\), and the sample median, \(0\), using both MSE and MAE.
Method. Treat each of the five observations as one case, calculate the absolute and squared deviations from each proposed constant, and divide each total by 5.
Prediction 3.6. The absolute deviations are \(3.6,3.6,3.6,5.4,5.4\). Their sum is \(10.8+10.8=21.6\), giving \(\operatorname{MAE}=21.6/5=4.32\). The squared deviations are \(12.96,12.96,12.96,29.16,29.16\). Their sum is \(38.88+58.32=97.20\), giving \(\operatorname{MSE}=97.20/5=19.44\).
Prediction 0. The absolute deviations are \(0,0,0,9,9\), with sum \(18\), so \(\operatorname{MAE}=18/5=3.6\). The squared deviations are \(0,0,0,81,81\), with sum \(162\), so \(\operatorname{MSE}=162/5=32.4\).
Conclusion and check. The median prediction has lower MAE (\(3.6<4.32\)), while the mean prediction has lower MSE (\(19.44<32.4\)). The mean is \(18/5=3.6\), and 0 is a median because three of the five values are at or below 0 and at least three are at or above 0.
Common mistake avoided. Do not assume a prediction minimizing one loss must minimize the other. Here the mean is optimal for squared deviations, but the median is optimal for absolute deviations.
Classification: Class Decisions Versus Probability Predictions
For classification, a model may produce estimated class probabilities before it produces a class label. Under zero-one loss, if the conditional probability of class \(k\) is \(P(Y=k\mid Z=z)\), predicting \(k\) has conditional expected loss \(1-P(Y=k\mid Z=z)\). Therefore the class with the largest conditional probability minimizes expected misclassification loss. This conclusion concerns the class decision; it does not say that misclassification rate measures the quality of the probability estimates.
For a binary response coded 0 or 1, a predicted probability \(\hat p_i\) can also be evaluated as a numerical prediction using squared or absolute error. Squared error on binary outcomes is commonly called the Brier score. To calculate misclassification rate from probabilities, a decision threshold must first be specified. A threshold of 0.5 is common, but it is a convention, not a mathematical requirement; changing it can change the predicted classes and the error rate.
Worked Example: Probabilities and Class Decisions Give Different Rankings
A hypothetical screening model is evaluated on four cases with true binary outcomes \(1,1,0,0\). Model A assigns probabilities \(0.51,0.51,0.49,0.49\); Model B assigns \(0.90,0.10,0.10,0.10\). Use a threshold of 0.50, with probabilities at least 0.50 classified as 1. Calculate misclassification rate, MAE, and squared error for each model.
Method. First threshold the probabilities to obtain class predictions. Then calculate the indicator errors for misclassification rate and the probability residuals for MAE and mean squared error.
Model A. Its class predictions are \(1,1,0,0\), so all four are correct and the misclassification rate is \(0/4=0\). Each absolute probability error is \(0.49\), so \(\operatorname{MAE}=4(0.49)/4=0.49\). Each squared error is \(0.49^2=0.2401\), so \(\operatorname{MSE}=4(0.2401)/4=0.2401\).
Model B. Its class predictions are \(1,0,0,0\), so the second case is wrong and the misclassification rate is \(1/4=0.25\). The absolute probability errors are \(0.10,0.90,0.10,0.10\), totaling \(1.20\), so \(\operatorname{MAE}=1.20/4=0.30\). The squared errors are \(0.01,0.81,0.01,0.01\), totaling \(0.84\), so \(\operatorname{MSE}=0.84/4=0.21\).
Conclusion and check. Misclassification rate favors Model A (\(0<0.25\)), while both probability-error measures favor Model B (\(0.30<0.49\) and \(0.21<0.2401\)). The four outcomes and four predictions are used in each average.
Common mistake avoided. Counting only correct labels would hide how close Model A’s probabilities are to the threshold and how far Model B’s incorrect probability is from the true outcome. Misclassification rate and probability-error measures answer different questions.
Choosing and Interpreting an Evaluation Loss
The evaluation measure should match the prediction goal and the consequences of error. Squared error may be appropriate when large numerical misses deserve especially strong penalties. Absolute error grows linearly with the size of a miss and has a direct interpretation in response units. Misclassification rate is straightforward when the only concern is whether the assigned class is correct and all mistakes have equal consequences.
A reported score does not identify a universally best model. It describes performance under a particular loss, on particular observations. When comparing scores, check that the response, units, denominator, class threshold, and loss definition are the same. A numerical MSE cannot be compared directly with an MAE as though they were on the same scale; even RMSE and MAE, while both in response units, emphasize errors differently.
Check Your Understanding
For each problem, identify the requested loss before comparing the predictions.
- Observed numerical responses are \(2,5,9\). Model A predicts \(4,4,4\), while Model B predicts \(2,6,8\). Calculate each model’s MAE and MSE and state which model each measure favors. (Answer: Model A has MAE \(8/3\) and MSE \(10\); Model B has MAE \(2/3\) and MSE \(2/3\). Both measures favor Model B.)
- Four responses are \(1,1,1,7\). Compare constant predictions 1 and 2 under MAE and MSE. Which prediction minimizes each? (Answer: Prediction 1 has MAE \(1.5\) and MSE \(9\); prediction 2 has MAE \(2\) and MSE \(7\). MAE selects 1 and MSE selects 2.)
- True binary labels are \(0,1,1\), with predicted probabilities \(0.40,0.55,0.80\). Using threshold 0.50, calculate misclassification rate, MAE, and MSE. (Answer: Misclassification rate \(0\); MAE \(0.3500\); MSE \(0.1342\) to four decimals.)
- True labels are \(1,0,0,1\). Model A assigns probabilities \(0.60,0.40,0.40,0.60\), and Model B assigns \(0.90,0.10,0.80,0.20\). With threshold 0.50, compare misclassification rate and mean squared probability error. (Answer: Model A has misclassification rate \(0\) and MSE \(0.1600\); Model B has misclassification rate \(0.5000\) and MSE \(0.3250\). Model A is preferred under both measures.)