Mean Squared Error

Why Mean Squared Error Is Used

A prediction error records the difference between an observed response and its prediction. Averaging the signed errors can hide poor predictions: an overprediction of 8 and an underprediction of 8 cancel. Mean squared error avoids that cancellation by squaring each error before averaging. The result is a single summary of prediction error in which larger misses receive more weight.

Mean squared error (MSE) is used to summarize predictions of a quantitative response. It is aligned with the SOA Exam PA (Predictive Analytics) syllabus, where candidates need to calculate and interpret predictive performance measures in context. MSE is useful, but its interpretation depends on the response scale and on the fact that squaring emphasizes large errors.

Definition: For $n$ observed responses $y_1,\ldots,y_n$ and corresponding predictions $\hat{y}_1,\ldots,\hat{y}_n$, define the prediction error for observation $i$ as $e_i=y_i-\hat{y}_i$. The mean squared error is the arithmetic mean of the squared prediction errors: $$ \operatorname{MSE}=\frac{1}{n}\sum_{i=1}^{n}(y_i-\hat{y}_i)^2 =\frac{1}{n}\sum_{i=1}^{n}e_i^2. $$

The observations being scored must have both an observed response and a prediction. In a model assessment, those predictions should correspond to the particular data set being evaluated, such as a validation set. For this definition, each of the $n$ observations receives equal weight. If an analysis calls for unequal weights, it must state and use a different, weighted definition.

MSE is a mean of losses, not an estimate of the variance of a sample. Therefore its denominator is $n$, not $n-1$. The calculation itself does not require normally distributed errors. For a finite data set, it is defined whenever the observed responses and predictions are finite.

How Squaring Changes the Summary

Squaring has two important effects. First, it removes the sign of each error, so overpredictions and underpredictions do not cancel. Second, it makes the contribution increase faster than the error itself: an error twice as large contributes four times as much squared error. For example, an error of 6 contributes $6^2=36$, while an error of 2 contributes $2^2=4$. The first contributes nine times as much to the sum of squared errors, even though its absolute size is only three times as large.

This sensitivity can be useful when large misses are especially undesirable. It also means that a small number of unusually large errors can dominate MSE. MSE alone does not explain whether a high value comes from many moderate errors or one very large error; inspect the individual errors or another summary when that distinction matters.

Worked Example: Calculating MSE for Hypothetical Claim Costs

A hypothetical insurer compares predicted and observed claim costs for four policies. Both columns are reported in thousands of dollars. Calculate MSE for these four observations.

Policy	Observed cost $y_i$	Predicted cost $\hat{y}_i$
1	100	110
2	120	100
3	80	90
4	150	130

Method. Find each observed-minus-predicted error, square each error, add the squares, and divide by the number of policies. The common mistake avoided here is averaging the signed errors or dividing by $n-1$.

For policy 1, $e_1=100-110=-10$, so $e_1^2=100$. For policy 2, $e_2=120-100=20$, so $e_2^2=400$. For policy 3, $e_3=80-90=-10$, so $e_3^2=100$. For policy 4, $e_4=150-130=20$, so $e_4^2=400$.

The sum of squared errors is $100+400+100+400=1{,}000$, in squared thousands of dollars. There are four policies, so $$ \operatorname{MSE}=\frac{1{,}000}{4}=250. $$

The result is $250$ $(\text{thousand dollars})^2$. This is not an average error of 250 thousand dollars; the unit is squared. The check is that the four squared errors average to $(100+400+100+400)/4=250$, as required.

Why a Large Error Has More Influence

The next example compares two sets of predictions for the same observations. Using the same observed values matters: it keeps the comparison focused on the predictions rather than on differences in the cases being scored.

Worked Example: Comparing Two Error Patterns

A hypothetical analyst predicts a monthly service count for four districts. The observed counts are 12, 14, 16, and 18. Model A predicts 11, 15, 15, and 19; Model B predicts 12, 13, 16, and 23. Calculate each MSE and identify which model has the lower value.

Method. Calculate the errors and squared errors for each model, then divide each sum by four. The common mistake avoided is judging the models only by whether their errors are positive or negative.

For Model A, observed-minus-predicted errors are $12-11=1$, $14-15=-1$, $16-15=1$, and $18-19=-1$. Their squares are $1,1,1,1$, with sum $4$. Therefore, $\operatorname{MSE}_A=4/4=1$ count squared.

For Model B, the errors are $12-12=0$, $14-13=1$, $16-16=0$, and $18-23=-5$. Their squares are $0,1,0,25$, with sum $26$. Therefore, $\operatorname{MSE}_B=26/4=6.5$ count squared.

Model A has the lower MSE: $1<6.5$. Model B’s error of $-5$ contributes 25 to its total squared error, while each error of magnitude 1 contributes only 1. That one large miss accounts for $25/26$ of Model B’s sum of squared errors. The check is that the squared errors for each model sum to the stated total before division.

An MSE comparison should be interpreted with the response scale and the set of observations in view. Because each error is squared, changing the response from dollars to thousands of dollars changes the numerical MSE substantially. If all responses and predictions are divided by 1,000, each error is divided by 1,000 and each squared error, and thus MSE, is divided by $1{,}000{,}000$. MSE values from different response scales should not be compared as though their numerical values were directly equivalent.

Why the Mean Minimizes Squared Error for a Constant Prediction

A useful justification for squared error comes from the special case in which every observation receives the same prediction $c$. The choice of $c$ that minimizes the average squared error is the arithmetic mean of the observed values. This result helps explain why squared-error methods are naturally associated with mean predictions.

Derivation: For fixed observed values $y_1,\ldots,y_n$, define the squared-error criterion for a constant prediction $c$ by $L(c)=\frac{1}{n}\sum_{i=1}^{n}(y_i-c)^2$. Differentiating gives $L'(c)=\frac{2}{n}\sum_{i=1}^{n}(c-y_i)=2(c-\bar{y})$, where $\bar{y}=\frac{1}{n}\sum_{i=1}^{n}y_i$. Setting the derivative to zero gives $c=\bar{y}$. Since $L''(c)=2>0$, this stationary point is a minimum. Thus the mean observed value minimizes squared error among constant predictions.

Worked Example: Finding the Best Constant Prediction

A hypothetical portfolio team records three annual claim counts: 2, 4, and 6. It will use one constant prediction for all three observations. Find the constant that minimizes MSE, calculate that MSE, and compare it with the MSE from using the constant 3.

Method. Use the mean-minimization result for the best constant, then calculate both MSE values directly. The common mistake avoided is assuming that a convenient central value, such as the median or an endpoint, automatically minimizes squared error.

The mean of the three counts is $(2+4+6)/3=12/3=4$, so the minimizing constant is 4. With this prediction, the errors $y_i-\hat{y}_i$ are $2-4=-2$, $4-4=0$, and $6-4=2$. The squared errors are $4,0,4$, whose sum is 8. Hence the MSE is $8/3=2.6667$ count squared, rounded to four decimal places.

For the alternative constant 3, the errors are $2-3=-1$, $4-3=1$, and $6-3=3$. The squared errors are $1,1,9$, whose sum is 11. The MSE is $11/3=3.6667$ count squared, rounded to four decimal places. Since $8/3<11/3$, the mean-based prediction has lower MSE. This verifies the minimizing result for these data.

Common Mistakes and Exam Traps

Leaving errors signed. Averaging signed errors measures directional tendency, not MSE. Square each error before adding.
Dividing by $n-1$. MSE is the mean of the $n$ squared prediction errors, so divide by $n$. The $n-1$ denominator belongs to a different calculation, the usual sample variance.
Forgetting the units are squared. If the response is measured in dollars, MSE is in dollars squared; if it is measured in counts, MSE is in count squared. Do not describe MSE as an error in the original response units.
Ignoring the effect of one large error. A few large misses can dominate the result. Check individual squared errors when the MSE is unexpectedly high.
Comparing values from different response scales or cases. A numerical MSE comparison is most interpretable when both values use the same response scale and the same evaluation observations.

For example, if errors are $-2$ and $4$, the wrong calculation $\big((-2)+4\big)/2=1$ is a mean signed error, not MSE. The correction is $\big((-2)^2+4^2\big)/2=(4+16)/2=10$, in squared response units. This trap is particularly easy to miss when positive and negative errors happen to produce a plausible-looking average.

Key takeaway: MSE is the average of squared observed-minus-predicted errors. Squaring prevents cancellation and gives large errors disproportionate influence; the resulting metric is expressed in squared response units. State the response scale and the observations being scored when interpreting or comparing it.

Check Your Understanding

For each problem, calculate or interpret MSE using the definition above. Answers are provided for checking.

A hypothetical model predicts values 4, 4, and 6 when the observed values are 3, 5, and 8. What is the MSE? (Answer: $2$ response-units squared.)
For four hypothetical policies, observed costs are 20, 25, 30, and 35 thousand dollars, while predictions are 18, 28, 30, and 31 thousand dollars. Calculate MSE in squared thousands of dollars. (Answer: $7.25$ $(\text{thousand dollars})^2$.)
Three observed values are 100, 100, and 100. Predictions are 95, 100, and 110. Calculate the MSE and give its units if the response is measured in dollars. (Answer: $125/3=41.6667$ dollars squared, rounded to four decimal places.)
A constant prediction is to be chosen for observed values 1, 3, and 8. Find the constant that minimizes MSE and give the minimum MSE, rounded to four decimal places. (Answer: The constant is 4, and the minimum MSE is $26/3=8.6667$ response-units squared.)
For the same four observations, Model A has errors $-2,2,0,2$, while Model B has errors $0,0,-4,0$. Calculate both MSE values and state which is lower. (Answer: Model A has MSE $12/4=3$; Model B has MSE $16/4=4$. Model A is lower.)

What You'll Learn