Why Give Different Errors Different Costs?
A numerical prediction can be wrong in different directions, and those directions may not have equal consequences. In a hypothetical reserve-planning exercise, understating the amount needed could create a funding shortfall, while overstating it could tie up resources. A conventional error summary treats equal-sized misses equally; a cost-sensitive summary instead assigns each miss the decision cost specified for its size and direction.
The key is to keep two ideas separate. A cost coefficient describes the consequence of a unit of prediction error. An observation weight describes how much a record represents in the summary, such as its exposure. Applying both can be appropriate, but they play different roles and enter the calculation differently.
This definition uses a linear cost per unit of miss: doubling the error in one direction doubles its loss in that direction. It is appropriate only when that is the stated or intended decision rule. If a problem specifies a fixed penalty for any miss, a threshold penalty, or a nonlinear cost, use that function instead of silently imposing a linear one. The displayed formula also does not by itself determine the costs; those must come from the decision context or be explicitly supplied.
Building the Summary Correctly
The positive-part expressions in the definition select exactly one direction of error. When \(e_i>0\), \(\max(e_i,0)=e_i\) and \(\max(-e_i,0)=0\), so \(L_i=c_i^-e_i\). When \(e_i<0\), the first maximum is zero and the second is \(-e_i=|e_i|\), so \(L_i=c_i^+|e_i|\). An exact prediction has \(e_i=0\) and loss zero.
The observation weight is applied after the loss is calculated. It does not change whether the prediction is an over- or underprediction, nor does it change the cost per unit of error. The denominator \(\sum_i w_i\) makes the result a mean loss per unit of the represented observation weight. If the goal is instead to report total weighted cost, use the numerator alone and label it as a total.
Calculate \(e_i=y_i-\hat y_i\), so positive values mean underprediction and negative values mean overprediction.
Use the underprediction cost when \(e_i>0\), the overprediction cost when \(e_i<0\), and zero loss when \(e_i=0\).
Multiply each loss by its observation or exposure weight. Do not substitute a cost coefficient for this weight.
Sum weighted losses for total weighted cost; divide that sum by total observation weight for mean cost per weight unit.
Worked Calculations
Worked Example: Unequal Costs for Underprediction and Overprediction
A hypothetical service has actual workloads of 10 and 20 units. Two forecasting models make predictions of 8 and 22 (Model A), and 9 and 16 (Model B), respectively. Treat the two observations equally. Underprediction costs 4 cost-units per workload unit, while overprediction costs 1 cost-unit per workload unit. Calculate each model’s mean asymmetric loss and identify the lower-loss model.
Method. For each prediction, calculate \(e=y-\hat y\), apply the cost for its direction, and average the two losses.
Model A, observation 1. The error is \(10-8=2\), an underprediction. Its loss is \(4(2)=8\).
Model A, observation 2. The error is \(20-22=-2\), an overprediction. Its loss is \(1(2)=2\). The mean loss is \((8+2)/2=5.00\) cost-units per observation.
Model B, observation 1. The error is \(10-9=1\), an underprediction. Its loss is \(4(1)=4\).
Model B, observation 2. The error is \(20-16=4\), also an underprediction. Its loss is \(4(4)=16\). The mean loss is \((4+16)/2=10.00\) cost-units per observation. Model A has the lower mean asymmetric loss.
Check and common mistake. Model B’s second prediction is below the actual workload, so its error is an underprediction, not an overprediction. Using the overprediction cost for that error would incorrectly give Model B a mean of \((4+4)/2=4\). Checking each signed error before choosing a cost prevents this direction error.
Worked Example: Adding Exposure Weights Without Changing Costs
A hypothetical portfolio has three records with actual amounts of 1,000, 500, and 200 dollars and predictions of 900, 550, and 180 dollars. Their exposure weights are 1, 2, and 3 units. The decision cost is 3 dollars per 1,000 dollars of underprediction and 1 dollar per 1,000 dollars of overprediction. Find the weighted mean cost per exposure unit.
Method. First calculate each record’s loss from its error direction and size. Then multiply those losses by exposure and divide by total exposure.
For the first record, \(e_1=1{,}000-900=100\) dollars, an underprediction of \(0.1\) thousand dollars. Its loss is \(3(0.1)=0.30\) dollars per exposure unit; after applying weight 1, its contribution is \(1(0.30)=0.30\) dollars.
For the second record, \(e_2=500-550=-50\) dollars, an overprediction of \(0.05\) thousand dollars. Its loss is \(1(0.05)=0.05\) dollars per exposure unit; its weighted contribution is \(2(0.05)=0.10\) dollars.
For the third record, \(e_3=200-180=20\) dollars, an underprediction of \(0.02\) thousand dollars. Its loss is \(3(0.02)=0.06\) dollars per exposure unit; its weighted contribution is \(3(0.06)=0.18\) dollars.
The total weighted loss is \(0.30+0.10+0.18=0.58\) dollars, and total exposure is \(1+2+3=6\) units. Therefore, the weighted mean is \(0.58/6=0.0967\) dollars per exposure unit, rounded to four decimal places. The total weighted cost is \(0.58\) dollars.
Check and common mistake. The exposure weights affect the contributions and denominator, but they are not the underprediction or overprediction costs. Dividing by the sum of the costs, or multiplying errors by exposure without first applying the directional cost, would describe a different calculation.
Worked Example: When Cost and MAE Rank Models Differently
Four hypothetical predictions are made for actual values \(120, 80, 60, 40\). Model A predicts \(110, 90, 55, 42\); Model B predicts \(110, 75, 55, 40\). Observations have equal weight. Underprediction costs 5 units per unit of error and overprediction costs 2. Compare the models by mean absolute error (MAE) and by mean asymmetric loss.
Method. Calculate each model’s absolute errors and directional losses separately. MAE ignores direction; the stated cost rule does not.
For Model A, the signed errors are \(10,-10,5,-2\). Its absolute errors total \(10+10+5+2=27\), so its MAE is \(27/4=6.75\). Its asymmetric losses are \(5(10)=50\), \(2(10)=20\), \(5(5)=25\), and \(2(2)=4\). These sum to 99, giving mean asymmetric loss \(99/4=24.75\).
For Model B, the signed errors are \(10,5,5,0\). Its absolute errors total \(10+5+5+0=20\), so its MAE is \(20/4=5.00\). All nonzero errors are underpredictions; their losses are \(5(10)=50\), \(5(5)=25\), and \(5(5)=25\), with zero loss for the exact prediction. The mean asymmetric loss is \(100/4=25.00\).
Model B has lower MAE, but Model A has lower mean asymmetric loss. The cost-sensitive choice is Model A under the specified decision costs. The comparison is checked by summing all four losses for each model before dividing by four; averaging only the nonzero errors would use the wrong denominator.
Why the Weighted Mean Has This Form
The formula for \(\overline L_w\) follows from what the summary is intended to measure. Each observation contributes a loss \(L_i\) for each unit of weight it represents. Its total represented loss is therefore \(w_iL_i\). Adding these contributions gives total weighted loss, \(\sum_i w_iL_i\). The total represented weight is \(\sum_i w_i\). Dividing total weighted loss by total weight gives loss per represented weight unit:
This justification assumes that the weights have the interpretation intended by the analysis and that their sum is positive. It does not establish that a chosen cost function is appropriate, or that an exposure-weighted result represents every possible target population. Those are modeling and decision questions, not algebraic consequences of the formula.
Cost Weights and Observation Weights Are Not Interchangeable
| Quantity | What it represents | Where it enters |
|---|---|---|
| Directional cost \(c_i^-\) or \(c_i^+\) | Consequence per unit of underprediction or overprediction | Inside the individual loss \(L_i\) |
| Observation weight \(w_i\) | How much an observation represents in the summary | Multiplies \(L_i\) in the numerator and contributes to the denominator |
A cost coefficient may have monetary units, such as dollars of consequence per unit of error. An exposure weight may be measured in policy-years or another exposure unit. Their units should be considered when interpreting the result. Multiplying by exposure produces weighted cost contributions; dividing by total exposure gives mean cost per exposure unit.
A cost-sensitive summary also answers a different question from an ordinary absolute-error summary. MAE describes average error magnitude without distinguishing direction. The asymmetric loss describes average loss under a specified decision rule. Neither number is universally preferable: the relevant measure depends on the decision being evaluated. For comparisons to be meaningful, use the same error convention, cost rule, and weighting interpretation for every model.
Common Mistakes
- Reversing error direction. With \(e_i=y_i-\hat y_i\), a positive error is an underprediction. If a different convention is adopted, state it and apply it consistently.
- Applying costs before identifying direction. A cost for underprediction does not apply to an overprediction just because its absolute error is large.
- Using cost coefficients as observation weights. Costs determine individual losses; observation weights determine representation in the mean.
- Reporting a total as a mean. A total weighted loss is \(\sum_i w_iL_i\); a mean per weight unit divides by \(\sum_i w_i\).
- Assuming the linear rule is universal. The asymmetric absolute-loss formula is not justified when the decision cost is fixed, threshold-based, or nonlinear unless that rule is explicitly represented.
Check Your Understanding
For each problem, identify the error direction before applying the stated costs. Assume the cost rule is linear in the size of the miss unless the problem says otherwise.
- A workload is 30 units and a forecast is 34. Underprediction costs 4 per unit and overprediction costs 2 per unit. Find the loss and state its direction. (Answer: overprediction; loss \(=2(4)=8\) cost-units.)
- Two equally weighted records have actual values 12 and 20 and predictions 10 and 24. Underprediction costs 3 per unit and overprediction costs 1 per unit. Calculate the mean asymmetric loss. (Answer: losses \(3(2)=6\) and \(1(4)=4\); mean \(=(6+4)/2=5\) cost-units per observation.)
- Two records have losses of 8 and 5 and exposure weights of 2 and 4. Find total weighted loss and mean loss per exposure unit. (Answer: total \(=2(8)+4(5)=36\); mean \(=36/(2+4)=6\) loss-units per exposure unit.)
- Two models have signed errors \(3,-5\) and \(-2,4\), respectively, on two equally weighted observations. Underprediction costs 2 per unit and overprediction costs 1 per unit. Calculate each model’s mean asymmetric loss and select the lower-loss model. (Answer: Model 1 losses \(6,5\), mean \(5.5\); Model 2 losses \(2,8\), mean \(5.0\); Model 2 has the lower mean loss.)
- A team reports a weighted loss numerator of 42 and total observation weight of 7. If the reported measure is intended to be mean loss per weight unit, calculate it and explain why 42 alone is not that mean. (Answer: \(42/7=6\) loss-units per weight unit; 42 is the total weighted loss.)