Evaluating Pure-Premium Predictions

Why Score Pure-Premium Predictions at the Policy Level?

A pure-premium model predicts expected loss per unit of exposure. Validation data, however, usually record each policy’s total loss during its observed exposure. A policy with six months of exposure should not automatically count as much as a full-year policy, and a zero-loss policy is still informative: it is an observed outcome that the model must score.

This tutorial defines an exposure-aware Tweedie deviance score and several aggregate error summaries. These measures answer different questions. Deviance assesses how well predictions fit the distributional behavior of the observations, while aggregate error describes whether predicted total losses are above or below the observed total. Neither summary should be interpreted as a substitute for the other.

Definition: For policy $i$, let $e_i>0$ be exposure in a chosen unit, $L_i\geq 0$ the observed aggregate loss for that exposure, and $m_i>0$ the predicted pure premium per exposure unit. The observed policy pure premium is $z_i=L_i/e_i$. The predicted aggregate loss is $e_i m_i$, and the predicted pure premium remains $m_i$. All policies must use the same exposure unit, loss definition, valuation basis, and prediction period.

The unit of exposure might be one earned vehicle-year, one property-year, or another appropriate measure. For example, if $e_i=0.5$ vehicle-year and the model predicts $m_i=\$400$ per vehicle-year, its predicted aggregate loss for that policy is $\$200$. The prediction is not $\$400$ of total loss unless exposure equals one.

Exposure-Weighted Tweedie Deviance

For nonnegative aggregate loss with a possible point mass at zero and a continuous positive-loss distribution, a common model is a Tweedie compound Poisson-gamma model with power parameter $1<p<2$. Under the exposure-scaled form used here, conditional on the rating information, the mean and variance of policy loss are

E[L_i]=e_i m_i,\qquad \operatorname{Var}(L_i)=\phi e_i m_i^p,

where $\phi>0$ is a dispersion parameter. This specification assumes that expected loss and variance scale with exposure and that the same Tweedie power and dispersion convention apply across the policies being compared. The model permits zero aggregate losses as well as positive losses. It is not an appropriate scoring model for negative outcomes or for a target that is not aggregate nonnegative loss.

Definition: For $z\geq 0$, $m>0$, and $1<p<2$, the Tweedie unit deviance is $$ d_p(z,m)=2\left[\frac{z^{2-p}}{(1-p)(2-p)}-\frac{z m^{1-p}}{1-p}+\frac{m^{2-p}}{2-p}\right], $$ where the first two terms are defined by their limits at $z=0$. In particular, $$ d_p(0,m)=\frac{2m^{2-p}}{2-p}. $$ For policy $i$, the exposure-weighted contribution is $e_i d_p(L_i/e_i,m_i)$. The unscaled validation deviance is $$ D_p=\sum_i e_i d_p(L_i/e_i,m_i). $$

A smaller deviance indicates better fit under this scoring rule. If a scaled deviance is reported, it is $D_p/\phi$, provided $\phi$ is defined consistently. When comparing models with the same $p$ and the same dispersion convention, the common positive scaling does not change their ranking.

Why the Exposure Weight Appears

The exposure weight is not an optional adjustment. It follows from treating the observed rate $z_i=L_i/e_i$ as a rate based on $e_i$ units of exposure. Its conditional mean is $m_i$, and its variance is $\phi m_i^p/e_i$, because dividing aggregate loss by $e_i$ divides variance by $e_i^2$:

\operatorname{Var}(L_i/e_i) =\frac{\phi e_i m_i^p}{e_i^2} =\frac{\phi m_i^p}{e_i}.

For a variance function proportional to $m^p$, the unit deviance is obtained by integrating the difference between the observed rate and the fitted mean against the inverse variance function. For fixed $z$, this gives

2e_i\int_{m_i}^{z_i}\frac{z_i-t}{\phi t^p}\,dt =\frac{e_i}{\phi}d_p(z_i,m_i).

The factor $e_i$ appears because the rate’s variance is inversely proportional to exposure. Summing over policies gives the exposure-weighted deviance above (or its dispersion-scaled version). This also explains why assigning equal weight to each policy’s rate deviance would not represent the stated model when exposures differ.

Worked Example: Scoring Zero and Positive Losses

A hypothetical validation set contains three policies. Exposure is measured in vehicle-years. The observed aggregate losses are $\$0$, $\$8{,}000$, and $\$8{,}000$, with exposures $1$, $2$, and $0.5$, respectively. Predicted pure premiums are $\$1{,}000$, $\$4{,}000$, and $\$9{,}000$ per vehicle-year. Calculate the unscaled Tweedie deviance for $p=1.5$. Use thousands of dollars throughout the calculation.

Method. Compute each observed rate $z_i=L_i/e_i$, evaluate its unit deviance, and multiply by exposure. For $p=1.5$, the definition simplifies to

d_{1.5}(z,m)=4\left(\frac{z}{\sqrt{m}}+\sqrt{m}-2\sqrt{z}\right), \qquad d_{1.5}(0,m)=4\sqrt{m}.

Policy 1. The observed rate is $z_1=0/1=0$, and the prediction is $m_1=1$. Thus $d_{1.5}(0,1)=4\sqrt{1}=4$. Its weighted contribution is $1(4)=4$.

Policy 2. The observed rate is $z_2=8/2=4$, and $m_2=4$. Therefore $d_{1.5}(4,4)=4(4/2+2-2\cdot2)=4(0)=0$. Its weighted contribution is $2(0)=0$.

Policy 3. The observed rate is $z_3=8/0.5=16$, and $m_3=9$. Therefore $d_{1.5}(16,9)=4(16/3+3-8)=4(1/3)=4/3$. Its weighted contribution is $0.5(4/3)=2/3$.

The total is $D_{1.5}=4+0+2/3=14/3$, or $4.667$ to three decimals. As a check, the second policy has observed rate equal to its prediction, so its deviance is zero; the zero-loss policy has a positive contribution because its prediction was positive.

Common mistake avoided: Substituting zero into only the positive-$z$ terms and treating the resulting contribution as undefined or omitting the policy is wrong. The limiting value at zero is $2m^{2-p}/(2-p)$, and the policy still receives its exposure weight.

Aggregate Error Summaries

Aggregate summaries answer whether predictions are systematically high or low in total, or how large their errors are without regard to direction. For $n$ policies, define observed and predicted portfolio losses as

L_{\mathrm{obs}}=\sum_{i=1}^n L_i,\qquad L_{\mathrm{pred}}=\sum_{i=1}^n e_i m_i.

The signed aggregate error is $L_{\mathrm{pred}}-L_{\mathrm{obs}}$. A negative value means the predictions understate the observed total; a positive value means they overstate it. Dividing by total exposure gives an error per exposure unit:

E_{\mathrm{agg}}= \frac{\sum_i e_i m_i-\sum_i L_i}{\sum_i e_i}.

A useful magnitude summary is aggregate absolute error per exposure, $\sum_i|e_i m_i-L_i|/\sum_i e_i$. Aggregate squared error per exposure can also be reported as $\sum_i(e_i m_i-L_i)^2/\sum_i e_i$, with units of squared currency per exposure. These are distinct measures: signed errors can cancel, while absolute and squared errors cannot. State the chosen measure and its denominator rather than calling all of them simply “average error.”

Worked Example: Separating Portfolio Bias From Error Magnitude

For a hypothetical property portfolio, three policies have exposures $0.5$, $1$, and $2$ property-years; observed aggregate losses of $\$0$, $\$6{,}000$, and $\$20{,}000$; and predicted pure premiums of $\$4{,}000$, $\$8{,}000$, and $\$6{,}000$ per property-year. Calculate the signed aggregate error, signed error per exposure, and aggregate absolute error per exposure. Work in thousands of dollars.

Method. First convert each predicted rate to a predicted aggregate loss. The predictions are $0.5(4)=2$, $1(8)=8$, and $2(6)=12$, for $L_{\mathrm{pred}}=2+8+12=22$. Observed aggregate loss is $L_{\mathrm{obs}}=0+6+20=26$. Thus the signed aggregate error is $22-26=-4$ thousand dollars: the model underpredicts the portfolio total by $\$4{,}000$.

Total exposure is $0.5+1+2=3.5$ property-years. The signed error per exposure is $-4/3.5=-1.142857$ thousand dollars per property-year, or $-\$1{,}142.86$ per property-year.

The absolute aggregate errors are $|2-0|=2$, $|8-6|=2$, and $|12-20|=8$, totaling $12$ thousand dollars. Therefore aggregate absolute error per exposure is $12/3.5=3.428571$ thousand dollars per property-year, or $\$3{,}428.57$ per property-year. Rounded only at the final step, the signed per-exposure error is $-\$1{,}142.86$ and the absolute error is $\$3{,}428.57$.

Check. The signed policy errors $2$, $2$, and $-8$ sum to $-4$, matching predicted total minus observed total. The absolute errors sum to $12$, which is larger because positive and negative errors do not cancel.

Common mistake avoided: Comparing the unweighted mean of policy-level pure-premium errors with the portfolio aggregate error gives a different quantity. The portfolio total is formed from $e_i m_i$ and $L_i$, then divided by total exposure only if a per-exposure result is wanted.

Different Scores Can Give Different Rankings

A model can have a smaller Tweedie deviance but a larger absolute aggregate error. Deviance evaluates policy-level discrepancies under a specified distributional variance function; the aggregate signed error focuses only on the portfolio total. A model’s errors may offset in aggregate even when its policy-level predictions are poor, or it may predict the total well while allocating losses poorly across policies.

Worked Example: Comparing Two Predictions by Two Measures

Use the three policies from the first worked example: exposures $1$, $2$, and $0.5$ vehicle-years, observed aggregate losses $0$, $8$, and $8$ thousand dollars, and Tweedie power $p=1.5$. Candidate A predicts pure premiums $1$, $4$, and $9$; Candidate B predicts $4$, $1$, and $16$, all in thousands per vehicle-year. Compare their unscaled Tweedie deviance and absolute aggregate error.

Candidate A deviance. The weighted contributions are $4$, $0$, and $2/3$, as calculated previously, so $D_A=14/3=4.667$.

Candidate B deviance. For policy 1, $z=0,m=4$, so the contribution is $1(4\sqrt{4})=8$. For policy 2, $z=4,m=1$, so $d_{1.5}(4,1)=4(4+1-4)=4$, giving contribution $2(4)=8$. For policy 3, $z=16,m=16$, so $d_{1.5}(16,16)=4(16/4+4-8)=0$, giving contribution $0.5(0)=0$. Thus $D_B=8+8+0=16$; Candidate A has lower deviance.

Absolute aggregate error. The observed total is $16$. Candidate A predicts $1(1)+2(4)+0.5(9)=13.5$, so its absolute aggregate error is $|13.5-16|=2.5$. Candidate B predicts $1(4)+2(1)+0.5(16)=14$, so its absolute aggregate error is $|14-16|=2$. Candidate B is closer on this aggregate measure.

Interpretation and check. The two criteria rank the candidates differently: A has lower deviance, while B has lower absolute aggregate error. The totals use the same exposure-weighted calculation, and neither ranking is an arithmetic contradiction. The score must match the validation question being asked.

Common mistake avoided: Do not infer that the candidate with the better aggregate total must have better policy-level predictive fit. Conversely, a lower deviance alone does not prove that aggregate indicated losses are closer.

Practical Scoring Checks

Before reporting any validation score, confirm that the observed and predicted quantities match. Losses should use a consistent valuation and maturity basis, and predictions should correspond to the same coverage, time period, and loss definition. Exposure must be positive for every scored policy and measured in the unit used for the predicted pure premium. A policy with zero exposure has no observed rate $L_i/e_i$ and should not be scored as if it had one.

Keep zero-loss policies. Zero observed loss is valid for a nonnegative aggregate-loss target and has a defined Tweedie deviance contribution.
Use the correct prediction unit. Multiply a per-exposure pure premium by exposure to obtain predicted aggregate loss.
Weight rate deviance by exposure. When scoring policy rates under the exposure-scaled Tweedie formulation, use $e_i d_p(L_i/e_i,m_i)$.
Label aggregate summaries. State whether a reported error is signed, absolute, squared, and whether it is a portfolio total or per exposure unit.
Hold the scoring setup fixed. Tweedie power, exposure unit, loss basis, and dispersion treatment must be consistent when comparing candidate predictions.

Key takeaway: Score pure-premium predictions against observed policy rates $L_i/e_i$, but account for exposure. Tweedie deviance captures distribution-based policy-level fit, while aggregate error summarizes portfolio-level overprediction or underprediction; report both when both questions matter.

Check Your Understanding

For each problem, calculate with the stated exposure unit and keep zero-loss policies in the score.

A policy has exposure $0.5$ vehicle-year, observed loss $\$2{,}000$, and predicted pure premium $\$6{,}000$ per vehicle-year. Find its observed pure premium and predicted aggregate loss. (Answer: observed pure premium $\$4{,}000$ per vehicle-year; predicted aggregate loss $\$3{,}000$)
For $p=1.5$, a policy has exposure $2$, observed aggregate loss $0$, and predicted rate $m=9$ in thousands per exposure unit. Find its exposure-weighted deviance contribution. (Answer: $2[4\sqrt{9}]=24$)
Two policies have exposures $1$ and $2$, observed aggregate losses $3$ and $10$ thousand dollars, and predicted rates $4$ and $4$ thousand dollars per exposure unit. Find signed aggregate error and signed error per exposure. (Answer: predicted total $12$, observed total $13$, signed error $-1$ thousand dollars, or $-1/3$ thousand dollars per exposure unit)
For $p=1.5$, a policy has exposure $0.5$, observed aggregate loss $8$, and predicted rate $m=4$, with monetary values in thousands. Calculate its weighted deviance contribution. (Answer: observed rate $z=16$; $d_{1.5}(16,4)=4(16/2+2-8)=8$, weighted contribution $0.5(8)=4$)
A validation portfolio has total exposure $4$, observed total loss $\$30{,}000$, and predicted total loss $\$34{,}000$. Find signed aggregate error per exposure and aggregate absolute error per exposure, given that the sum of absolute policy-level aggregate errors is $\$10{,}000$. (Answer: signed error $+\$1{,}000$ per exposure; absolute error $\$2{,}500$ per exposure)

What You'll Learn