Defining the Validation Target and Prediction Unit

Why the Validation Target Must Be Defined First

A model prediction is meaningful only in relation to a specific outcome, time period, and unit. A frequency model might predict claims per policy-year, a severity model might predict dollars per claim, and a pure-premium model might predict dollars per policy-year. These are different quantities. Comparing any of them with an outcome measured on a different basis can make a sound model appear inaccurate—or make a flawed comparison look reasonable.

For validation, specify the target before calculating a score or summarizing prediction errors. State what is being predicted, over what forecast horizon, and for which observational unit. Exposure is central to this definition: a policy observed for three months does not contribute the same amount of time at risk as a policy observed for a full year.

Definition: The prediction target is the outcome a model is intended to predict. The forecast horizon is the period over which that outcome is measured. The unit of analysis is the entity for which one target and its corresponding prediction are defined, such as one policy-period or one claim.

These three choices should agree. For example, if the target is the number of claims during a six-month policy period, a corresponding prediction must be an expected count for that six-month period—not an annualized claim rate unless the comparison also converts the observed count to an annualized basis.

Frequency: Counts Over an Exposure Period

A frequency model predicts the number of claims associated with a risk during a stated exposure period. Let $N_i$ denote the claim count for observational unit $i$, and let $e_i$ be its exposure measured in a stated unit, commonly policy-years. A frequency rate $\lambda_i$ is expressed in claims per exposure unit. Under a Poisson frequency model with constant rate over the period and independent claim occurrences,

N_i \sim \operatorname{Poisson}(\lambda_i e_i), \qquad E[N_i]=\lambda_i e_i.

The model’s rate and the target count are not interchangeable. The rate is a frequency per unit of exposure; the expected count is the rate multiplied by the exposure represented by the prediction unit. If the unit is a policy-period with $e_i=0.5$ policy-years, its expected count is half the annual rate.

A frequency target also needs a precise event definition. The count might include all covered claims occurring during the period, or only claims meeting a stated coverage and reporting definition. Whatever definition is used, the observed count must refer to the same kind of claim and the same period as the prediction.

Worked Example: Converting an Annual Frequency Rate to a Policy-Period Target

A hypothetical equipment insurer uses a Poisson frequency model with rate $\lambda=0.12$ claims per policy-year. Each policy-period in a particular group covers six months, or $e=0.5$ policy-years. Find the expected claim count and the probability of at least one claim per policy-period. Assume the rate is constant over the period and claim occurrences follow the stated Poisson model. There is no discounting or interest adjustment; mortality is not relevant.

Method. Convert the annual rate to the expected count over the policy’s actual exposure. The Poisson mean is:

\lambda e = 0.12(0.5)=0.06 \text{ claims per policy-period}.

Thus, $N\sim\operatorname{Poisson}(0.06)$, and the expected count is $E[N]=0.06$. The probability of at least one claim is one minus the probability of zero claims:

P(N\geq 1)=1-P(N=0) =1-e^{-0.06} =0.058235\ldots \approx 0.0582.

For 80 such policy-periods, total exposure is $80(0.5)=40$ policy-years, so the expected portfolio count is $0.12(40)=4.8$. The calculation checks: $80(0.06)=4.8$. Common mistake avoided: treating the annual rate, 0.12, as the expected count for each six-month policy-period. The period-specific expected count is 0.06.

Severity: Dollars Per Claim

A severity model predicts the amount associated with one claim, not the number of claims on a policy. Its unit of analysis is generally a claim record, with one response value per claim under a stated payment definition. Examples include indemnity paid per covered claim or total covered loss per claim. The target should specify whether the amount is gross or net of a deductible, limited by a policy maximum, and measured at a particular valuation basis.

Exposure is not usually the denominator for a claim-level severity prediction: a claim is the observation. A policy with more exposure can generate more claims, but its claim-level severity outcome remains a dollar amount for each claim. A collection of claim records may therefore have many rows for one policy, while a policy-period frequency or pure-premium data set may have one row for that same policy-period.

For example, if claim payments are modeled as Gamma distributed with shape $\alpha$ and scale $\theta$, the model-implied mean payment per claim is $\alpha\theta$, and the variance is $\alpha\theta^2$. This is a distributional statement about the chosen claim-level target; it does not itself predict the number of claims.

Worked Example: Interpreting a Claim-Level Severity Prediction

A hypothetical insurer models covered payments per claim using a Gamma distribution with shape $\alpha=4$ and scale $\theta=\$250$. The target is the payment for one covered claim, with no deductible or limit adjustment. For a set of 75 claim records, calculate the model-implied average payment per claim and the sum of the predicted means. Assume independent claim amounts with the stated common distribution. There is no discounting or interest adjustment; mortality is not relevant.

Method. For this Gamma parameterization, the mean is shape times scale:

E[Y]=\alpha\theta=4(\$250)=\$1{,}000.

Each claim record therefore has a predicted mean payment of $1,000. Across 75 claim records, the sum of predicted means is:

75(\$1{,}000)=\$75{,}000.

As a check on the specified distribution, its variance is $\alpha\theta^2=4(\$250)^2=\$250{,}000$, corresponding to a standard deviation of $500. The average predicted payment remains $1,000 per claim; it is not $75,000 per claim. Common mistake avoided: treating the 75 claim rows as 75 policy-periods or dividing their predicted payments by policy exposure. The target here is dollars per claim.

Pure Premium: Expected Loss Per Exposure Unit

A pure-premium model predicts expected loss per unit of exposure, before expenses, profit provisions, or other premium components. Its unit might be dollars per policy-year. To combine frequency and severity predictions, define $Y_j$ as the payment for claim $j$. If, conditional on the risk characteristics, claim count $N$ has mean $\lambda e$, claim amounts have mean $\mu_Y$, and the claim amounts are independent of the count and identically distributed, then the expected aggregate loss over exposure $e$ is:

E\left[\sum_{j=1}^{N}Y_j\right] =E[N]\mu_Y =\lambda e\mu_Y.

Dividing by exposure gives the pure premium per exposure unit, $\lambda\mu_Y$. This result does not require that frequency be Poisson; it requires the stated expected count and the independence and common-mean assumptions for severity. If severity depends on claim count, or the claim amounts do not share a common conditional mean, the product of a single frequency rate and a single severity mean may not represent expected loss.

The justification follows by conditioning on the claim count. For a fixed value $N=n$, the expected sum of $n$ claim amounts is $n\mu_Y$. Taking the expectation over $N$ yields:

E\left[\sum_{j=1}^{N}Y_j\right] =E\left[E\left[\sum_{j=1}^{N}Y_j\mid N\right]\right] =E[N\mu_Y] =\mu_Y E[N] =\lambda e\mu_Y.

This derivation explains why a frequency prediction and a severity prediction can be combined, but only when their definitions and assumptions align. For instance, a severity mean net of a deductible should not be combined with a frequency rate whose claim definition excludes claims below that deductible unless the model’s target is defined consistently.

Worked Example: Building a Pure-Premium Prediction

A hypothetical property portfolio has a frequency rate of $\lambda=0.08$ claims per policy-year. Claim payment per covered claim follows a Gamma distribution with shape $\alpha=3$ and scale $\theta=\$500$, independent of the claim count. The payment is the full covered payment under the modeled contract, with no additional deductible or limit adjustment. Calculate pure premium per policy-year and expected aggregate loss for 25 policies, each with 0.4 policy-years of exposure. There is no discounting or interest adjustment; mortality is not relevant.

Method. First find expected severity per claim:

\mu_Y=\alpha\theta=3(\$500)=\$1{,}500.

The pure premium per policy-year is frequency times mean severity:

\lambda\mu_Y =0.08(\$1{,}500) =\$120 \text{ per policy-year}.

Total exposure for the 25 policies is $25(0.4)=10$ policy-years. Expected claim count is $0.08(10)=0.8$ claims, and expected aggregate loss is:

0.8(\$1{,}500)=\$1{,}200.

The direct pure-premium calculation gives the same result: $10(\$120)=\$1{,}200$. Equivalently, each policy-period has expected loss $0.4(\$120)=\$48$, and $25(\$48)=\$1{,}200$. These equal totals check the calculation. Common mistake avoided: multiplying the per-policy-year pure premium by 25 without adjusting for the 0.4-year exposure, which would incorrectly give $3,000.

Choosing the Observation That Matches the Target

The unit of analysis is not automatically the unit in the available file. A policy-period table may contain one row per policy and coverage period, with exposure and claim count on each row. A claim table may contain one row per claim, potentially several rows linked to the same policy-period. Neither row structure is inherently correct for every model; the right structure depends on the target.

Model target	Typical observation	Outcome and basis
Claim frequency	Risk-period or policy-period	Claim count for the defined period, interpreted with its exposure
Claim severity	Claim	Payment or loss amount per claim on a stated basis
Pure premium	Risk-period or policy-period	Loss per exposure unit, or expected aggregate loss for the period

Suppose one policy has three claims during a one-year period. In a frequency file, the policy-period can be one observation with count three and exposure one policy-year. In a severity file, those claims can be three observations, one per claim. In a pure-premium analysis, the target could be the policy-period’s aggregate loss, or loss per policy-year. These representations answer different questions; they are not interchangeable simply because they refer to the same insured risk.

Name the outcome.
State whether the target is a count, a claim amount, or aggregate loss per exposure unit, including any payment or coverage definition.

Fix the horizon.
Identify the exact period represented by the target and express exposure in compatible units.

Choose the observation.
Use a policy-period for a policy-period count or loss target, and a claim record for a per-claim severity target.

Put prediction and outcome on the same basis.
Compare a count with an expected count, a claim amount with a predicted claim amount, or a period loss with a period loss prediction.

Common Target-Definition Errors

Comparing an annual rate with a short-period count. Multiply the rate by the period’s exposure to obtain the expected count for that period.
Using claim rows as though they were policy-period rows. Several claims can arise on one policy, so claim count, claim severity, and policy-period loss have different observation units.
Ignoring exposure when aggregating predictions. A per-exposure pure premium must be multiplied by the exposure represented by each observation to obtain expected period loss.
Mixing payment definitions. A gross severity prediction and a net-of-deductible observed payment do not share the same target unless the prediction is put on the same basis.

A clearly stated target makes validation interpretable: it identifies what a prediction error means and prevents a unit conversion from being mistaken for model misfit. Defining the target, horizon, and unit is therefore a necessary part of evaluating frequency, severity, and pure-premium predictions. This framing is aligned with the CAS Exam 8 (Advanced Ratemaking) syllabus.

Check Your Understanding

For each problem, identify the appropriate prediction basis and calculate the requested quantity.

A hypothetical auto frequency model has a Poisson rate of 0.18 claims per policy-year. A policy-period has 0.25 policy-years of exposure. What is its expected count, and what is the probability of at least one claim? (Answer: Expected count $0.045$; probability $1-e^{-0.045}=0.0440$.)
Claim payments follow a Gamma distribution with shape 2.5 and scale $400. What is the expected payment for one claim? (Answer: $1,000 per claim.)
A model predicts 0.06 claims per policy-year and mean severity of $2,200 per claim. For 15 policy-periods with 0.5 policy-years each, find total exposure and expected aggregate loss. (Answer: 7.5 policy-years and $990.)
A policy-period has one row with 0.8 policy-years of exposure and six linked claim records. Which unit is appropriate for a claim-frequency target, and which for a per-claim severity target? (Answer: One policy-period for frequency, with count six and exposure 0.8; six claim-level observations for severity.)
A hypothetical model has annual frequency rate 0.24 claims per policy-year and mean severity $900 per claim. For 40 six-month policy-periods, calculate expected claims, expected aggregate loss, and pure premium per policy-year. Assume frequency and severity meet the independence and common-mean conditions. (Answer: Total exposure 20 policy-years, expected claims 4.8, expected aggregate loss $4,320, and pure premium $216 per policy-year.)

What You'll Learn