Construct a Frequency-Severity Case Dataset

Why Dataset Construction Determines the Target

A claim file often has one row per payment, adjustment, or status update, while a policy file has one row per policy period. Neither file is automatically a modeling dataset. Before fitting a model, the analyst must decide what one modeling row represents, which claims belong to that row, and what quantity the model is meant to predict.

For frequency and severity work, a common structure is a policy-period table for claim counts and a claim-level table for claim amounts. These tables answer different questions. The policy-period table must preserve periods with no claims; the claim-level table has one row for each claim included under a stated severity definition. A payment transaction is not necessarily a claim, and a policy period is not necessarily a claim.

Definition: A policy-period observation is one policy’s coverage over a specified observation window. Let $N_i$ be the number of distinct claims attributed to policy period $i$, and let $e_i$ be its earned exposure measured in a stated unit, such as policy-years. The observed frequency for that period is $N_i/e_i$, provided $e_i>0$. A claim-level severity observation is one claim’s amount under a consistently defined amount basis and valuation date.

The frequency value $N_i/e_i$ is a constructed observed rate, not a guarantee of a future rate. The claims and exposure must refer to the same coverage, time window, and population. If exposure is recorded in policy-years, a period covering six months contributes $0.5$ policy-years, not $6$ or $1$. For a policy-period with no claims, $N_i=0$ and its observed frequency is zero when $e_i>0$. It remains a valid frequency observation.

For severity, specify what the amount measures. It might be paid loss through a valuation date, incurred loss on a specified basis, or another defined amount. The choice must be applied consistently. If a claim has several transactions, aggregate them to the selected claim amount before treating the claim as one severity observation. If the intended amount is still developing, the dataset should identify the valuation convention rather than silently mixing incomplete and mature amounts.

A Construction Sequence

The following sequence keeps the observation unit, claim linkage, and target definition explicit. It is especially useful when the source files contain repeated policy records or multiple rows per claim.

Set the unit and observation window.
For a frequency target, define one row as one policy over one stated coverage period. Specify the period’s start and end and the exposure unit.

Align exposure and claims.
Assign a claim to a policy period using the date appropriate to the question, commonly the date of loss for a loss-occurrence view. Do not substitute a report or payment date without a reasoned definition.

Count distinct claims.
Use the claim identifier to distinguish claims from transactions. A claim with several payments contributes one to the claim count.

Preserve zero-claim periods.
Start with the complete eligible policy-period list, then attach claim counts. A missing match to the claim file means zero only after confirming that the claim file is complete for the window.

Construct severity rows.
Aggregate transaction records to the chosen claim-level amount, then create one severity row per eligible claim. Do not create a zero-dollar severity row for a period with no claims.

The join direction in Step 4 matters. If the analyst starts with the claim file and summarizes it, policy periods with no matching claim disappear. That can be acceptable for a claim-level severity table, but it is not a complete policy-period frequency table. A reliable process builds the eligible policy-period spine first and attaches counts to it.

Worked Example: Building Both Tables From Policy Periods and Claims

A hypothetical insurer is preparing a frequency dataset for three policy periods. Their earned exposures are $0.50$, $1.00$, and $0.75$ policy-years. In the observation window, policy A has two distinct claims of \$600 and \$900. Policy B has no claims. Policy C has two claims: claim C1 has two payment transactions of \$400 and \$250, while claim C2 has one transaction of \$800. Assume all listed records are complete, belong to the stated periods by loss date, and define claim amount as the sum of listed payments.

Method. Keep one row per policy period for frequency, count distinct claim IDs, and aggregate transactions within a claim for severity.

Policy A has $N_A=2$, so its observed frequency is $2/0.50=4$ claims per policy-year. Its claim-level amounts are \$600 and \$900.

Policy B has $N_B=0$, so its observed frequency is $0/1.00=0$ claims per policy-year. It contributes a frequency row, but no severity row.

For policy C, claim C1’s amount is $\$400+\$250=\$650$, not two claims. Together with C2, policy C has $N_C=2$. Its observed frequency is $2/0.75=2.6667$ claims per policy-year, to four decimal places. Its severity rows are \$650 and \$800.

The policy-period table therefore contains three rows: A $(e=0.50,N=2)$, B $(e=1.00,N=0)$, and C $(e=0.75,N=2)$. Total claims are $2+0+2=4$, and total exposure is $0.50+1.00+0.75=2.25$ policy-years. The combined observed frequency is $4/2.25=1.7778$ claims per policy-year, to four decimal places.

The claim-level table has four rows with amounts \$600, \$900, \$650, and \$800. Their total is $\$600+\$900+\$650+\$800=\$2,950$; the claim-level mean is $\$2,950/4=\$737.50$. The check is that the four severity rows correspond to the four distinct claims, while the frequency table still has all three policy periods.

Common mistake avoided: counting the two C1 payment transactions as two claims would incorrectly produce five claims. The claim identifier shows that they are components of one claim amount.

Aggregation and Weighting Must Match the Question

A claim-level average gives each claim one contribution. An average of policy-period averages gives each policy period one contribution, whether it contains one claim or many. Those calculations are not interchangeable. The difference is not a computational defect; they summarize the data with different weighting.

\bar{X}_{\text{claim}} = \frac{\sum_{i=1}^{m}\sum_{j=1}^{N_i} X_{ij}} {\sum_{i=1}^{m}N_i}, \qquad \bar{X}_{\text{period}} = \frac{1}{m_+}\sum_{i:N_i>0} \left(\frac{1}{N_i}\sum_{j=1}^{N_i}X_{ij}\right).

Here $X_{ij}$ is the defined amount for claim $j$ in policy period $i$, and $m_+$ is the number of periods with at least one claim. The first expression includes each claim once. The second first calculates a mean within each claim-bearing period, then gives each such period equal weight. Periods with no claims have no within-period severity mean; they are not assigned a severity mean of zero.

The claim-level expression follows directly by listing all eligible claims: each $X_{ij}$ appears once in the numerator and the number of listed claim amounts is $\sum_i N_i$. The period-average expression instead has $m_+$ terms, one per claim-bearing period. Unless claim counts are equal across periods or period means happen to align, the two averages need not agree.

Worked Example: Pooled Severity Versus an Average of Period Means

A hypothetical portfolio has two policy periods. The first period has one claim with severity \$700. The second has two claims with severities \$1,000 and \$1,400. Calculate the pooled claim-level mean and the unweighted mean of the two period means. Also calculate the combined observed claim frequency if the periods have exposures of $0.50$ and $1.00$ policy-years.

Method. For pooled severity, add all claim amounts and divide by the number of claims. For the unweighted period calculation, first find each period’s mean and then average the two means. For combined frequency, divide the total count by total exposure.

The first period mean is $\$700/1=\$700$. The second period mean is $(\$1,000+\$1,400)/2=\$2,400/2=\$1,200$. The unweighted mean of period means is $(\$700+\$1,200)/2=\$1,900/2=\$950.00$.

The pooled claim-level mean uses all three claims: $(\$700+\$1,000+\$1,400)/3=\$3,100/3=\$1,033.333\ldots$, or \$1,033.33 to the nearest cent. The two results differ because the second period contributes two claims to the pooled mean but only one period mean to the unweighted calculation.

The counts are 1 and 2, for a total of 3 claims. Total exposure is $0.50+1.00=1.50$ policy-years, so combined observed frequency is $3/1.50=2.0000$ claims per policy-year.

Check. The claim amounts total \$3,100 in both calculations’ source data; the pooled average times three claims is \$3,100. The period means are \$700 and \$1,200, whose unweighted average is \$950.00. Common mistake avoided: averaging period means and calling the result the pooled claim mean. That would give \$950.00 instead of the claim-weighted \$1,033.33.

Choose the Target From the Decision

The intended decision determines what the response should represent. If a task requires a prediction of the number of claims for each policy period, the response must be a policy-period count, including zero counts. If the task concerns the amount of an individual claim, the response belongs at claim level and requires one consistently constructed amount per claim. A table can support more than one analysis, but its rows and responses must not be confused.

For example, a policy-period total loss can be constructed by summing the selected claim amounts assigned to that period. For a period with no claims, that total is zero. This is a distinct response from claim severity: it combines claim occurrence and amounts at the policy-period level. The distinction is important even when both values originate from the same claim file.

Before creating targets, write down a compact specification: observation unit, time window, claim-attribution date, claim identifier, amount basis, valuation date, and exposure unit. These choices should be made using information available for the intended prediction and evaluation. A claim reported after the prediction date, for instance, cannot be included in a predictor merely because it appears in a later extract. Keeping the target definition separate from predictors also reduces accidental leakage.

Worked Example: Aggregating Transactions for a Claim-Level Decision

A hypothetical analyst needs a claim-level dataset to estimate the average payment amount per claim for a defined historical window. There are three policy periods: R1 with $0.40$ policy-years of exposure, R2 with $0.80$, and R3 with $0.50$. The claim transaction file shows R1 claim Q1 with payments of \$300 and \$200, R1 claim Q2 with a payment of \$900, no claims for R2, and R3 claim Q3 with payments of \$250 and \$350. Assume each transaction is in the window and the target is total payment per distinct claim as of the stated valuation date.

Method. Group by policy-period ID and claim ID, sum transaction amounts within each claim, and then create the severity response from those claim totals.

For Q1, the claim amount is $\$300+\$200=\$500$. Q2 has one payment, so its amount is \$900. Q3 has amount $\$250+\$350=\$600$. The severity table therefore has three observations: \$500, \$900, and \$600. The total is $\$500+\$900+\$600=\$2,000$, and the mean payment per claim is $\$2,000/3=\$666.666\ldots$, or \$666.67 to the nearest cent.

For the companion policy-period count table, R1 has two distinct claims, R2 has zero, and R3 has one. Their observed frequencies are $2/0.40=5$, $0/0.80=0$, and $1/0.50=2$ claims per policy-year. The combined count is 3 and exposure is $0.40+0.80+0.50=1.70$ policy-years, giving $3/1.70=1.7647$ claims per policy-year to four decimal places.

Check. The claim totals sum to \$2,000, which is also the total of all transaction payments: $\$300+\$200+\$900+\$250+\$350=\$2,000$. There are three distinct claim IDs, not five claims. R2 remains in the frequency table with a zero count but does not create a fourth, zero-dollar severity record.

Common Construction Traps

Dropping zero-claim periods. A claim-only summary cannot represent policy periods with no matching claim. Start from eligible policy-periods when constructing frequency observations.
Counting transactions as claims. Count distinct claim identifiers; aggregate transactions within each claim under the specified amount definition.
Mixing dates and windows. A claim’s report date, loss date, and payment date can differ. State which date assigns it to the observation window and apply that rule consistently.
Using the wrong exposure unit. Match the denominator to the stated unit. If exposure is in policy-years, $0.25$ years is one quarter of a policy-year.
Replacing undefined severity with zero. A zero-claim period has no claim severity observation. Zero may be valid for a policy-period total, but it is not a claim-level severity.
Comparing differently weighted averages without naming them. A pooled claim mean weights each claim equally; an average of period means weights each included period equally.

Key takeaway: A sound frequency-severity dataset begins with a clear observation unit and target definition. Preserve every eligible policy period for frequency, count distinct claims, aggregate transactions to the chosen claim amount for severity, and verify that the resulting rows match the decision being studied. These data-construction choices are aligned with the SOA Exam PA (Predictive Analytics) syllabus.

Check Your Understanding

For each problem, identify the appropriate construction and check the arithmetic before comparing with the answer.

A policy period has exposure $0.25$ policy-years and no claims in a complete claim file. What is its observed frequency, and should it remain in a frequency dataset? (Answer: 0 claims per policy-year; yes, retain the period.)
A claim has three payment transactions of \$120, \$80, and \$300. Under a target defined as total payment per distinct claim, how many claim-level rows and what amount should be recorded? (Answer: 1 row with severity \$500.)
Period A has two severities, \$400 and \$800; period B has one severity, \$1,500. Find the pooled mean and the unweighted mean of period means. (Answer: pooled mean $(400+800+1,500)/3=\$900.00$; period means are \$600 and \$1,500, so their unweighted mean is \$1,050.00.)
Three policy periods have exposures $0.50$, $0.75$, and $1.00$ policy-years, with counts 1, 0, and 3. Find the combined observed frequency. (Answer: $4/(0.50+0.75+1.00)=4/2.25=1.7778$ claims per policy-year.)
A data extract contains four transaction rows for two claim IDs: claim A has two payments totaling \$700, and claim B has two payments totaling \$1,300. A policy-period claim-count task also includes a second period with exposure $0.50$ and no claims; the first period has exposure $1.00$. What claim count belongs to each period, and what is the combined frequency? (Answer: counts are 2 and 0; combined frequency is $2/(1.00+0.50)=1.3333$ claims per policy-year.)

What You'll Learn