Why Discrete Severity Models Need a Goodness-of-Fit Test
A discrete severity model assigns probabilities to particular loss amounts. To assess whether observed losses are consistent with such a model, compare the observed number of losses in each category with the number the model predicts. The Pearson chi-square test makes this comparison while accounting for the size of each expected count.
The calculation depends on using the model’s probabilities, not just the number of possible loss amounts. If several possible amounts are combined into one category, add their probabilities to obtain that category’s probability. Expected counts then follow from the total sample size.
The null hypothesis is that the severity distribution is the specified model. The alternative is that the distribution differs from that model in at least one category probability. A small statistic indicates that the observed category counts are relatively close to the model’s expected counts; a large statistic indicates greater discrepancy.
The categories must cover every possible observation and must not overlap. For discrete severity, categories may be individual support values or groups of support values. If category \(j\) contains support points \(x_a,\ldots,x_b\), its probability is the sum of their model probabilities: \(q_j=P(X\in C_j)=\sum_{x\in C_j}P(X=x)\).
Building the Statistic From Model Probabilities
A reliable calculation follows a consistent sequence. First identify the support values and the null probabilities. Then assign each support value to exactly one category, sum probabilities within categories, and multiply each category probability by \(n\). Finally, compare observed and expected counts using the Pearson contribution.
Write down the null probability at each support value and make the categories exhaustive and mutually exclusive.
Add the model probabilities of all support values in each category. Check that the category probabilities sum to 1.
For each category, use \(E_j=nq_j\). Check that the expected counts sum to the sample size.
Sum \((O_j-E_j)^2/E_j\), use the appropriate degrees of freedom, and compare with a critical value or obtain a p-value.
The approximation to a chi-square distribution relies on the multinomial count model and sufficiently large expected counts. A common practical guideline is to aim for expected counts of at least 5 in every category; it is a guideline, not a universal guarantee. If categories are combined to address sparse expected counts, combine adjacent or otherwise substantively coherent categories and add their model probabilities as well as their observed counts. Do not combine categories based on which choice produces a preferred test result.
Worked Example: A Four-Point Severity Model
A hypothetical insurer models a claim’s severity as one of \(100,200,300,400\) dollars, with respective probabilities \(0.10,0.20,0.30,0.40\). In a sample of 60 independent claims, 12 are in the first two amounts and 48 are in the last two. Test the specified model at the 5% level. The 5% critical value for one degree of freedom is 3.841.
Method. Use the Pearson chi-square test with the two stated grouped categories. Under the null model, the probability of the first category is \(0.10+0.20=0.30\); that of the second is \(0.30+0.40=0.70\). They sum to 1.
The expected counts are \(E_1=60(0.30)=18\) and \(E_2=60(0.70)=42\). They sum to 60, as required. The observed counts \(12\) and \(48\) also sum to 60.
The category contributions are \((12-18)^2/18=36/18=2.0000\) and \((48-42)^2/42=36/42=0.8571\). Thus \(X^2=2.0000+0.8571=2.8571\), rounded to four decimal places.
There are \(k=2\) categories and no model parameters were estimated from these data, so the degrees of freedom are \(k-1=1\). Since \(2.8571<3.841\), do not reject the specified model at the 5% level. This does not prove the model is correct; it means this test does not provide sufficient evidence to reject it at that level. The check is consistent: the observed counts differ from their expected counts by six in opposite directions, and both contributions are positive.
Common mistake avoided: Treating the two individual support probabilities as two separate test categories would ignore the problem’s specified grouping. The test uses the probability \(0.30\) for the combined first category, not either \(0.10\) or \(0.20\) alone.
Why the Pearson Formula Has This Form
The denominator in each Pearson contribution is the expected count, not its square. One way to justify the formula is to begin with the variability of category counts under the null model. For a multinomial sample with fixed category probabilities \(q_1,\ldots,q_k\), the count \(O_j\) has mean \(nq_j=E_j\) and variance \(nq_j(1-q_j)\). The counts are also constrained by \(\sum_j O_j=n\), so only \(k-1\) count deviations can vary independently.
For a single category, a residual measured in standard-deviation units would be \((O_j-E_j)/\sqrt{E_j}\) if the variance were \(E_j\). Squaring that standardized residual gives \((O_j-E_j)^2/E_j\). The Pearson statistic sums these squared, expected-count-scaled discrepancies across categories. The chi-square approximation accounts for the dependence among counts caused by their fixed total.
More formally, under the null model and with sufficiently large expected counts, the multinomial count vector has an approximately normal distribution on the \(k-1\) dimensional space defined by \(\sum_j(O_j-E_j)=0\). The Pearson quadratic form on that space is approximately chi-square with \(k-1\) degrees of freedom. If \(r\) parameters have been estimated from the same data, the usual degrees-of-freedom adjustment is \(k-1-r\), subject to the conditions for that adjustment. The worked examples here use fully specified models.
Worked Example: Testing Four Individual Loss Amounts
A hypothetical equipment policy has severity support \(50,100,150,200\) dollars, with model probabilities \(0.20,0.30,0.30,0.20\), respectively. In 100 independent claims, the observed counts are \(16,34,27,23\). Test the specified distribution at the 5% level, using the critical value 7.815 for three degrees of freedom.
Method. Each possible amount is its own category. The probabilities sum to \(0.20+0.30+0.30+0.20=1\), so all support values are represented.
The expected counts are \(100(0.20)=20\), \(100(0.30)=30\), \(100(0.30)=30\), and \(100(0.20)=20\). Their total is 100, matching the sample size. The observed counts also sum to \(16+34+27+23=100\).
The contributions are \((16-20)^2/20=16/20=0.8000\), \((34-30)^2/30=16/30=0.5333\), \((27-30)^2/30=9/30=0.3000\), and \((23-20)^2/20=9/20=0.4500\). Therefore \(X^2=0.8000+0.5333+0.3000+0.4500=2.0833\), rounded to four decimal places.
With four categories and no estimated parameters, the degrees of freedom are \(4-1=3\). Since \(2.0833<7.815\), do not reject the model at the 5% level. As a check, each observed count is close to its corresponding expected count, and the statistic is therefore modest compared with the stated critical value.
Common mistake avoided: The Pearson contribution is not the absolute difference divided by the expected count. For the first category, the correct contribution is \(4^2/20=0.8\), not \(4/20\).
A Common Calculation Trap
A frequent error is to say that the difference \(O_j-E_j\) is “standardized by dividing by the expected count.” That is not the Pearson calculation. Its standardized residual is \((O_j-E_j)/\sqrt{E_j}\); squaring it gives the contribution \((O_j-E_j)^2/E_j\). Dividing the squared difference by \(E_j^2\) would be wrong as well.
For example, if \(O_j=12\) and \(E_j=18\), then the squared difference is \(36\), and the contribution is \(36/18=2\). The standardized residual is \(-6/\sqrt{18}\), whose square is also 2. Using \(36/18^2\) would instead give about \(0.1111\), understating the discrepancy. Always square the count difference and divide once by the expected count.
Worked Example: Pooling Support Values Before Testing
A hypothetical repair-cost model assigns probabilities \(0.05,0.10,0.20,0.25,0.25,0.15\) to costs of \(100,200,300,400,500,600\) dollars, respectively. For a sample of 100 claims, the observed counts for the three categories \(100\)–\(200\), \(300\)–\(400\), and \(500\)–\(600\) dollars are \(11,39,50\). Test the specified model at the 5% level; use 5.991 as the critical value for two degrees of freedom.
Method. Add probabilities within each given category before calculating expected counts. The category probabilities are \(0.05+0.10=0.15\), \(0.20+0.25=0.45\), and \(0.25+0.15=0.40\). They sum to 1.
The expected counts are \(100(0.15)=15\), \(100(0.45)=45\), and \(100(0.40)=40\). These sum to 100, as do the observed counts \(11+39+50=100\).
The contributions are \((11-15)^2/15=16/15=1.0667\), \((39-45)^2/45=36/45=0.8000\), and \((50-40)^2/40=100/40=2.5000\). Thus \(X^2=1.0667+0.8000+2.5000=4.3667\), rounded to four decimal places.
There are three categories and no estimated parameters, giving \(3-1=2\) degrees of freedom. Since \(4.3667<5.991\), do not reject the specified model at the 5% level. The third category contributes the most to the statistic, but the total remains below the critical value. As a final check, the expected counts sum to the sample size, and each category was counted once.
Common mistake avoided: Computing expected counts from only one support probability per category would misstate the model. For example, the first category’s expected count is \(100(0.05+0.10)=15\), not \(100(0.05)=5\).
Interpreting and Reporting the Test
A useful report identifies the model and categories, gives observed and expected counts, states the statistic and degrees of freedom, and makes a conclusion at the stated significance level. The conclusion should concern compatibility with the specified model, not whether that model is true. A failure to reject can reflect limited information in the sample, particularly when categories are few or the test has limited ability to detect the departure of interest.
- Check that the category probabilities add to 1 and that the expected counts add to \(n\).
- Use \(E_j=nq_j\), where \(q_j\) is the probability of the entire category under the null model.
- Use \((O_j-E_j)^2/E_j\) for each contribution.
- For fixed probabilities, use \(k-1\) degrees of freedom when the chi-square approximation is appropriate.
- Do not treat a non-significant result as proof that the model is correct.
Check Your Understanding
For each problem, form the category probabilities and expected counts before calculating or interpreting the test statistic.
- A hypothetical model assigns probabilities \(0.20,0.50,0.30\) to losses of \(10,20,30\) dollars. In 50 claims the counts are \(8,28,14\). Find the Pearson statistic and state whether the model is rejected at 5%, using the critical value 5.991 for two degrees of freedom. (Answer: \(X^2=0.8267\); do not reject at 5%.)
- A discrete model has four ordered outcomes \(A,B,C,D\), with probabilities \(0.10,0.20,0.30,0.40\). For 80 claims, the counts are \(5,18,25,32\). Calculate the statistic and degrees of freedom. (Answer: \(X^2=1.4167\), with 3 degrees of freedom.)
- A model assigns probabilities \(0.05,0.10,0.15,0.20,0.25,0.25\) to six increasing loss amounts. For 100 claims, group the first two, the next two, and the final two amounts; the observed category counts are \(12,38,50\). Calculate the statistic. (Answer: expected counts \(15,35,50\), and \(X^2=0.8571\).)
- A fixed model gives four categories probabilities \(0.10,0.20,0.30,0.40\). In 120 observations, the counts are \(18,18,42,42\). Calculate the statistic and decide at 5%, using a critical value of 7.815 for three degrees of freedom. (Answer: expected counts \(12,24,36,48\), \(X^2=6.2500\); do not reject at 5%.)