Why Test a Fitted Normal Severity Model?
A normal model is specified by a mean and a variance, but fitting those parameters does not establish that the model describes the observed severities adequately. A symmetric, bell-shaped model may be convenient for calculation, yet observed losses can show skewness, unusually heavy tails, or a few extreme observations. A goodness-of-fit assessment therefore combines graphical evidence with a numerical procedure whose reference distribution matches how the model was fitted.
For a severity \(X\) modeled as normal, write \(X\sim N(\mu,\sigma^2)\), where \(\sigma^2>0\) is the variance. When both parameters are estimated from a sample \(x_1,\ldots,x_n\), the maximum likelihood estimates are the sample mean and the variance with divisor \(n\). They are not the sample variance with divisor \(n-1\). That distinction matters because the fitted distribution used to calculate expected probabilities must reflect the parameter-estimation rule actually used.
The uncensored, untruncated assumption is important: if observations are truncated or censored, the likelihood and fitting procedure must reflect that observation mechanism. Likewise, loss data that are known to be strictly positive deserve a support check. A normal distribution assigns some probability to negative values, even if that probability is small. A small negative probability does not by itself prove that a normal model is unsuitable, but it should be considered alongside the data and intended use.
Graphical Checks Before Numerical Tests
A histogram with a fitted normal density overlaid can reveal broad asymmetry, multiple modes, or a tail that appears heavier than the model predicts. The fitted empirical distribution function and normal cumulative distribution function can be plotted together to show where cumulative probabilities diverge. A normal Q-Q plot compares ordered observations with the corresponding fitted normal quantiles. Points near a straight line support the model’s shape; systematic curvature can indicate skewness, and departures concentrated at either end can indicate tail mismatch.
These plots are diagnostics, not formal proof of fit. Their appearance depends on sample size, bin choices, and scale. A numerical statistic helps summarize departures, but its interpretation also depends on calibration. When the mean and variance are estimated from the same observations being tested, treating them as if they were fixed in advance generally gives the wrong reference distribution.
Worked Example: Fitted Normal Model and a KS Diagnostic
A hypothetical portfolio has eight uncensored severity observations, in thousands of dollars: \(-1.5,-1.0,-0.5,-0.2,0.2,0.5,1.0,1.5\). For this diagnostic exercise the measurements are on a scale that may take either sign. Fit a normal model by maximum likelihood, calculate the Kolmogorov-Smirnov statistic, and interpret it using a supplied fitted-model calibration whose 5% critical value is \(0.285\).
Fit the model. The observations sum to zero, so \(\hat{\mu}=0\). Their squared values sum to \(2(1.5^2+1.0^2+0.5^2+0.2^2)=2(2.25+1+0.25+0.04)=7.08\). Thus \(\hat{\sigma}^2=7.08/8=0.885\) and \(\hat{\sigma}=\sqrt{0.885}\approx0.9407\). The fitted model is \(N(0,0.885)\).
Calculate the empirical and fitted CDF differences. For ordered observations \(x_{(i)}\), let \(F_{\hat{\mu},\hat{\sigma}}(x_{(i)})\) be the fitted normal CDF. The two one-sided discrepancies are \(D^+=\max_i\{i/n-F_{\hat{\mu},\hat{\sigma}}(x_{(i)})\}\) and \(D^-=\max_i\{F_{\hat{\mu},\hat{\sigma}}(x_{(i)})-(i-1)/n\}\). The largest discrepancy across either side is \(D=\max(D^+,D^-)\).
For example, at \(x_{(2)}=-1.0\), the standardized value is \(-1/0.9407\approx-1.063\), and the fitted CDF is approximately \(0.1439\). This gives \(i/n-F(x_{(i)})=2/8-0.1439=0.1061\). At the corresponding observation \(x_{(7)}=1.0\), the other one-sided difference is \(0.8561-6/8=0.1061\). Evaluating the other observations gives no larger difference, so \(D^+\approx0.1061\), \(D^-\approx0.1061\), and \(D\approx0.1061\).
Make the calibrated decision. Since \(0.1061<0.285\), do not reject the fitted normal model at the 5% level using the supplied calibration. This conclusion means the test has not found sufficient evidence against the model; it does not establish that the model is true. The critical value here is specific to the stated fitted-model calibration, not a universal value for every sample size or fitting procedure.
Method and mistake avoided. The method is a fitted-normal KS comparison. The common mistake is to use the ordinary fixed-distribution KS critical value after estimating \(\mu\) and \(\sigma\) from these same eight observations. Instead, the reference distribution must account for refitting both parameters.
Numerical Procedures and Their Calibration
The KS statistic measures the greatest vertical distance between the empirical CDF and the fitted CDF. Its sensitivity is spread across the distribution, so it can summarize central and tail departures without first dividing observations into categories. The Anderson-Darling statistic gives more weight to discrepancies in the tails. For ordered data and a continuous fitted CDF, its usual form is
Both statistics depend on the fitted parameters through the fitted CDF. Their reference distributions must therefore use the same parameter-estimation rule as the observed statistic. One standard approach is a parametric bootstrap: simulate samples from the fitted normal model, refit the mean and variance in each sample, and recalculate the statistic each time. The simulated statistics approximate the null reference distribution. A valid comparison requires refitting in every replicate; keeping the original estimates fixed would simulate a different procedure.
The Pearson chi-square procedure instead compares observed counts in \(k\) categories with expected counts. If category \(j\) has fitted probability \(p_j\), then \(E_j=np_j\), and the statistic is
Here \(O_j\) is the observed count. With \(r\) parameters estimated from the same data, the usual large-sample reference degrees of freedom are \(k-1-r\), provided the categories and fitting procedure meet the conditions for that approximation. For a normal model with both mean and variance estimated, \(r=2\), giving \(k-3\) degrees of freedom. Expected counts must be adequate for the chi-square approximation; sparse adjacent categories may need to be combined before interpreting the test.
Worked Example: Chi-Square Test Using Fitted-Normal Categories
A hypothetical sample contains \(n=100\) uncensored severities in thousands of dollars. The maximum likelihood estimates are \(\hat{\mu}=50\) and \(\hat{\sigma}=10\). Five categories are formed at the fitted normal quintiles, and the observed counts are \(14,18,24,22,22\). Calculate the Pearson statistic and make a 5% decision.
Find the boundaries and expected counts. The standard normal quintile cutoffs are approximately \(-0.8416,-0.2533,0.2533,0.8416\). Transforming by \(x=50+10z\) gives boundaries \(41.584,47.467,52.533,58.416\), rounded here to three decimals. By construction, each category has fitted probability \(0.20\), so each expected count is \(100(0.20)=20\).
Calculate the statistic. The observed-minus-expected differences are \(-6,-2,4,2,2\). Therefore \(\chi^2=((-6)^2+(-2)^2+4^2+2^2+2^2)/20=(36+4+16+4+4)/20=64/20=3.20\).
Set the degrees of freedom and decide. There are \(k=5\) categories and two parameters estimated from the same sample, so the approximate degrees of freedom are \(5-1-2=2\). The 5% critical value for a chi-square distribution with 2 degrees of freedom is approximately \(5.991\). Since \(3.20<5.991\), do not reject the fitted normal model at the 5% level. The corresponding upper-tail probability is approximately \(0.2019\).
Method and mistake avoided. This is a Pearson chi-square test with fitted category probabilities and a parameter-adjusted degrees-of-freedom calculation. Using \(4\) degrees of freedom would incorrectly ignore the estimation of the mean and variance. The conclusion is approximate and assumes expected counts are suitable; here every expected count is 20.
Tail-Sensitive Assessment and a Bootstrap Decision
A tail-sensitive statistic can be useful when unusually small or large severities are especially important to the modeling purpose. The Anderson-Darling calculation uses fitted probabilities at every ordered observation, with logarithmic terms that make tail discrepancies influential. But its sensitivity does not remove the need for calibration: after fitting a normal model, a fixed-parameter reference distribution is not automatically appropriate.
Worked Example: Anderson-Darling Statistic for a Fitted Normal Model
A hypothetical diagnostic sample, in arbitrary measurement units, is \(-1.5,-0.8,-0.3,0.3,0.8,1.5\). Both normal parameters are estimated by maximum likelihood. A parametric-bootstrap reference procedure that refits both parameters in each replicate supplies a 5% critical value of \(0.80\) for this sample size and fitting rule. Calculate the Anderson-Darling statistic and decide.
Estimate the parameters. The sample mean is zero. The squared observations sum to \(2(1.5^2+0.8^2+0.3^2)=2(2.25+0.64+0.09)=5.96\). Thus \(\hat{\sigma}^2=5.96/6=0.9933\) and \(\hat{\sigma}\approx0.9967\).
Obtain fitted probabilities. The fitted normal CDF values at the three positive observations \(0.3,0.8,1.5\) are approximately \(0.6183,0.7889,0.9339\). By symmetry, the CDF values at \(-1.5,-0.8,-0.3\) are approximately \(0.0661,0.2111,0.3817\). Substituting these six values, in increasing observation order, into the Anderson-Darling formula gives
Compare with the supplied calibration. Since \(0.15<0.80\), do not reject the fitted normal model at the 5% level under this bootstrap calibration. The critical value is explicitly tied to the stated sample size and procedure; it should not be reused as a universal Anderson-Darling cutoff.
Method and mistake avoided. This is a tail-sensitive Anderson-Darling assessment with a refitted parametric-bootstrap reference. A common mistake is to calculate the statistic correctly but compare it with an unadjusted critical value, overlooking that the normal parameters were estimated from the observations.
Putting the Evidence Together
A useful assessment reports more than a reject-or-not-reject decision. State how the parameters were fitted, which statistic was used, and how its reference distribution was calibrated. Pair the numerical result with a fitted CDF plot or normal Q-Q plot, and consider whether the observed departures occur in the center or tails. A non-rejection is not proof of normality; a rejection indicates evidence of departure under the test and calibration used.
For severity applications, also ask whether the normal model’s support is compatible with the loss variable. For example, if the fitted model assigns appreciable probability to negative losses, this is a substantive concern even if a goodness-of-fit test does not reject. Conversely, a very small estimated negative probability does not eliminate the need to inspect tail behavior or consider the model’s intended use. The final judgment should distinguish statistical fit from practical suitability.
Report the estimation rule and the fitted mean and variance. Check that the observation assumptions match the fitting method.
Use a fitted density, CDF comparison, or normal Q-Q plot to locate possible departures and assess their pattern.
Use KS for a maximum CDF discrepancy, Anderson-Darling for greater tail sensitivity, or chi-square for grouped counts. Account for fitted parameters in the reference procedure.
Report the statistic, calibration, and decision, then consider support and practical consequences rather than treating non-rejection as proof.
Check Your Understanding
For each problem, distinguish the calculation from the calibration needed to interpret it.
- A normal severity model is fitted by maximum likelihood to 80 uncensored observations. It has estimated mean \(80\) and standard deviation \(12\). What fitted probability lies below zero? Give the probability to four decimals. (Answer: \(\Phi(-80/12)=\Phi(-6.6667)\approx0.0000\) to four decimals.)
- A chi-square goodness-of-fit test uses six categories for a normal model whose mean and variance were estimated from the same observations. What are the usual approximate degrees of freedom, assuming adequate expected counts? (Answer: \(6-1-2=3\) degrees of freedom.)
- A sample of 150 observations is grouped into five fitted-normal quintile categories. The observed counts are \(25,33,28,34,30\). Calculate the Pearson statistic and approximate upper-tail probability using the parameter-adjusted degrees of freedom. (Answer: Expected counts are 30 each; \(\chi^2=(25+9+4+16+0)/30=1.80\), with 2 degrees of freedom and upper-tail probability \(e^{-1.80/2}\approx0.4066\).)
- A fitted-normal KS procedure gives \(D^+=0.071\) and \(D^-=0.094\). A calibration that refits both normal parameters gives a 5% critical value of \(0.120\). What is the test statistic and decision? (Answer: \(D=\max(0.071,0.094)=0.094<0.120\); do not reject at the 5% level.)