Diagnosing Unusual Zero-Claim Frequencies

Why Zero-Claim Policies Are Informative

A frequency model predicts more than an average number of claims. It also predicts how many policies should have no claims at all. That prediction can be compared with the observed zero-claim proportion, which is often an accessible first check on whether a model describes the portfolio’s frequency pattern.

A difference between observed and predicted zeros is not, by itself, a diagnosis. It may reflect ordinary sampling variation, an incorrect mean frequency, heterogeneity among risks, dependence, or data and exposure issues. The useful question is first whether the difference is large relative to the model’s expected sampling variation, and then what explanations are plausible.

Definition: For a policy or other defined exposure unit, let $N$ be its claim count and define $q_0=P(N=0)$, the model’s probability of zero claims. For $n$ independent, identically distributed exposure units, let $Z$ be the number with zero claims. The observed zero proportion is $Z/n$, and the model’s expected zero count is $nq_0$.

The comparison assumes that the units have the same exposure basis and that their zero-claim indicators are independent with a common probability $q_0$. If exposures differ, the zero probabilities may differ too; applying one common probability to all policies would then be inappropriate. The earlier material on frequency exposure scaling treats exposure adjustments; here the comparison is for units on a common basis.

Zero Probabilities Under Three Frequency Models

The model’s zero probability depends on its count distribution. For a binomial count, $N\sim\operatorname{Binomial}(m,p)$, where there are $m$ independent trials and each has claim probability $p$, zero claims means every trial has no claim. For a Poisson count with mean $\lambda$, the probability is the count distribution’s probability at zero. For a negative binomial count, specify its parameters before calculating: in this tutorial, the mean is $\mu$ and the shape parameter is $r>0$, so $\operatorname{Var}(N)=\mu+\mu^2/r$.

Zero-probability formulas: Under the stated parameterizations, $$ \begin{aligned} N\sim\operatorname{Binomial}(m,p):\quad &q_0=(1-p)^m,\\ N\sim\operatorname{Poisson}(\lambda):\quad &q_0=e^{-\lambda},\\ N\sim\operatorname{NegativeBinomial}(\mu,r):\quad &q_0=\left(\frac{r}{r+\mu}\right)^r. \end{aligned} $$ For the negative binomial formula, $r$ is the shape parameter and $\mu$ is the mean; the formula is not interchangeable with one using a different parameterization unless the parameters are converted first.

The binomial and Poisson formulas follow directly from their probability mass functions. For the negative binomial convention used here, the probability of $k$ claims is $ \binom{k+r-1}{k}(r/(r+\mu))^r(\mu/(r+\mu))^k $ when $r$ is a positive integer. Setting $k=0$ gives the displayed zero probability; the same zero-probability expression applies for positive real $r$ under the corresponding gamma-function form of the negative binomial distribution.

Sampling Variation in the Observed Zero Count

Even when the frequency model is correct, the observed number of zero-claim policies will not usually equal $nq_0$. To quantify the ordinary variation, define $I_j=1$ if unit $j$ has zero claims and $I_j=0$ otherwise. Then $Z=\sum_{j=1}^n I_j$. Each $I_j$ is Bernoulli with probability $q_0$, and under the independence and common-probability assumptions the sum is binomial.

Result: If the $n$ units are independent and each has zero-claim probability $q_0$, then $Z\sim\operatorname{Binomial}(n,q_0)$, so $$ E[Z]=nq_0,\qquad \operatorname{Var}(Z)=nq_0(1-q_0). $$ When the expected counts of both zeros and nonzeros are sufficiently large for a normal approximation to be reasonable, a descriptive standardized difference is $$ z_0=\frac{Z-nq_0}{\sqrt{nq_0(1-q_0)}}. $$

This result follows because each indicator has $E[I_j]=q_0$ and $\operatorname{Var}(I_j)=q_0(1-q_0)$. Independence makes the variance of their sum the sum of their variances, while linearity of expectation gives the sum’s mean. Thus, the denominator in $z_0$ is the standard deviation of the zero count under the model, not the standard deviation of an individual claim count.

The value $z_0$ is a useful scale for describing a discrepancy: a positive value means more zeros than predicted, and a negative value means fewer. It is not automatically a formal goodness-of-fit test. In particular, if the model parameters were estimated from the same data, treating them as fixed can understate uncertainty; a full inferential analysis would need to account for that estimation.

Worked Example: Checking a Poisson Prediction

In a hypothetical portfolio of 100 policies on the same exposure basis, 40 policies have no claims. A Poisson model with mean $\lambda=1$ claim per policy is proposed. Calculate the predicted zero count and the standardized difference. State whether the observed count is above or below the prediction.

Method. Calculate the Poisson zero probability, then use the binomial variation of the zero indicators. This avoids using the variance of the claim count in place of the variance of the number of zero-claim policies.

For one policy, $q_0=e^{-1}=0.3678794$. Therefore, the expected number of zero-claim policies is $$ E[Z]=100(0.3678794)=36.78794. $$

The variance and standard deviation of the zero count are $$ \operatorname{Var}(Z)=100(0.3678794)(1-0.3678794) =100(0.2325442)=23.25442, $$ $$ \operatorname{SD}(Z)=\sqrt{23.25442}=4.822284. $$

The observed count exceeds its expected value by $40-36.78794=3.212056$. Hence $$ z_0=\frac{40-100e^{-1}}{\sqrt{100e^{-1}(1-e^{-1})}} =\frac{3.212056}{4.822284}\approx 0.666. $$

The observed count is above the prediction, by about $0.666$ standard deviations. This is a small standardized difference; it does not, on its own, provide strong evidence against the model. The probability is displayed to seven significant digits during the calculation, and the final standardized difference is rounded to three decimals.

Comparing Models Through Their Zero Predictions

Zero frequencies can help distinguish models that have the same mean but different distributional shapes. For a fixed mean, a negative binomial model with finite $r$ has greater variance than a Poisson model. Its probability of zero can also be greater, reflecting more probability in the lower tail alongside a heavier upper tail. The exact comparison should still be calculated for the parameters in question rather than inferred from the mean alone.

For example, if individual risks have Poisson means that vary across policies, a single Poisson model fitted at the average mean may predict too few zeros. If the risk-specific mean is a random variable $\Lambda$, the mixed model’s zero probability is $E[e^{-\Lambda}]$. Because $e^{-x}$ is convex, Jensen’s inequality gives $$ E[e^{-\Lambda}]\geq e^{-E[\Lambda]}. $$ Thus, unmodelled variation in risk means can produce excess zeros relative to a Poisson model with the same average claim frequency. This is one possible explanation, not a conclusion that can be drawn from zeros alone.

Worked Example: Comparing Three Models at the Same Mean

A hypothetical book has 200 policies with a common exposure basis. It records 108 zero-claim policies. Compare a binomial model with $m=4$ and $p=0.2$, a Poisson model with $\lambda=0.8$, and a negative binomial model with mean $\mu=0.8$ and shape $r=2$. For each model, calculate the expected zero count and the standardized difference.

Method. Each model has mean frequency $0.8$. Calculate each model’s own zero probability, then use $\operatorname{Var}(Z)=200q_0(1-q_0)$. The common mean does not imply a common zero probability.

For the binomial model, $$ q_0=(1-0.2)^4=0.8^4=0.4096,\qquad E[Z]=200(0.4096)=81.92. $$ Its zero-count variance is $$ 200(0.4096)(0.5904)=48.365568, $$ so its standard deviation is $\sqrt{48.365568}=6.9545$. Thus $$ z_0=\frac{108-81.92}{6.9545}\approx 3.750. $$

For the Poisson model, $$ q_0=e^{-0.8}=0.4493290,\qquad E[Z]=200(0.4493290)=89.8658. $$ Its variance is $$ 200(0.4493290)(1-0.4493290)=49.4865, $$ and its standard deviation is $\sqrt{49.4865}=7.0347$. Therefore $$ z_0=\frac{108-89.8658}{7.0347}\approx 2.578. $$

For the negative binomial model, $$ q_0=\left(\frac{2}{2+0.8}\right)^2 =\left(\frac{5}{7}\right)^2 =\frac{25}{49}=0.5102041, \qquad E[Z]=200\left(\frac{25}{49}\right)=102.0408. $$ Its variance is $$ 200\left(\frac{25}{49}\right)\left(\frac{24}{49}\right) =49.9792, $$ and its standard deviation is $\sqrt{49.9792}=7.0696$. Hence $$ z_0=\frac{108-102.0408}{7.0696}\approx 0.843. $$

The observed zero count is above the prediction under all three models, but the standardized difference is smallest under the negative binomial model. This comparison suggests that its zero prediction is closer for these data; it does not establish that the negative binomial is the correct model. The calculation avoids the common mistake of choosing by mean alone: all three models have mean $0.8$, yet their predicted zero counts differ substantially. Final standardized differences are rounded to three decimals.

What Excess or Deficient Zeros Can Signal

An excess of zeros means the observed zero proportion $Z/n$ is greater than the model’s $q_0$. Possible explanations include risk heterogeneity not represented in a single-rate model, a group of policies with little or no effective exposure, a mean frequency that is too high for some units, or a data issue affecting claim capture. A more dispersed count model may predict more zeros, but that possibility needs to be assessed using the rest of the count distribution as well.

A deficiency of zeros means the observed zero proportion is below the prediction. It can be consistent with a model mean that is too low, with less variation in underlying risk than the model assumes, or with a portfolio whose risks are more uniformly exposed to claims. It may also result from incorrect exposure definitions or recording. Neither direction identifies a unique cause.

The zero comparison is most useful as a diagnostic summary alongside the mean and the full frequency pattern. A model can match the observed zero proportion yet fit other claim counts poorly. Conversely, a modest zero-count discrepancy may be entirely compatible with random variation. If the data are grouped by risk or exposure, examine whether a single common $q_0$ is defensible before interpreting a portfolio-wide comparison.

Worked Example: Interpreting Deficient Zeros

A hypothetical portfolio contains 200 equal-exposure policies, of which 100 have zero claims. A Poisson model with mean $\lambda=0.5$ per policy is proposed. Calculate its expected zero count and standardized difference, then give a careful interpretation.

Method. Compute the zero probability and standard deviation of the zero count. The sign of the standardized difference identifies whether zeros are deficient or in excess; its size describes the discrepancy relative to sampling variation.

Under the model, $$ q_0=e^{-0.5}=0.6065307,\qquad E[Z]=200(0.6065307)=121.3061. $$ The zero-count variance is $$ 200(0.6065307)(1-0.6065307) =200(0.2386512)=47.7302, $$ so the standard deviation is $\sqrt{47.7302}=6.9087$. The standardized difference is $$ z_0=\frac{100-121.3061}{6.9087}\approx -3.084. $$

There are fewer zeros than the model predicts, by about $3.084$ standard deviations under the stated calculation. This is a notable discrepancy worth investigating, but it does not prove that the Poisson model is wrong or identify why the discrepancy occurred. For example, the assumed mean, exposure basis, or independence assumption may need review. The final standardized difference is rounded to three decimals.

Common Exam Trap: Using the Wrong Variance

The statistic concerns $Z$, the count of policies with zero claims. Its variance is $nq_0(1-q_0)$. It is not $n\lambda$, the variance of the total claim count under a Poisson model, and it is not $nq_0$, which omits the Bernoulli factor $1-q_0$. For the Poisson example above, the correct standard deviation is $\sqrt{47.7302}=6.9087$. Using $\sqrt{nq_0}=\sqrt{121.3061}$ instead would give about $11.014$, an incorrect scale for the zero-count difference.

Also distinguish the model’s parameters from the observed zero proportion. Substituting $Z/n$ for $q_0$ makes the expected count equal to the observed count by construction and destroys the intended comparison. Use the zero probability implied by the specified model and parameters.

Key takeaway: Calculate $q_0$ from the chosen frequency model, compare $Z$ with $nq_0$, and scale the difference by $\sqrt{nq_0(1-q_0)}$ when its assumptions and approximation are suitable. Treat excess or deficient zeros as evidence to investigate, not as a diagnosis by themselves.

Check Your Understanding

For each problem, use the stated frequency model and equal-exposure assumptions. Interpret comparisons as diagnostics, not conclusive tests.

A hypothetical set of 100 policies follows a binomial count model with $m=3$ and $p=0.1$. If 72 policies have zero claims, calculate the model’s zero probability and expected zero count. (Answer: $q_0=0.9^3=0.729$; expected zero count $=72.9$, so the observed count is $0.9$ below expectation.)
For 250 equal-exposure policies, a Poisson model has $\lambda=0.4$, and 170 policies have zero claims. Find the expected zero count and the observed-minus-expected difference. (Answer: $q_0=e^{-0.4}=0.6703$; expected zero count $\approx167.6$; difference $\approx+2.4$ policies.)
A negative binomial model has mean $\mu=1.2$ and shape $r=3$ for 140 policies. Calculate its zero probability and expected zero count. If 60 policies have zero claims, say whether zeros are in excess or deficient. (Answer: $q_0=(3/4.2)^3=(5/7)^3=125/343\approx0.3644$; expected count $\approx51.02$; the observed zeros are in excess by about $8.98$.)
A Poisson model with $\lambda=0.7$ is applied to 160 equal-exposure policies, and 80 have zero claims. Calculate $z_0$ to three decimals. (Answer: $q_0=e^{-0.7}\approx0.4966$; expected count $\approx79.454$; standard deviation $\sqrt{160q_0(1-q_0)}\approx6.324$; $z_0\approx0.086$.)
For 90 equal-exposure policies, compare binomial $m=5,p=0.1$, Poisson $\lambda=0.5$, and negative binomial $\mu=0.5,r=1$. If 58 policies have zero claims, calculate the three expected zero counts and identify which is closest. (Answer: binomial $q_0=0.9^5=0.59049$, expected $53.144$; Poisson $q_0=e^{-0.5}\approx0.60653$, expected $54.588$; negative binomial $q_0=(1/1.5)^1=2/3$, expected $60$. The negative binomial prediction is closest.)

What You'll Learn