Why Different Risk Rates Change Count Variation
Suppose two policyholders have different underlying claim rates. If we know which policyholder we are observing, a Poisson model may describe that person’s count adequately. But if the policyholder is selected from a mix of risk types and the type is not observed, the count varies for two reasons: random claim arrivals within a risk type, and variation in the underlying rate between risk types. The second source adds variance.
This distinction matters when assessing whether observed counts are more variable than a single Poisson model predicts. A mixture of Poisson counts can have variance greater than its mean even though each risk type has a conditional Poisson count. The key is to condition on the risk rate first, then average over the rate distribution.
The conditional Poisson assumptions give \(E[N\mid\Lambda]=\Lambda\) and \(\operatorname{Var}(N\mid\Lambda)=\Lambda\). When rates are known and fixed, these are the usual Poisson mean and variance. When the rate varies across the population under study, the unconditional variance also reflects the variability of \(\Lambda\).
The Mean and Variance of a Mixed Count
The law of total expectation gives the unconditional mean. The law of total variance separates the count variance into its within-rate and between-rate components.
The first variance term, \(E[\Lambda]\), is the average Poisson variance within risk types. The second, \(\operatorname{Var}(\Lambda)\), is the added variance from mixing risks with different rates. Since a variance cannot be negative, the unconditional count variance is at least its mean. It equals the mean only when the underlying rate has no variation (apart from the degenerate case of zero rate).
Why the Extra Variance Appears
The variance formula follows directly from conditioning. Imagine grouping risks by their rate. Within a group with rate \(\lambda\), counts fluctuate around \(\lambda\), with variance \(\lambda\). Across groups, however, the conditional means themselves differ. The total variance must include both the average within-group fluctuation and the variation among those conditional means.
For a given rate \(\Lambda\), use the Poisson results \(E[N\mid\Lambda]=\Lambda\) and \(\operatorname{Var}(N\mid\Lambda)=\Lambda\).
The law of total expectation gives \(E[N]=E[\Lambda]\).
The law of total variance gives \(E[\Lambda]+\operatorname{Var}(\Lambda)\), not just the average conditional variance.
For a fixed exposure of \(t\) time units, suppose \(N\mid\Lambda\sim\operatorname{Poisson}(t\Lambda)\), where \(\Lambda\) is a rate per unit of time. The same argument gives
The squared exposure in the second term is important: exposure scales the conditional Poisson mean by \(t\), and it scales the variation in that mean by \(t^2\). Rates must be expressed per unit of the same time period used for \(t\).
Worked Example: A Two-Type Risk Mix
A hypothetical insurer selects one policy at random for a one-year study. A low-frequency risk has probability \(0.25\) and annual claim rate \(0.8\); a higher-frequency risk has probability \(0.75\) and annual claim rate \(1.6\). Conditional on type, the annual claim count is Poisson. Find the unconditional mean and variance, and compare the probability of no claims with that from a Poisson model having the same mean.
Method and assumptions. Let \(\Lambda\) equal \(0.8\) with probability \(0.25\) and \(1.6\) with probability \(0.75\). Use the conditional Poisson model and the total expectation and variance formulas. This avoids treating the mixed count as Poisson with a single fixed rate.
Mean rate and count mean.
Rate variance. First calculate the squared deviations from the mean rate:
Count variance.
The variance exceeds the mean by \(0.12\), exactly the variance of the risk rate. For the zero-count probability, average the conditional Poisson probabilities:
A Poisson count with the same mean \(1.40\) would instead give \(P(N=0)=e^{-1.4}=0.2466\). The mixture has the higher zero-count probability because it combines risks with rates both below and above the average. Probabilities are rounded to four decimal places. Check: \(1.52-1.40=0.12\), matching \(\operatorname{Var}(\Lambda)\). The common mistake avoided is using the mean rate as though every selected risk actually had that fixed rate.
Worked Example: Heterogeneity Over Multiple Years
A hypothetical policy is observed for two years. Its annual rate is \(0.5\), \(1.0\), or \(2.0\), with probabilities \(0.20\), \(0.50\), and \(0.30\), respectively. Conditional on its rate, the two-year claim count is Poisson with mean equal to rate times exposure. Find the unconditional count mean and variance, then compare the variance with that of a homogeneous Poisson model having the same mean.
Method and assumptions. The rate is fixed for the policy throughout both years, and the exposure is \(t=2\). Calculate the distribution of the two-year conditional mean \(2\Lambda\), then apply the total variance formula.
Mean rate.
Thus the expected two-year count is \(2(1.20)=2.40\). For the rate variance, first find the second moment:
The mean of the conditional two-year count is \(2\Lambda\), whose variance is \(2^2(0.31)=1.24\). The average conditional Poisson variance is \(2E[\Lambda]=2.40\). Therefore,
A homogeneous Poisson count with mean \(2.40\) has variance \(2.40\), which is \(1.24\) lower. The units are consistent: the exposure-scaled rate is a count mean, while the between-rate variance is multiplied by \(t^2=4\). Check: \(3.64-2.40=1.24=4(0.31)\). The common mistake avoided is multiplying the rate variance by exposure only once instead of by exposure squared.
Randomly Selected Risks Versus Fixed Portfolios
A crucial qualification is that heterogeneity across a fixed set of risks does not automatically make the total portfolio count overdispersed. Suppose each risk has a known, fixed rate, and the risks’ counts are independent. The sum of independent Poisson counts is Poisson with mean and variance equal to the sum of their rates. The formula for a random \(\Lambda\) applies when the risk rate itself is random for the unit being modeled, such as when one policy is sampled from a mixed population.
The distinction is whether rate composition is random in the quantity being analyzed. In a random sample of individual policies, the selected rate varies from policy to policy. In a portfolio that contains a fixed number of each risk type, the portfolio’s total rate is fixed, even though individual policies are heterogeneous.
Worked Example: Fixed Composition or Random Composition?
A hypothetical portfolio has 40 policies observed for one year. Half have a fixed annual rate of \(0.5\), and half have a fixed annual rate of \(1.5\). Conditional on the assigned rates, all policy counts are independent Poisson. Find the total count mean and variance for this fixed portfolio. Then suppose instead that each of 40 policies is independently assigned either rate with probability \(0.5\). Find the unconditional variance of the total count in this randomly composed portfolio.
Method and assumptions. For the fixed portfolio, add the means and variances of independent Poisson counts. For the randomly composed portfolio, each policy’s rate is itself random and the independent rate selections create additional portfolio-level variation.
Fixed composition. There are 20 policies at each rate, so the total mean is
The independent Poisson counts have variances equal to their respective means, so the total variance is also \(20(0.5)+20(1.5)=40\). Check: the total is a sum of independent Poisson variables with fixed rates, so its mean and variance are both \(40\).
Random composition. For one randomly assigned policy, \(E[\Lambda]=0.5(0.5)+0.5(1.5)=1.0\), and
The marginal count variance for one policy is \(1.0+0.25=1.25\). Since the 40 rate assignments and conditional counts are independent across policies, the total variance is \(40(1.25)=50\), while the total mean remains \(40(1.0)=40\). Check: the extra variance is \(50-40=10\), equal to the variance of the sum of 40 independent rates, \(40(0.25)=10\). The common mistake avoided is applying individual-level mixture variance to a portfolio whose class counts are fixed.
Common Mistakes and Practical Use
The variance decomposition is useful whenever counts are pooled across risks, territories, exposure groups, or other categories with differing underlying rates. It explains one way that observed count variation can exceed a single-rate Poisson model’s prediction, without requiring the conditional count model within each group to depart from Poisson.
- Dropping the between-risk term. Using only \(E[\operatorname{Var}(N\mid\Lambda)]=E[\Lambda]\) ignores that the conditional means vary. Add \(\operatorname{Var}(\Lambda)\).
- Mixing up the mean and variance of the rate. The extra count variance is \(\operatorname{Var}(\Lambda)\), not \(E[\Lambda^2]\). If using moments, calculate \(E[\Lambda^2]-(E[\Lambda])^2\).
- Applying the formula to the wrong unit. A random individual from a heterogeneous population and a fixed-composition portfolio are different models. Identify whether rates or group composition are random for the count under analysis.
- Using inconsistent exposure units. For exposure \(t\), use \(t\Lambda\) as the conditional Poisson mean and \(t^2\operatorname{Var}(\Lambda)\) as the between-rate term.
The central calculation is short, but the modeling decision comes first: specify what is random, what is fixed, and what has been conditioned upon. This conditional-variance reasoning is aligned with the SOA Exam ASTAM (Advanced Short-Term Actuarial Mathematics) syllabus.
Check Your Understanding
For each problem, use the conditional Poisson model and state the unconditional count mean and variance as requested.
- A randomly selected one-year risk has rate \(0.4\) with probability \(0.5\) and rate \(1.2\) with probability \(0.5\). Find the count mean and variance. (Answer: mean \(0.8\), variance \(0.96\))
- A risk has annual rate \(0.6\) with probability \(0.75\) and \(1.8\) with probability \(0.25\), fixed over three years. Find the three-year count mean and variance. (Answer: mean \(2.7\), variance \(5.13\))
- A one-year mixed-risk count has mean \(1.5\), and the rate variance is \(0.25\). Find the count variance and its excess over the mean. (Answer: variance \(1.75\), excess \(0.25\))
- A fixed portfolio has 10 independent policies at annual rate \(0.5\) and 10 at rate \(1.5\). Find its one-year total count mean and variance. (Answer: mean \(20\), variance \(20\))
- Each of 20 independent policies is randomly assigned annual rate \(0.5\) with probability \(0.4\) or \(1.5\) with probability \(0.6\). Find the one-year total count mean and variance. (Answer: mean \(22\), variance \(26.8\))