Deriving Prediction Error Under The Overdispersed Poisson Model

Introduction

A reserve prediction is uncertain for two different reasons. Future claim amounts vary even if the model parameters were known exactly, and the parameters themselves must be estimated from observed claims. Under an overdispersed Poisson (ODP) model, the first source is called process variance; the second is parameter variance. A reserve standard error that includes only one of them does not describe the full prediction error.

This tutorial derives the two contributions and shows how to combine them. The calculations use fitted exposures and reserve quantities, including cases where several future cells share an estimated parameter. The focus is prediction error for the total future reserve, not a comparison of reserving methods.

Definition: In an ODP model, an observed incremental amount \(C_k\), conditional on its model parameters, has mean \(\mu_k\) and variance \(\phi\mu_k\), where \(\phi>0\) is the dispersion parameter. The ODP variance specification does not, by itself, require the response to have a Poisson probability distribution. For reserve prediction, process variance measures variation in future outcomes conditional on fitted parameters; parameter variance measures uncertainty in the estimated conditional mean.

The Two Contributions to Prediction Error

Let \(R\) be the total future reserve and let \(\boldsymbol{\theta}\) denote the unknown model parameters. Conditional on those parameters and the observed data, suppose future cells are independent and have means \(\mu_k(\boldsymbol{\theta})\) and variances \(\phi\mu_k(\boldsymbol{\theta})\). The law of total variance gives

\operatorname{Var}(R\mid\text{observed data}) = E\!\left[\operatorname{Var}(R\mid\boldsymbol{\theta},\text{observed data})\mid\text{observed data}\right] + \operatorname{Var}\!\left(E[R\mid\boldsymbol{\theta},\text{observed data}]\mid\text{observed data}\right).

The first term is process variance: conditional future cell variances add when the cells are independent. The second is parameter variance: the predicted total changes when the fitted parameters change. In practice, both are estimated by plugging in fitted parameters and an estimated dispersion.

\widehat{\operatorname{Var}}_{\mathrm{pred}}(R) = \underbrace{\phi\sum_{k\in F}\hat{\mu}_k}_{\text{process contribution}} + \underbrace{\widehat{\operatorname{Var}}\!\left(\sum_{k\in F}\hat{\mu}_k\right)}_{\text{parameter contribution}}, \qquad \widehat{\operatorname{SE}}_{\mathrm{pred}}(R) = \sqrt{\widehat{\operatorname{Var}}_{\mathrm{pred}}(R)}.

Here \(F\) is the set of future cells and \(\hat{\mu}_k\) is the fitted mean for cell \(k\). This plug-in calculation treats the dispersion estimate \(\phi\) as fixed. If uncertainty in the dispersion estimate is to be included, it requires an additional treatment; it is not a third term supplied automatically by this formula.

Derivation for an Exposure-Rate Model

A simple model makes the two sources especially clear. Suppose training observations have total exposure \(E\) and total observed amount \(Y\). A common rate parameter \(\theta\) gives fitted mean \(\theta E\) for training and mean \(\theta F\) for future exposure \(F\). Assume the training and future amounts are independent conditional on \(\theta\), with ODP variance proportional to their means. The rate estimate is \(\hat{\theta}=Y/E\).

The variance of the training total is \(\phi\theta E\). Since \(\hat{\theta}=Y/E\), its estimated sampling variance is therefore \(\phi\theta E/E^2=\phi\theta/E\). The future total has conditional mean \(\theta F\) and conditional variance \(\phi\theta F\). Its predicted mean is \(F\hat{\theta}\), so the parameter contribution is \(F^2\operatorname{Var}(\hat{\theta})\). Substituting the fitted rate gives the practical formula below.

\widehat{\operatorname{Var}}_{\mathrm{process}}(R)=\phi F\hat{\theta}, \qquad \widehat{\operatorname{Var}}_{\mathrm{parameter}}(R) =F^2\frac{\phi\hat{\theta}}{E}, \qquad \widehat{\operatorname{Var}}_{\mathrm{pred}}(R) =\phi F\hat{\theta}+F^2\frac{\phi\hat{\theta}}{E}.

This is an estimated prediction variance under the stated model, not an exact guarantee for any particular future outcome. It depends on the ODP variance assumption, the exposure-rate structure, and the treatment of the fitted rate as approximately unbiased with variance \(\phi\theta/E\). It does not apply unchanged when future cells use different parameters or when their dependence is ignored.

Worked Example: One Fitted Rate and One Future Reserve

A hypothetical portfolio has training exposure \(E=80\) exposure units and observed aggregate claims \(Y=100\) thousand dollars. The future period has exposure \(F=40\) units. Assume the exposure-rate ODP model above, with fixed \(\phi=1.2\). There is no discounting or mortality adjustment in this example. Calculate the predicted reserve and its prediction standard error.

Method. Estimate the rate, calculate process and parameter variances separately, and then take the square root of their sum. The ODP variance specification is \(\operatorname{Var}(Y)=\phi E\theta\) for training and \(\operatorname{Var}(R\mid\theta)=\phi F\theta\) for future claims.

Fitted rate and reserve mean.

\hat{\theta}=\frac{Y}{E}=\frac{100}{80}=1.25 \quad\text{thousand dollars per exposure unit}, \qquad \hat{R}=F\hat{\theta}=40(1.25)=50 \quad\text{thousand dollars}.

Process contribution. The fitted future mean is 50, so the estimated future process variance is

\widehat{\operatorname{Var}}_{\mathrm{process}}(R) =\phi\hat{R}=1.2(50)=60 \quad\text{thousand-dollars squared}.

Parameter contribution. First estimate the rate variance:

\widehat{\operatorname{Var}}(\hat{\theta}) =\frac{\phi\hat{\theta}}{E} =\frac{1.2(1.25)}{80} =\frac{1.5}{80} =0.01875.

Multiplying by the square of the future exposure gives

\widehat{\operatorname{Var}}_{\mathrm{parameter}}(R) =F^2\widehat{\operatorname{Var}}(\hat{\theta}) =40^2(0.01875)=1{,}600(0.01875)=30 \quad\text{thousand-dollars squared}.

Total and check. The total variance is \(60+30=90\) thousand-dollars squared, and the standard error is \(\sqrt{90}=9.4868\) thousand dollars, rounded to four decimal places. The parameter term is positive and smaller than the process term here; omitting it would give \(\sqrt{60}\), not the required total prediction standard error.

Several Parameters and Shared Future Cells

In a reserving triangle, future cell means usually depend on fitted parameters such as development effects or accident-period effects. If a generalized linear model uses a log link, write the mean in cell \(k\) as \(\mu_k=e_k\exp(\mathbf{x}_k^\mathsf{T}\boldsymbol{\beta})\), where \(e_k\) is a known exposure or offset and \(\mathbf{x}_k\) is the covariate vector. Let \(X_T\) contain the training-cell covariates and let \(W_T\) be diagonal with fitted training means on its diagonal. Under the usual large-sample GLM approximation, with identifiable parameters and a full-rank design after any constraints, the fitted parameter covariance is approximately \(\phi(X_T^\mathsf{T}W_TX_T)^{-1}\).

The gradient of the total future mean with respect to \(\boldsymbol{\beta}\) is \(\mathbf{g}=\sum_{k\in F}\hat{\mu}_k\mathbf{x}_k\). A first-order Taylor approximation, also called the delta method, then gives the following parameter contribution.

\widehat{\operatorname{Var}}_{\mathrm{parameter}}(R) \approx \mathbf{g}^\mathsf{T} \left[\phi(X_T^\mathsf{T}W_TX_T)^{-1}\right] \mathbf{g}, \qquad \widehat{\operatorname{Var}}_{\mathrm{process}}(R) = \phi\sum_{k\in F}\hat{\mu}_k.

This expression includes covariance among fitted parameters as well as the variance of each parameter. Equivalently, if individual future cell means have covariance matrix \(\widehat{\Sigma}_F\) due to shared estimated parameters, the parameter variance of their sum is \(\mathbf{1}^\mathsf{T}\widehat{\Sigma}_F\mathbf{1}\). Adding only the diagonal cell variances loses the covariance terms.

Fit the training model.
Obtain fitted training means, future cell means, and the covariance matrix of the estimated parameters under the chosen ODP model.

Calculate process variance.
For conditionally independent future cells, multiply each fitted future mean by \(\phi\) and add the results.

Calculate parameter variance.
Use the covariance of the fitted future total, retaining covariance terms for cells that share estimated parameters.

Combine and report.
Add the two variance contributions, then take the square root. Keep variance units squared until this final step.

Worked Example: Two Independent Groups With Separate Rates

A hypothetical reserve is split into two independent groups, with amounts measured in thousands of dollars. Both groups use fixed dispersion \(\phi=1.4\), but each has its own exposure-rate parameter. Group A has training exposure 100 and observed total 90, followed by future exposure 50. Group B has training exposure 200 and observed total 260, followed by future exposure 100. Assume independent training and future amounts conditional on the group rates, and independent rate estimates across groups. There is no discounting or mortality adjustment. Find the total prediction standard error.

Method. Apply the exposure-rate formula separately to each group, then add the four variance contributions. Because the groups and their fitted rates are independent, there are no cross-group covariance terms.

Group A. Its fitted rate is \(90/100=0.9\) thousand dollars per exposure unit, and its predicted future mean is \(50(0.9)=45\) thousand dollars. The process variance is \(1.4(45)=63\) thousand-dollars squared. Its parameter contribution is

50^2\frac{1.4(0.9)}{100} =2{,}500\left(\frac{1.26}{100}\right) =2{,}500(0.0126) =31.5 \quad\text{thousand-dollars squared}.

Group B. Its fitted rate is \(260/200=1.3\), and its predicted future mean is \(100(1.3)=130\) thousand dollars. The process variance is \(1.4(130)=182\) thousand-dollars squared. Its parameter contribution is

100^2\frac{1.4(1.3)}{200} =10{,}000\left(\frac{1.82}{200}\right) =10{,}000(0.0091) =91 \quad\text{thousand-dollars squared}.

Total and check. The predicted total is \(45+130=175\) thousand dollars. The total variance is \(63+31.5+182+91=367.5\) thousand-dollars squared. Therefore the standard error is \(\sqrt{367.5}=19.1703\) thousand dollars, rounded to four decimal places. The total exceeds the process-only variance \(63+182=245\), as it must when the parameter contributions are positive.

Shared Parameters: A Covariance Check

Worked Example: Two Future Cells Using One Rate

A hypothetical triangle model has one common rate estimated from training exposure \(E=120\) and observed total \(Y=180\) thousand dollars. Two future cells have exposures 20 and 30. Assume \(\phi=1.2\), a common rate for both future cells, independent future claim processes conditional on that rate, and independence between training and future amounts conditional on the rate. There is no discounting or mortality adjustment. Calculate the prediction standard error for their combined reserve.

Method. The process variances add across future cells. Parameter uncertainty, however, affects both predicted cell means through the same fitted rate, so calculate parameter variance for their combined mean (or include their covariance).

The fitted rate is \(180/120=1.5\), so the cell means are \(20(1.5)=30\) and \(30(1.5)=45\) thousand dollars. The combined predicted reserve is \(30+45=75\) thousand dollars. Process variance is

1.2(30)+1.2(45)=36+54=90 \quad\text{thousand-dollars squared}.

The estimated rate variance is \(1.2(1.5)/120=1.8/120=0.015\). The combined future exposure is \(20+30=50\), so parameter variance is \(50^2(0.015)=2{,}500(0.015)=37.5\) thousand-dollars squared. As a covariance check, the separate parameter variances are \(20^2(0.015)=6\) and \(30^2(0.015)=13.5\); their covariance is \(20(30)(0.015)=9\). Thus \(6+13.5+2(9)=37.5\), matching the combined-rate calculation.

The total prediction variance is \(90+37.5=127.5\) thousand-dollars squared, and the standard error is \(\sqrt{127.5}=11.2916\) thousand dollars, rounded to four decimal places. Treating the cell estimates as parameter-independent would omit the covariance of 9 and understate the parameter contribution.

Common Mistakes and Why the Distinction Matters

A frequent exam trap is to calculate the process contribution correctly but either omit parameter variance or combine the two contributions incorrectly. In the two-group example, a mistaken calculation might add the four contributions as \(63+31.5+182+90.5=367\), yielding \(\sqrt{367}=19.1572\) thousand dollars. The last term is wrong: Group B's parameter variance is \(100^2[1.4(1.3)/200]=91\), not 90.5. The corrected total is 367.5 and the corrected standard error is 19.1703 thousand dollars.

Do not confuse a reserve estimate with its prediction error. The fitted reserve is a mean prediction; the prediction variance includes future process variation and estimated-parameter uncertainty.
Do not add standard errors. Add variance contributions first, then take one square root. For independent groups, \(\operatorname{Var}(R_A+R_B)=\operatorname{Var}(R_A)+\operatorname{Var}(R_B)\).
Do not discard shared-parameter covariance. If future cells use the same estimated rate or development parameters, their predicted means are generally correlated even when the future claim processes are conditionally independent.
Track units. If reserve amounts are in thousands of dollars, variance is in thousand-dollars squared and the standard error returns to thousands of dollars.

These distinctions matter because parameter uncertainty can be substantial when training exposure is limited or the future exposure is large. The formulas make that relationship visible: process variance grows with future fitted mean, while the rate-model parameter contribution also depends on the square of future exposure and decreases as training exposure increases.

Key takeaway: Under the stated ODP assumptions, total reserve prediction variance is the future process variance plus the variance of the fitted future mean. When future cells share fitted parameters, compute the variance of their total mean with the covariance terms included.

Check Your Understanding

For each problem, calculate the requested total prediction variance or standard error under the stated ODP assumptions.

A single exposure-rate group has training exposure 60, observed total 72, future exposure 25, and fixed \(\phi=1.5\). Amounts are in thousands of dollars. Find the total prediction variance and standard error. (Answer: variance \(=63.75\) thousand-dollars squared; standard error \(=7.9844\) thousand dollars.)
Two independent groups have common fixed \(\phi=1\). Group A has training exposure 50, observed total 60, and future exposure 20. Group B has training exposure 80, observed total 72, and future exposure 40. Amounts are in thousands of dollars. Find the combined prediction variance and standard error. (Answer: variance \(=87.6\) thousand-dollars squared; standard error \(=9.3596\) thousand dollars.)
A common-rate model has training exposure 200, observed total 240, fixed \(\phi=1.5\), and two future exposures of 30 and 50. Amounts are in thousands of dollars. Find the prediction variance for the combined future reserve, including shared-rate parameter uncertainty. (Answer: variance \(=201.6\) thousand-dollars squared; standard error \(=14.1986\) thousand dollars.)
A common-rate model has training exposure 100, observed total 80, and \(\phi=2\). Two future cells have exposures 10 and 30. Calculate the total variance, including the covariance induced by the common fitted rate. (Answer: process variance \(=64\), parameter variance \(=25.6\), total variance \(=89.6\) thousand-dollars squared; standard error \(=9.4657\) thousand dollars.)
For a two-parameter fitted model, the gradient of the combined future mean is \(\mathbf{g}=(20,30)^\mathsf{T}\), and the estimated parameter covariance matrix is \(\begin{pmatrix}0.04&-0.01\\-0.01&0.09\end{pmatrix}\). The fitted future cell means sum to 100 and \(\phi=1.2\). Find the process contribution, parameter contribution, total variance, and standard error. (Answer: process variance \(=120\), parameter variance \(=85\), total variance \(=205\); standard error \(=14.3178\), in the corresponding monetary units.)

What You'll Learn