Fisher Information and Covariance of GLM Estimates

Why Information Matters in a Ratemaking GLM

A fitted GLM provides coefficient estimates, but those estimates are not equally precise. Their uncertainty depends on the amount and pattern of data, the model’s mean and variance assumptions, and how the rating variables are represented in the design matrix. Fisher information summarizes that precision locally: more information generally means smaller coefficient variances.

For a ratemaking application, coefficient standard errors help indicate whether estimated effects are precise or uncertain. They are not measures of the variability of future claims themselves. The latter is response or process variability; coefficient covariance describes uncertainty in the estimated model parameters.

Definition: For a parameter vector \( \beta \), the Fisher information matrix is the expected negative Hessian of the log-likelihood, \( \mathcal{I}(\beta)=E[-\partial^2\ell(\beta)/\partial\beta\partial\beta'] \). Under regularity conditions it is also the covariance matrix of the score vector. For a full-rank GLM, the large-sample covariance estimate of the coefficient estimator is \( \widehat{\operatorname{Cov}}(\hat{\beta})\approx\mathcal{I}(\hat{\beta})^{-1} \). The standard error of \( \hat{\beta}_j \) is the square root of the \(j\)th diagonal entry of that covariance matrix.

Expected Information for a GLM

Assume independent responses \(Y_i\) from an exponential-dispersion family, with mean \( \mu_i \), variance \( \operatorname{Var}(Y_i)=\phi V(\mu_i) \), and dispersion parameter \( \phi \). The linear predictor is \( \eta_i=x_i'\beta \), and the link satisfies \(g(\mu_i)=\eta_i\). Here \(x_i\) is the column vector of predictors for observation \(i\). The response distribution, link, variance function, and any offsets determine the fitted means and thus the information.

Under these assumptions, the expected information has the weighted cross-product form below. The expression applies to independent observations and a correctly specified mean and variance model. If observations are dependent or the assumed variance structure is wrong, this formula need not describe the actual sampling covariance; an appropriate dependence or robust covariance method may be required.

\mathcal{I}(\beta)=X'WX,\qquad W=\operatorname{diag}(w_1,\ldots,w_n),\qquad w_i=\frac{1}{\phi V(\mu_i)} \left(\frac{d\mu_i}{d\eta_i}\right)^2.

The expected information is evaluated at the fitted parameter values when estimating covariance. If \( \phi \) is known, use that value. If it is estimated, the usual model-based calculation substitutes an estimate of dispersion. For a Poisson model with the canonical log link, \(V(\mu_i)=\mu_i\) and \(d\mu_i/d\eta_i=\mu_i\), so \(w_i=\mu_i\). For a normal model with identity link and known variance \( \sigma^2 \), \(w_i=1/\sigma^2\).

Why the Weighted Cross-Product Appears

A short derivation explains both the formula and the role of the variance function. For an exponential-dispersion observation, the derivative of its log-likelihood with respect to its linear predictor is \( (Y_i-\mu_i)(d\mu_i/d\eta_i)/[\phi V(\mu_i)] \). By the chain rule, the score contribution for \( \beta \) is this scalar multiplied by \(x_i\). The score has expectation zero because \(E[Y_i-\mu_i]=0\).

U(\beta)=\sum_i x_i \frac{(Y_i-\mu_i)(d\mu_i/d\eta_i)} {\phi V(\mu_i)}

Independence makes the covariance of the total score equal to the sum of the covariances of its contributions. Since \( \operatorname{Var}(Y_i)=\phi V(\mu_i) \), the covariance contribution for observation \(i\) is \(x_ix_i'w_i\). Summing gives \(X'WX\). Under the standard regularity conditions, this score covariance equals the expected negative Hessian, giving the stated Fisher information.

\operatorname{Cov}(U(\beta)) =\sum_i x_ix_i' \frac{\operatorname{Var}(Y_i)(d\mu_i/d\eta_i)^2} {\phi^2 V(\mu_i)^2} =\sum_i x_ix_i'w_i =X'WX.

If this matrix is nonsingular, its inverse describes approximate joint coefficient uncertainty. The off-diagonal entries are covariances, not standard errors. A negative covariance can arise because changing one coefficient can be partly offset by changing another while preserving fitted values for the observed design.

Worked Example: Gaussian Model With a Two-Level Rating Factor

A hypothetical insurer models a continuous claim-cost response using an intercept and an indicator for a second rating class. There are four observations in the reference class, with responses 10, 12, 11, and 13, and two in the second class, with responses 15 and 17. Assume independent normal responses with identity link and known variance \( \sigma^2=9 \). Find the coefficient estimates, covariance matrix, and standard errors.

Method and assumptions. For this normal identity-link model, \(Y_i\sim N(x_i'\beta,9)\), so every observation has information weight \(1/9\). Use \(x_i=(1,0)'\) for each reference-class observation and \(x_i=(1,1)'\) for each second-class observation.

The reference-class response mean is \( (10+12+11+13)/4=46/4=11.5 \). The second-class mean is \( (15+17)/2=32/2=16 \). Thus the fitted intercept is \( \hat{\beta}_0=11.5 \), and the fitted class difference is \( \hat{\beta}_1=16-11.5=4.5 \).

Each reference row contributes \( \begin{pmatrix}1&0\\0&0\end{pmatrix} \) to \(X'X\), and each second-class row contributes \( \begin{pmatrix}1&1\\1&1\end{pmatrix} \). Therefore,

X'X= 4\begin{pmatrix}1&0\\0&0\end{pmatrix} +2\begin{pmatrix}1&1\\1&1\end{pmatrix} =\begin{pmatrix}6&2\\2&2\end{pmatrix}, \qquad \mathcal{I}=\frac{1}{9}\begin{pmatrix}6&2\\2&2\end{pmatrix}.

The determinant of the integer matrix is \(6(2)-2(2)=8\). Its inverse is \( \frac{1}{8}\begin{pmatrix}2&-2\\-2&6\end{pmatrix} \), so multiplying by 9 gives the covariance matrix:

\widehat{\operatorname{Cov}}(\hat{\beta}) =\mathcal{I}^{-1} =\begin{pmatrix}2.25&-2.25\\-2.25&6.75\end{pmatrix}.

The standard errors are \( \sqrt{2.25}=1.5000 \) and \( \sqrt{6.75}=2.5981 \), rounded to four decimals. The covariance matrix is exact for this normal model with known variance and fixed design. Common mistake avoided: the intercept appears in every row, including both second-class rows; omitting those intercept contributions understates the information for the intercept.

Using the Inverse and Reading Its Entries

Matrix inversion combines information across coefficients. In particular, an estimate’s variance is not generally the reciprocal of the corresponding diagonal information entry: that shortcut works only when the information matrix is diagonal. Correlated columns of \(X\) produce off-diagonal information entries, and the complete matrix must be inverted.

The covariance also allows uncertainty for a linear combination. For any fixed vector \(c\), the variance of \(c'\hat{\beta}\) is estimated by \(c'\widehat{\operatorname{Cov}}(\hat{\beta})c\). For example, with an intercept and class indicator, the fitted linear predictor for the second class is \( \hat{\beta}_0+\hat{\beta}_1 \), so its variance uses both coefficient variances and twice their covariance.

Worked Example: Checking the Design Contributions in a Gaussian Model

For a hypothetical portfolio segmentation, three independent responses use design row \( (1,0) \), and two use \( (1,1) \). Assume \(Y_i\sim N(x_i'\beta,4)\), with identity link and known variance 4. Calculate the coefficient covariance matrix and standard errors. The response values are not needed for this covariance calculation because the known normal variance and design determine the information.

Method. Each observation has weight \(1/4\). The design cross-product is formed by adding three reference-row contributions and two second-class contributions:

X'X=3\begin{pmatrix}1&0\\0&0\end{pmatrix} +2\begin{pmatrix}1&1\\1&1\end{pmatrix} =\begin{pmatrix}5&2\\2&2\end{pmatrix}, \qquad \mathcal{I}=\frac{1}{4}\begin{pmatrix}5&2\\2&2\end{pmatrix}.

The determinant of \(X'X\) is \(5(2)-2(2)=6\), giving \( (X'X)^{-1}=\frac{1}{6}\begin{pmatrix}2&-2\\-2&5\end{pmatrix} \). Since \( \mathcal{I}^{-1}=4(X'X)^{-1} \),

\widehat{\operatorname{Cov}}(\hat{\beta}) =\begin{pmatrix}1.3333&-1.3333\\-1.3333&3.3333\end{pmatrix}.

The standard errors are \( \sqrt{1.3333}=1.1547 \) and \( \sqrt{3.3333}=1.8257 \), rounded to four decimals. Common mistake avoided: the intercept information is based on all five observations, not only the three reference-class rows.

Worked Example: Poisson Frequency GLM With Two Rating Classes

Consider a hypothetical frequency model with a log link and one unit of exposure in each of two rating cells. The observed claim counts are 8 in the reference class and 12 in the second class. Assume independent Poisson counts, so each cell count has variance equal to its mean. Find the fitted coefficients, expected information, covariance matrix, and standard errors.

Method and fitted values. With one observation per cell, the fitted means are \( \hat{\mu}_0=8 \) and \( \hat{\mu}_1=12 \). The model is \( \log(\mu)=\beta_0+\beta_1z \), where \(z=0\) for the reference cell and \(z=1\) for the second. Thus \( \hat{\beta}_0=\log(8) \) and \( \hat{\beta}_1=\log(12/8)=\log(1.5) \).

For a Poisson log-link model the weight is the fitted mean. The reference cell contributes \(8\begin{pmatrix}1&0\\0&0\end{pmatrix}\) and the second cell contributes \(12\begin{pmatrix}1&1\\1&1\end{pmatrix}\), so

\mathcal{I}(\hat{\beta}) =\begin{pmatrix}20&12\\12&12\end{pmatrix}.

The determinant is \(20(12)-12(12)=240-144=96\). Therefore,

\widehat{\operatorname{Cov}}(\hat{\beta}) =\frac{1}{96}\begin{pmatrix}12&-12\\-12&20\end{pmatrix} =\begin{pmatrix}0.1250&-0.1250\\-0.1250&0.2083\end{pmatrix}.

The standard errors are \( \sqrt{0.1250}=0.3536 \) and \( \sqrt{0.2083}=0.4564 \), rounded to four decimals. As a check, the variance of the second class’s fitted log mean \( \hat{\beta}_0+\hat{\beta}_1 \) is \(0.1250+2(-0.1250)+0.2083=0.0833\), which is also \(1/12\), the reciprocal of that cell’s fitted Poisson mean. Common mistake avoided: use the fitted means as the Poisson expected-information weights; do not substitute arbitrary observed residuals for those weights.

Common Traps and Conditions for Use

The inverse-information calculation is a local, model-based uncertainty approximation for maximum-likelihood estimates. In regular, sufficiently large samples, the estimator is approximately multivariate normal with covariance \( \mathcal{I}(\hat{\beta})^{-1} \). For the Gaussian identity-link model with known variance, that covariance is exact. For other GLMs it is generally an asymptotic approximation.

Dropping design contributions. Every observation contributes its weight times \(x_ix_i'\). An intercept contributes to the intercept row and column for every observation.
Confusing information with covariance. The information matrix is inverted to obtain the coefficient covariance matrix; its diagonal entries are not themselves coefficient variances.
Ignoring covariances. For a linear combination, include the off-diagonal terms. For two coefficients, \( \operatorname{Var}(\hat{\beta}_0+\hat{\beta}_1)=\operatorname{Var}(\hat{\beta}_0)+\operatorname{Var}(\hat{\beta}_1)+2\operatorname{Cov}(\hat{\beta}_0,\hat{\beta}_1) \).
Using the formula with a rank-deficient design. If the columns of \(X\) are linearly dependent, \(X'WX\) is singular and has no ordinary inverse. The coefficients are not separately identifiable without changing the parameterization or adding information.
Treating standard errors as claim volatility. Coefficient standard errors quantify uncertainty about fitted parameters, not the full predictive distribution of future losses.

The information matrix therefore connects model structure to estimation precision. Its weights express how informative each response is under the assumed GLM, while \(X\) expresses which coefficients that response informs. This makes a careful row-by-row matrix construction an important check before interpreting any standard error.

These calculations are aligned with the CAS Exam 8 (Advanced Ratemaking) syllabus.

Check Your Understanding

For each problem, compute the requested quantity using the stated model assumptions.

In a hypothetical Gaussian identity-link model, two observations have design row \( (1,0) \), and three have row \( (1,1) \). The independent responses have known variance 16. Find the covariance matrix and coefficient standard errors. (Answer: covariance \( \begin{pmatrix}8&-8\\-8&13.3333\end{pmatrix} \); standard errors \(2.8284\) and \(3.6515\).)
A hypothetical two-cell Poisson frequency model has a log link, unit exposures, and fitted counts 6 in the reference cell and 10 in the second cell. Find the covariance matrix and standard errors for the intercept and indicator coefficient. (Answer: covariance \( \begin{pmatrix}0.1667&-0.1667\\-0.1667&0.2667\end{pmatrix} \); standard errors \(0.4082\) and \(0.5164\).)
For a Poisson log-link model, one fitted cell has mean 4 and design row \( (1,0) \), while another has mean 9 and design row \( (1,1) \). Find the covariance matrix and standard errors. (Answer: covariance \( \begin{pmatrix}0.2500&-0.2500\\-0.2500&0.3611\end{pmatrix} \); standard errors \(0.5000\) and \(0.6009\).)
Using the covariance matrix in question 2, find the estimated standard error of the second class’s fitted log mean, \( \hat{\beta}_0+\hat{\beta}_1 \). (Answer: variance \(0.1000\); standard error \(0.3162\).)
A Gamma GLM has log link, known dispersion \( \phi=0.5 \), and three independent observations: two with design row \( (1,0) \), one with row \( (1,1) \). Use \( \operatorname{Var}(Y_i)=\phi\mu_i^2 \) to find the coefficient covariance matrix. (Answer: information \( \begin{pmatrix}6&2\\2&2\end{pmatrix} \); covariance \( \begin{pmatrix}0.2500&-0.2500\\-0.2500&0.7500\end{pmatrix} \).)

What You'll Learn