End-to-End Calculation of an Approximate Population Fisher Matrix

From a Model to a Cohort Information Matrix

A population Fisher matrix is useful only when its entries can be traced back to the model and the planned observations. The calculation has several linked parts: specify the marginal mean and covariance for each subject, differentiate both with respect to the parameters, calculate each subject’s expected information, and add those matrices across independent subjects. This tutorial carries out each step for a small Gaussian mixed-effects example.

The example has two mean parameters and two covariance parameters. Two planned subjects contribute two observations each, and a third contributes one observation. The repeated observations within a subject may be correlated; the calculation uses their joint covariance rather than treating the records as independent. Subjects, in contrast, are assumed independent of one another.

Definition: For a subject with marginal observations $Y_i\sim N(\mu_i(\theta),V_i(\theta))$, the expected Fisher information for parameters $\theta_a,\theta_b$ is $$ \mathcal I_{i,ab} = \left(\frac{\partial\mu_i}{\partial\theta_a}\right)^\mathsf{T} V_i^{-1} \left(\frac{\partial\mu_i}{\partial\theta_b}\right) + \frac{1}{2}\operatorname{tr} \left( V_i^{-1}\frac{\partial V_i}{\partial\theta_a} V_i^{-1}\frac{\partial V_i}{\partial\theta_b} \right). $$ The first term contributes when the parameter affects the mean; the second contributes when it affects the covariance. The cohort information is $\mathcal I_{\mathrm{cohort}}=\sum_i\mathcal I_i$ when subjects are independent.

This formula assumes a multivariate normal marginal model, differentiable mean and covariance functions, and a positive-definite $V_i$. Here, the mean parameters are distinct from the covariance parameters, and the covariance does not depend on the mean parameters. Under a normal model, the expected cross-information between mean and covariance parameters is then zero. The resulting matrix has separate mean and covariance blocks.

In a nonlinear mixed-effects model, the marginal distribution need not be exactly normal. A first-order population calculation can use an approximate marginal mean and covariance, together with their derivatives at nominal parameter values, in the same expression. The calculation below is exact for the specified Gaussian marginal model and also illustrates the bookkeeping used in that approximation.

Specify the Example and Its Assumptions

Let the mean be $\mu_i=X_i\beta$, where $\beta=(\beta_0,\beta_1)^\mathsf{T}$, and use the mean function $\beta_0+\beta_1x$. Two subjects are measured twice, both times at $x=1$; the third is measured once at $x=0.5$. At the nominal covariance parameters, the two-observation subjects have $V_2=3I_2$, and the single-observation subject has $V_1=[2]$.

These marginal covariances can be embedded in a mixed-effects model with a subject random intercept of variance $0.5$ and positive-definite residual covariance: $R_2=3I_2-0.5\mathbf 1\mathbf 1^\mathsf{T}$ for a two-observation subject and $R_1=[1.5]$ for a single-observation subject. The covariance parameters $q$ and $r$ are dimensionless local parameters, nominally zero. Their derivatives below may be assigned to the residual covariance components. The residual covariance remains positive definite in a sufficiently small neighborhood of the nominal values, so this defines a valid local Gaussian mixed-effects model.

For a two-observation subject, define the covariance derivatives using the following matrices, with $V_2^{-1/2}=I_2/\sqrt{3}$:

A_q=V_2^{-1/2}V_{2,q}V_2^{-1/2} = \begin{bmatrix} 2/\sqrt{3}&0\\ 0&0 \end{bmatrix}, \qquad A_r=V_2^{-1/2}V_{2,r}V_2^{-1/2} = \begin{bmatrix} 7\sqrt{3}/144&0\\ 0&\sqrt{5039/6912} \end{bmatrix}.

Here $V_{2,q}$ and $V_{2,r}$ denote derivatives with respect to $q$ and $r$, evaluated at the nominal values. For the single-observation subject, the corresponding standardized derivatives are $V_1^{-1/2}V_{1,q}V_1^{-1/2}=1/2$ and $V_1^{-1/2}V_{1,r}V_1^{-1/2}=1$. These derivative choices define a concrete local example; they are not intended as a universal covariance parameterization.

Worked Example: Mean Information for a Two-Observation Subject

For either two-observation subject, the mean-sensitivity matrix has both rows equal to $[1,1]$, because both observations are at $x=1$. Since $V_2^{-1}=(1/3)I_2$, the mean contribution is

\mathcal I_{2,\mathrm{mean}} =X_2^\mathsf{T}V_2^{-1}X_2 = \begin{bmatrix}1&1\\1&1\end{bmatrix} \frac{1}{3}I_2 \begin{bmatrix}1&1\\1&1\end{bmatrix} = \begin{bmatrix}2/3&2/3\\2/3&2/3\end{bmatrix}.

The repeated rows provide information about the combination $\beta_0+\beta_1$, but by themselves do not distinguish $\beta_0$ from $\beta_1$. This is reflected in the rank-one matrix: its two columns are identical. The observation covariance is $3I_2$, so no within-subject correlation is assumed for this particular example’s nominal two-record covariance.

Calculate the Covariance Contributions

The covariance term can be calculated using standardized derivative matrices $A_a=V^{-1/2}V_{,a}V^{-1/2}$. By cyclic invariance of the trace, its entry for parameters $a$ and $b$ is $\frac12\operatorname{tr}(A_aA_b)$. For the diagonal matrices specified above, this reduces to one-half the sum of the products of corresponding diagonal entries.

Worked Example: Covariance Information for a Two-Observation Subject

For $q$, the standardized derivative has diagonal entries $2/\sqrt{3}$ and 0. Therefore,

\mathcal I_{2,qq} =\frac12\operatorname{tr}(A_q^2) =\frac12\left(\frac{4}{3}\right) =\frac23.

For the cross-entry, the only nonzero product is between the first diagonal entries:

\mathcal I_{2,qr} =\frac12\operatorname{tr}(A_qA_r) =\frac12\left(\frac{2}{\sqrt3}\frac{7\sqrt3}{144}\right) =\frac{7}{144}.

Finally, the squared diagonal entries of $A_r$ sum to $53/72$, giving

\mathcal I_{2,rr} =\frac12\operatorname{tr}(A_r^2) =\frac12\left[\left(\frac{7\sqrt3}{144}\right)^2+\frac{5039}{6912}\right] =\frac{53}{144}.

Thus the two-observation subject’s covariance block is $\begin{bmatrix}2/3&7/144\\7/144&53/144\end{bmatrix}$. Its determinant is positive, so it is positive definite. The derivatives, rather than the covariance $V_2$ alone, determine this block: knowing only a nominal covariance is not enough to calculate information about covariance parameters.

Worked Example: Information for a Single-Observation Subject

At $x=0.5$, the mean-sensitivity row is $[1,0.5]$ and $V_1^{-1}=1/2$. The mean block is

\mathcal I_{1,\mathrm{mean}} =X_1^\mathsf{T}V_1^{-1}X_1 =\frac12 \begin{bmatrix}1\\0.5\end{bmatrix} \begin{bmatrix}1&0.5\end{bmatrix} = \begin{bmatrix}1/2&1/4\\1/4&1/8\end{bmatrix}.

The standardized covariance derivatives are the scalars $1/2$ and $1$. The covariance information is therefore

\mathcal I_{1,\mathrm{cov}} = \frac12 \begin{bmatrix} (1/2)^2&(1/2)(1)\\ (1/2)(1)&1^2 \end{bmatrix} = \begin{bmatrix}1/8&1/4\\1/4&1/2\end{bmatrix}.

This is rank one because one scalar observation supplies only one covariance score direction. That is not an arithmetic defect: the subject’s covariance information alone cannot distinguish two covariance parameters.

Assemble and Check the Cohort Matrix

Order the full parameter vector as $(\beta_0,\beta_1,q,r)$. There are two subjects of the two-observation type and one of the single-observation type. Independence across subjects means their expected information matrices add. Within each subject, the mean and covariance blocks have zero cross-information under the stated Gaussian model.

Worked Example: Adding the Subject Contributions

The mean block is two copies of the two-observation contribution plus the single-observation contribution. For example, its intercept-intercept entry is $2(2/3)+1/2=11/6$, while its intercept-slope entry is $2(2/3)+1/4=19/12$. The complete cohort matrix is

\mathcal I_{\mathrm{cohort}} = \begin{bmatrix} 11/6&19/12&0&0\\ 19/12&35/24&0&0\\ 0&0&35/24&25/72\\ 0&0&25/72&89/72 \end{bmatrix}.

Check the covariance entries separately: $qq=2(2/3)+1/8=35/24$; $qr=2(7/144)+1/4=25/72$; and $rr=2(53/144)+1/2=89/72$. These checks guard against adding a subject contribution only once, or mixing a subject-level entry with a cohort total. The matrix is symmetric, as expected for Fisher information.

For a positive-definite information matrix, eigenvectors identify parameter combinations and the corresponding eigenvalues quantify information along those directions. Larger eigenvalues indicate more local information for the associated, appropriately scaled directions; small eigenvalues indicate weakly informed combinations. In this example the mean and covariance blocks can be examined separately. The covariance block has trace $97/36$ and determinant $545/324$, so its eigenvalues are

\lambda_{\pm} =\frac{97\pm\sqrt{689}}{72} \approx 1.7118,\ 0.9827.

Both covariance directions have positive information, and the smaller eigenvalue is not close to zero relative to the larger one. The mean block has eigenvalues approximately $3.240$ and $0.05144$. Its small eigenvalue agrees with the design structure: the two repeated $x=1$ observations measure $\beta_0+\beta_1$, while the single $x=0.5$ observation only partially separates the two mean parameters.

Eigenvalues depend on parameter units and scaling. They are most directly interpretable after parameters have been scaled to meaningful comparable units, or within a parameter block whose scaling is understood. In particular, do not compare a mean-block eigenvalue to a covariance-block eigenvalue as if their raw magnitudes alone ranked scientific importance. The earlier material on scaling parameters before information diagnostics addresses this issue.

Assumptions, Diagnostics, and Alternatives

Check the marginal model.
Confirm that the observation vector has the stated mean and positive-definite covariance at the nominal values. If using a first-order approximation to a nonlinear model, inspect whether local mean and covariance approximations are adequate over the design’s relevant range.

Check derivatives and dimensions.
Each mean derivative must have one entry per observation; each covariance derivative must have the same dimensions as $V_i$. Verify derivatives numerically with small parameter perturbations when analytic derivatives are not available.

Check each subject matrix.
Confirm symmetry and positive semidefiniteness up to numerical tolerance. A negative eigenvalue beyond rounding error can indicate a derivative, inversion, or coding error.

Check the cohort sum.
Multiply each subject-type contribution by its planned count, then add. Check selected entries independently and confirm the final matrix is symmetric before interpreting eigenvalues.

A frequent mistake is to treat observations as independent because subjects are independent. Independence across subjects justifies adding subject matrices; it does not justify replacing a subject’s joint $V_i$ with a diagonal matrix. That replacement discards within-subject covariance and generally changes both mean and covariance information. The earlier material on the independence error in population information explains why this can produce misleading design comparisons.

Another common error is to sum a subject-type matrix without multiplying by the number of subjects of that type. The correction is mechanical but consequential: for instance, the cohort $qq$ entry here is $2(2/3)+1/8=35/24$, not the single-subject value or a value obtained by multiplying only one block. A final check should recompute several entries directly from the stated counts.

This local matrix is not a guarantee that an estimator will attain its information-based precision in a finite sample. It relies on the chosen nominal parameters and model; misspecification, weak identification, boundary variance parameters, or a poor first-order approximation can undermine its interpretation. A first-order calculation is often practical for comparing designs, while exact marginal likelihood information or FOCE-based calculations may be preferable when the approximation is inadequate. These alternatives answer related questions using different approximations; they do not remove the need to specify the observation schedule and covariance structure correctly.

Key takeaway: Build information subject by subject from both mean sensitivities and covariance derivatives, use the full within-subject covariance, and sum each contribution according to the cohort allocation. Interpret eigenvalues only after checking the model, matrix arithmetic, and parameter scaling.

Check Your Understanding

Answer the questions, then check the concise solutions below.

Why does the cohort information equal the sum of subject-level information matrices in this example?
For the two-observation subjects, what is the cohort mean-block intercept-intercept entry?
What is the cohort covariance-block $q,r$ entry, and which contributions make it up?
What does the small eigenvalue of the mean block suggest about the planned mean sensitivities?
Why should raw eigenvalues from differently scaled parameter blocks not be compared directly?

Solutions. (1) Subjects are independent, so their expected log-likelihood information adds. (2) There are two such subjects, each contributing $2/3$, and the single-observation subject contributes $1/2$: $2(2/3)+1/2=11/6$. (3) The two two-observation subjects contribute $2(7/144)=7/72$, and the single-observation subject contributes $1/4$, for $7/72+18/72=25/72$. (4) The mean sensitivities do not strongly distinguish the two parameters; repeated observations at $x=1$ measure their sum, and the one observation at $x=0.5$ only partly resolves the separate effects. (5) Eigenvalues change when parameter units change, so magnitudes across differently scaled parameters do not have a unit-free interpretation.