Block Structure of the Population Fisher Matrix

Why Partition the Population Fisher Matrix?

A population Fisher information matrix can be large: fixed effects, random-effect covariance parameters, and residual-error parameters may all be estimated from the same subject-level observations. Partitioning the matrix makes it possible to ask a focused question: how does the expected information vary along directions belonging to one parameter group? This is useful for organizing calculations and checking whether a design is informative about a particular group.

The partition does not, by itself, isolate the uncertainty of one group when the other groups are also unknown. A diagonal block describes information in the score directions for that group; the rest of the matrix still matters to joint estimation. Keeping this distinction clear is central to interpreting the blocks correctly.

Definition: Write the population parameter vector as \(\theta=(\beta^\mathsf{T},\omega^\mathsf{T},\lambda^\mathsf{T})^\mathsf{T}\), where \(\beta\) contains fixed effects, \(\omega\) contains random-effect covariance parameters, and \(\lambda\) contains residual-error parameters. The population Fisher matrix, partitioned in that order, has diagonal blocks \(I_{\beta\beta}\), \(I_{\omega\omega}\), and \(I_{\lambda\lambda}\). Each is the principal submatrix for its parameter group: it contains the rows and columns of the full matrix indexed by that group.

For a common working model, the observations from subject \(i\), collected in \(Y_i\), have an approximate multivariate normal distribution \(Y_i\sim N(m_i(\theta),V_i(\theta))\). Here \(m_i\) is the subject-level population mean vector and \(V_i\) is its covariance matrix. The formula below assumes independent subjects, differentiable mean and covariance functions, positive-definite \(V_i\), and regularity conditions that allow differentiation under the expectation. The population Fisher information is the sum of the subject-level expected information matrices.

I_{ab} = \sum_i \left[ \left(\frac{\partial m_i}{\partial a}\right)^\mathsf{T} V_i^{-1} \left(\frac{\partial m_i}{\partial b}\right) + \frac{1}{2}\operatorname{tr} \left( V_i^{-1}\frac{\partial V_i}{\partial a} V_i^{-1}\frac{\partial V_i}{\partial b} \right) \right].

In this expression, \(a\) and \(b\) can denote individual scalar parameters or parameter groups; group derivatives collect the corresponding derivatives into matrices. The first term is information associated with changes in the mean, and the second with changes in the covariance. Either term can contribute to a diagonal block if the relevant group changes that part of the distribution. In many models fixed effects affect the mean but not the covariance, but this is a model property, not a universal rule.

What a Diagonal Block Means

Let \(s_\beta\), \(s_\omega\), and \(s_\lambda\) denote the corresponding components of the score vector, the gradient of the log likelihood. Under the usual regularity conditions, Fisher information is the expected outer product of the score. Consequently,

I_{\beta\beta}=E(s_\beta s_\beta^\mathsf{T}),\qquad I_{\omega\omega}=E(s_\omega s_\omega^\mathsf{T}),\qquad I_{\lambda\lambda}=E(s_\lambda s_\lambda^\mathsf{T}).

Thus a diagonal block describes the expected variation of the score components for that group. Equivalently, it describes curvature of the expected log likelihood along parameter changes in that group while the other parameter coordinates are held fixed. A direction with a larger information quadratic form generally produces a sharper expected likelihood change in that direction.

For example, for a fixed-effect direction \(u\), the scalar \(u^\mathsf{T}I_{\beta\beta}u\) is information in that direction. It is not automatically the information available about that direction after accounting for unknown covariance and residual parameters. That joint-estimation question depends on the full matrix, including its other blocks.

Deriving the Directional Interpretation

The score representation gives a direct justification. For any vector \(u\) of the appropriate dimension, the quadratic form in the fixed-effect block is

u^\mathsf{T}I_{\beta\beta}u = u^\mathsf{T}E(s_\beta s_\beta^\mathsf{T})u = E\left[(u^\mathsf{T}s_\beta)^2\right] \geq 0.

The same argument applies to the \(\omega\) and \(\lambda\) blocks. Each diagonal block is therefore positive semidefinite under the stated regularity conditions. A zero value in a direction means the corresponding score projection has zero variance under the model; that direction has no local Fisher information in the block. A positive value indicates nonzero local information, but does not alone establish that the full parameter vector is estimable with useful precision.

Worked Example: Reading the Blocks of a Partitioned Matrix

Suppose a hypothetical design gives the following Fisher matrix at a specified parameter value. The first two coordinates are fixed effects \((\beta_1,\beta_2)\), the third is a random-effect variance parameter \(\omega_1\), and the last two are residual-error parameters \((\lambda_1,\lambda_2)\). The numbers are illustrative information entries, not results from a real study.

I= \begin{pmatrix} 12 & 2 & 0.6 & 0.2 & 0.1\\ 2 & 9 & 0.3 & 0.1 & 0.2\\ 0.6 & 0.3 & 4 & 0.4 & 0.2\\ 0.2 & 0.1 & 0.4 & 6 & 0.5\\ 0.1 & 0.2 & 0.2 & 0.5 & 3 \end{pmatrix}.

The fixed-effect block is the submatrix in rows and columns 1–2:

I_{\beta\beta}= \begin{pmatrix}12&2\\2&9\end{pmatrix}.

For the direction \(u=(1,0)^\mathsf{T}\), \(u^\mathsf{T}I_{\beta\beta}u=12\) information units for that coordinate direction. For \(u=(1,1)^\mathsf{T}\), the value is \(12+2+2+9=25\). The random-effect covariance block is the single entry \(I_{\omega\omega}=(4)\). The residual-error block is

I_{\lambda\lambda}= \begin{pmatrix}6&0.5\\0.5&3\end{pmatrix}.

This residual block has diagonal entries 6 and 3 and an internal off-diagonal entry 0.5. The off-diagonal entry is inside the residual-error group; it does not turn the block into a cross-group block. The nonzero entries elsewhere in the full matrix show that the parameter groups are not separated in the joint information matrix. The three extracted blocks summarize their own score components, not the full joint estimation problem.

How Mean and Covariance Derivatives Shape the Blocks

The Gaussian information formula clarifies why different parameter groups can have different-looking diagonal blocks. For a fixed-effect block, changes in \(m_i\) often provide its main contribution. For random-effect covariance and residual-error blocks, changes in \(V_i\) are often central. However, if a parameter group changes both the mean and covariance, both terms in the formula contribute. Model structure, not the group label alone, determines which derivatives are zero.

A diagonal block is also parameterization-dependent. If a parameter group is re-expressed through a smooth transformation, the corresponding derivatives and information entries change with that coordinate system. Comparisons of numerical block entries across parameterizations therefore require care: entries have units tied to the chosen parameters, and a rescaling changes their numerical values. Interpret a block in the parameterization actually used for estimation and design.

Worked Example: A Fixed-Effect Block From Mean Sensitivity

Consider a hypothetical population model with \(n=10\) independent subjects. Each subject has one observation with \(Y_i\sim N(\beta,1)\), where the residual variance is known to be \(1\). The only unknown parameter is the fixed effect \(\beta\), measured in concentration units. Here \(m_i=\beta\), \(V_i=1\), \(\partial m_i/\partial\beta=1\), and \(\partial V_i/\partial\beta=0\). The fixed-effect information is therefore

I_{\beta\beta} = \sum_{i=1}^{10} \left[ 1\cdot 1^{-1}\cdot 1 +\frac{1}{2}\operatorname{tr}(1^{-1}\cdot0\cdot1^{-1}\cdot0) \right] =10.

Each subject contributes 1 information unit per squared concentration unit, so the total is 10. The covariance term is zero because \(\beta\) does not change \(V_i\). This calculation is valid for the stated model: subjects are independent, the variance is positive, and the mean is differentiable in \(\beta\). If the residual variance were also estimated, this same diagonal entry would still describe the fixed-effect score component, but the full matrix would include additional coordinates and could have nonzero cross-group entries.

Worked Example: A Residual-Error Block From Variance Derivatives

Suppose, hypothetically, that each of \(n=12\) independent subjects contributes two independent measurements with distribution \(N(0,V)\), where \(V=\operatorname{diag}(\lambda_1,\lambda_2)\). The two residual variances are \(\lambda_1=0.25\) and \(\lambda_2=1.00\), in squared concentration units. The derivative matrices are \(\partial V/\partial\lambda_1=\operatorname{diag}(1,0)\) and \(\partial V/\partial\lambda_2=\operatorname{diag}(0,1)\). The mean is fixed and does not contribute to this block.

For a normal covariance parameter, the information entry for one subject is one half the trace of the product of the two covariance derivatives with \(V^{-1}\) between them. The two diagonal entries, summed across subjects, are

I_{\lambda_1\lambda_1} = 12\left[\frac{1}{2}\left(\frac{1}{0.25}\right)^2\right] =12(8)=96, \qquad I_{\lambda_2\lambda_2} = 12\left[\frac{1}{2}\left(\frac{1}{1.00}\right)^2\right] =12(0.5)=6.

The mixed derivative product has zero trace because the two derivative matrices act on different diagonal elements, so \(I_{\lambda_1\lambda_2}=0\). Thus \(I_{\lambda\lambda}=\operatorname{diag}(96,6)\) at these values. The units are reciprocal squared variance units. The larger first entry reflects the local sensitivity of this normal model to changes in \(\lambda_1\) at its specified value; it is not a unit-free comparison of the practical precision of two differently scaled parameters.

Common Misinterpretations and Diagnostics

Calling the full matrix block diagonal. Partitioning only arranges the coordinates. It does not imply that entries connecting different parameter groups are zero.
Treating a diagonal block as adjusted information. The block is the principal submatrix for that group, not a summary that has already accounted for estimation of the other groups.
Assuming the group label determines the derivative term. Fixed effects can affect covariance, and covariance parameters can affect the mean in some models. Check the derivatives in the specified model.
Comparing raw entries across scales. Information entries depend on parameter units and parameterization. A large number can result from the scale used, rather than a meaningful design advantage.

Practical checks include confirming that the blocks have dimensions matching the number of parameters in each group, that each computed diagonal block is symmetric, and that its eigenvalues are nonnegative up to numerical tolerance. A materially negative eigenvalue can indicate an implementation error, an invalid approximation, or numerical instability. Also verify that the parameter ordering used to extract the blocks matches the ordering used in the likelihood code. These checks do not establish that a block is well-conditioned or that the joint model is identifiable; they are safeguards against basic structural mistakes.

What the Blocks Can and Cannot Establish

The main alternative to blockwise inspection is to examine the full Fisher matrix directly. The full matrix is necessary for joint uncertainty and for questions about how parameter groups interact. A diagonal block is more focused and easier to interpret as expected score variation within one group, but it deliberately leaves the other groups’ score components out of that summary. Likewise, reporting only scalar diagonal entries discards information about directions combining parameters within the same group.

The block interpretation relies on the assumed likelihood or approximation. If the Gaussian population approximation is poor, its Fisher matrix may not faithfully represent the information under the actual model. Information is also local: it is evaluated at parameter values and a design, and may change as either changes. Near a boundary, such as a variance approaching zero, regularity conditions may fail and standard local interpretations can become unreliable.

In design work, the diagonal blocks are useful for organizing parameter-specific calculations and spotting weakly informed directions within groups. They are not, by themselves, a complete ranking of designs for joint estimation. For that purpose, the full matrix and the intended estimation objective must be considered.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions below.

If \(\omega\) contains three random-effect covariance parameters, what are the dimensions of \(I_{\omega\omega}\)?
Why does \(u^\mathsf{T}I_{\beta\beta}u\geq0\) hold for any fixed-effect direction \(u\), under regularity conditions?
In the residual-error example, what would \(I_{\lambda_1\lambda_1}\) be for 6 subjects instead of 12, with all else unchanged?
Does a large diagonal entry in \(I_{\beta\beta}\) prove that the corresponding fixed effect can be estimated precisely when covariance parameters are also unknown? Explain.

Solutions. (1) It is \(3\times3\), since it has one row and column for each random-effect covariance parameter. (2) It equals \(E[(u^\mathsf{T}s_\beta)^2]\), an expected square, so it cannot be negative. (3) The information is proportional to the number of independent subjects: \(6[\frac{1}{2}(1/0.25)^2]=6(8)=48\). (4) No. The diagonal block describes information in the fixed-effect score directions, but joint precision also depends on the full matrix and its other parameter groups.