Marginal Fisher Information in a Random-Intercept Model

Why Marginal Information Is Different

In a nonlinear mixed-effects design calculation, information depends on what is treated as observed. A subject-specific random effect is latent: it is not another measured outcome. If the likelihood is formed by integrating that effect out, observations from the same subject are correlated in the marginal model. The resulting Fisher information therefore need not equal information calculated as if the random effects were known.

This tutorial derives that marginal information for a Gaussian random-intercept model. The model is deliberately simple enough to calculate by hand, while retaining the central feature of mixed-effects information: integration changes the covariance structure and hence the information available from a profile.

Model and Assumptions

The fixed effects are collected in \(\beta\). The vector \(\mathbf{1}_{m_i}\) has \(m_i\) entries, each equal to one. The model assumes a common random-intercept variance and residual variance across subjects, independent subjects, and a correctly specified linear mean \(X_i\beta\). The design matrices are treated as fixed. These assumptions make the marginal distribution and its information available in closed form.

Integrating over \(b_i\), rather than conditioning on its unobserved value, gives

Thus two different observations on the same subject have covariance \(\tau^2\), while each observation has variance \(\tau^2+\sigma^2\). Profiles from different subjects are independent. Write \(V_{i,a}\) for the derivative of \(V_i\) with respect to a variance parameter \(a\). Here \(V_{i,\tau^2}=\mathbf{1}\mathbf{1}^{T}\) and \(V_{i,\sigma^2}=I\).

Marginal Fisher Information

For a parameter vector, Fisher information is the expected outer product of the log-likelihood score, equivalently the negative expected Hessian when the usual regularity conditions hold. For the marginal Gaussian likelihood, the information for the fixed effects and variance components is

Here \(a\) and \(b\) range over \(\tau^2\) and \(\sigma^2\). The expected cross-information between \(\beta\) and either variance parameter is zero in this Gaussian model. These are expected-information results, evaluated at specified parameter values; they are not claims that the realized likelihood or observed information is block diagonal.

A short derivation makes the role of marginalization explicit. For one subject, put \(r_i=y_i-X_i\beta\). The score for the fixed effects is \(X_i^TV_i^{-1}r_i\). The score for a variance parameter \(a\) is

Since \(E(r_i)=0\) and \(E(r_ir_i^T)=V_i\), the covariance of the fixed-effect score is \(X_i^TV_i^{-1}X_i\). For the variance scores, the Gaussian quadratic-form identity gives covariance \(\frac{1}{2}\operatorname{tr}(V_i^{-1}V_{i,a}V_i^{-1}V_{i,b})\). The cross-covariance is zero because it involves a centered third moment of a multivariate normal vector, and all such third moments are zero. Independent subjects contribute additively, producing the sums above.

The expression for \(\mathcal{I}_{\beta\beta}\) also shows directly how the integrated random intercept matters: it is \(V_i^{-1}\), not the residual-only matrix \((\sigma^2 I)^{-1}\), that weights the design. Information for a variance component depends on how changing that component changes the entire marginal covariance matrix.

Using the Result Carefully

Marginal information is useful when asking how a proposed profile informs population parameters under a specified random-effects model. It incorporates both the within-subject correlation and the loss of independent information that comes from a shared latent intercept. For unequal profile sizes or sampling schedules, calculate each subject's matrix with its own \(X_i\) and \(V_i\), then add the contributions. The calculation is local: it depends on the variance values used to construct \(V_i\).

Several checks can reveal a faulty calculation or an unsuitable model. Each \(V_i\) must be positive definite for the inverse and regular Gaussian information formula to apply; \(\sigma^2>0\) ensures this here, including when \(\tau^2=0\). The resulting information matrix should be symmetric and positive semidefinite. If variance-component information is singular or nearly singular, some combination of components may be weakly distinguished by the proposed replication pattern. A singular matrix is not fixed simply by collecting more subjects on a schedule that preserves the same lack of separation.

The standard regular information interpretation is simplest at interior parameter values \(\tau^2>0\) and \(\sigma^2>0\). At the boundary \(\tau^2=0\), variance-component inference can have nonstandard behavior, so ordinary interior asymptotic approximations require caution. The Gaussian result also relies on the assumed residual covariance and distribution. If residuals are correlated, heteroscedastic, or non-Gaussian, the marginal covariance and possibly the information expression must be changed.

A common mistake is to calculate \(X_i^T(\sigma^2I)^{-1}X_i\) and call it marginal fixed-effect information. That calculation conditions on the random intercept as though it were known; it omits the extra shared variation and correlation induced by the latent effect. The correction is to integrate over the random intercept, form \(V_i=\tau^2\mathbf{1}\mathbf{1}^T+\sigma^2I\), and use \(X_i^TV_i^{-1}X_i\). Another alternative is observed information from a fitted marginal likelihood, which reflects the particular data set; it is not interchangeable with the expected information used here for design calculations.

The central distinction is therefore between information conditional on subject effects and information in the marginal distribution of observed profiles. For population design, the latter matches the likelihood in which those effects are unobserved. Its matrix structure reveals not only how precisely fixed effects may be estimated, but also whether the planned replication can distinguish sources of variability.