Worked Example With a Random-Intercept Model

From a Random-Intercept Model to Population Information

Repeated measurements on one subject share features that measurements from different subjects do not. A random-intercept model represents this dependence by giving each subject a shared, unobserved deviation from the population mean. Deriving the resulting covariance is essential: if the shared deviation is ignored, repeated observations can appear to contain more independent information than they do.

This tutorial works through the model’s marginal mean and covariance and then derives a local population Fisher information matrix for a fixed intercept and two variance components. The calculation is evaluated at specified nominal parameter values, as is customary in local design. Because the model here is linear and Gaussian, its marginal moments and Gaussian distribution are exact under the stated assumptions; the matrix is therefore also the marginal Gaussian Fisher information for this particular model.

Here \(Y_{ij}\) is one measured outcome. The model has no occasion-specific fixed effect: conditional on a subject’s random intercept, its expected outcome is the same at every occasion. The design is described by the number of occasions \(m\), and the model parameters are \(\theta=(\beta,\omega^2,\sigma^2)\). Use \(\omega^2\) and \(\sigma^2\), rather than their square roots, as the variance-component parameters in the information matrix.

Marginal Mean and Covariance

Write the \(m\) measurements from one subject as the vector \(Y_i=(Y_{i1},\ldots,Y_{im})^\mathsf{T}\), and let \(\mathbf{1}\) be the \(m\)-vector of ones. Then \(Y_i=\beta\mathbf{1}+b_i\mathbf{1}+\epsilon_i\), where \(\epsilon_i\) is the vector of residuals. Taking expectations over both sources of variation gives the marginal mean. For the covariance, independence of \(b_i\) and \(\epsilon_i\) makes their covariance contribution zero, while the shared random intercept contributes to every pair of occasions:

Thus \(Y_i\sim N(\beta\mathbf{1},V)\), where \(V=\omega^2\mathbf{1}\mathbf{1}^\mathsf{T}+\sigma^2 I_m\). Each measurement has marginal variance \(\omega^2+\sigma^2\), and any two distinct measurements on the same subject have covariance \(\omega^2\). Their marginal correlation is \(\omega^2/(\omega^2+\sigma^2)\). This is the compound-symmetry covariance structure: all measurements have the same variance and all distinct within-subject pairs have the same covariance.

Deriving the Fisher Information

For one subject, write the Gaussian log-likelihood, omitting a constant that does not depend on \(\theta\), as \(-\tfrac12\log|V|-\tfrac12(Y_i-\mu)^\mathsf{T}V^{-1}(Y_i-\mu)\), where \(\mu=\beta\mathbf{1}\). For a multivariate normal model with differentiable mean \(\mu\) and covariance \(V\), the expected Fisher information for parameters \(\theta_a,\theta_b\) is

This expression follows by differentiating the Gaussian log-likelihood and taking the expected product of its scores. The linear score terms in \(Y_i-\mu\) contribute the mean term. The centered quadratic terms contribute the trace term, using the covariance of quadratic forms in a multivariate normal vector. The expected cross-product of a linear term and a centered quadratic term is zero because centered Gaussian variables have zero third moments. These facts also explain why the fixed-mean and covariance parameters have zero expected cross-information in this model.

In this case, \(\partial\mu/\partial\beta=\mathbf{1}\), while the mean does not depend on either variance component. Also, \(\partial V/\partial\omega^2=\mathbf{1}\mathbf{1}^\mathsf{T}\) and \(\partial V/\partial\sigma^2=I_m\). Define \(\lambda=\sigma^2+m\omega^2\). The covariance matrix has eigenvalue \(\lambda\) in the direction \(\mathbf{1}\) and eigenvalue \(\sigma^2\) in each of the \(m-1\) orthogonal contrast directions. Consequently, \(\mathbf{1}^\mathsf{T}V^{-1}\mathbf{1}=m/\lambda\); the trace calculation for the variance components uses one contribution in the mean direction and \(m-1\) in the contrast directions. Combining these results gives the one-subject matrix:

The row and column order is \((\beta,\omega^2,\sigma^2)\). The fixed-intercept information is \(m/\lambda\). For \(N\) independent subjects with the same design and nominal parameters, the population matrix is \(N I(\theta)\), because independent subjects’ log-likelihoods and their expected information add. With differing numbers of occasions, calculate each subject-design contribution using its own \(m\) and add the contributions.

Interpreting the Matrix and Checking the Model

The variance-component block captures how the measurement schedule distinguishes shared between-subject variation from occasion-level residual variation. With only one measurement per subject, \(m=1\), the two variance components enter the marginal variance only through their sum. Indeed, the variance-component block then has identical rows and columns and is rank deficient. There is no information in this design for separating \(\omega^2\) from \(\sigma^2\). At least two repeated measurements are needed to distinguish the covariance of distinct measurements from their additional residual variance.

The formula also makes the role of the model assumptions visible. The model implies equal marginal variances across occasions and equal within-subject covariances. It further implies that measurements from different subjects are independent. Before relying on its information matrix, check whether the scientific setting and data support these features. Strong occasion trends in the mean, changing residual variance, or a pattern of within-subject covariance that is not roughly constant can indicate that the random-intercept model is inadequate. A negative estimated random-intercept variance is not a valid value under the model; it may signal weak information, model mismatch, or estimation difficulty rather than a meaningful negative variance.

A useful diagnostic is to compare observed within-subject patterns with the model’s implied structure: the same-subject covariance should be approximately level across distinct occasion pairs, while between-subject covariance is zero under the stated independence assumption. Also inspect whether the fitted variance components are near a boundary and whether the information matrix is poorly conditioned. A singular or nearly singular matrix indicates that some parameter combinations are not well distinguished by the design and nominal values.

The main alternative simplifications each lose something. Treating every measurement as independent discards the shared intercept and generally overstates information from repeated observations. Reducing each subject to a single average avoids pretending that occasions are independent, but discards the contrast information that helps distinguish \(\omega^2\) from \(\sigma^2\). The random-intercept model retains both the subject-level average and the within-subject contrasts. It is still limited: it cannot represent changing subject effects over time, unequal within-subject covariances, or dependence between subjects. The model and its nominal parameter values should therefore be treated as design assumptions to assess, not as guaranteed descriptions of every repeated-measure setting.

Common Pitfall and Practical Use

For design use, evaluate the matrix at nominal values of \(\omega^2\) and \(\sigma^2\), confirm that the planned number of repeated measurements matches the assumed schedule, and scale or sum information across subjects as appropriate. This produces a local information calculation: changing the nominal variance components can change the matrix and the apparent precision. The calculation is most useful when its covariance assumptions are credible and the design supplies enough repeated measurements to separate the variance components.