Worked Example With Random Intercept and Slope

Why Observation Times Matter

A random intercept allows subjects to differ in their baseline level. A random slope also allows their response to change differently over time or another design variable. The two random effects may be correlated: subjects with higher baselines might, for example, tend to have steeper or shallower slopes. A design therefore needs to do more than collect repeated measurements. Its observation times must make the variance and covariance components distinguishable.

The focus here is the approximate population Fisher information for a linear random-intercept-and-slope model. It is an expected, local measure of how much a design can inform the model parameters at specified nominal parameter values. In the Gaussian linear model below, the marginal distribution is exact; using that distribution for a more general nonlinear mixed model is the first-order approximation.

Here \(\omega_{00}\) is the random-intercept variance, \(\omega_{11}\) is the random-slope variance, and \(\omega_{01}\) is their covariance. If both variances are positive, their correlation is \(\rho=\omega_{01}/\sqrt{\omega_{00}\omega_{11}}\). The covariance parameterization is convenient for constructing the Fisher matrix; the correlation parameterization is a transformation of it.

Marginal Covariance and Its Derivatives

For one subject, collect the measurements in a vector \(Y_i\), and let \(Z_i\) have rows \((1,t_{ij})\). Let \(X_i\) have rows \((1,t_{ij})\) for the fixed linear mean. Then \(Y_i\) is marginally multivariate normal with mean \(X_i\beta\) and covariance \(V_i=Z_i\Omega Z_i^\mathsf{T}+\sigma^2 I\). In elementwise form,

This expression makes the role of time explicit. The random-intercept–slope covariance contributes in proportion to the sum of the two observation times. In particular, it contributes \(2\omega_{01}t_{ij}\) to the variance at time \(t_{ij}\), and \(\omega_{01}(t_{ij}+t_{ik})\) to the covariance between two distinct measurements.

For the parameter vector \(\theta=(\beta_0,\beta_1,\omega_{00},\omega_{01},\omega_{11},\sigma^2)\), the covariance derivatives are

For a multivariate normal vector with mean \(X_i\beta\) and covariance \(V_i\), the expected Fisher information has a mean block \(X_i^\mathsf{T}V_i^{-1}X_i\) and a covariance block with entries \(\frac12\operatorname{tr}(V_i^{-1}V_{i,a}V_i^{-1}V_{i,b})\), where \(V_{i,a}\) denotes the derivative with respect to covariance parameter \(a\). The mean–covariance block is zero under this Gaussian model. Independent subjects contribute additively, so the population information is the sum of the per-subject matrices.

Why the Covariance Information Formula Holds

For one subject, the part of the log likelihood involving covariance is \(-\frac12\log|V|-\frac12(Y-X\beta)^\mathsf{T}V^{-1}(Y-X\beta)\), up to a constant. Differentiating with respect to covariance parameter \(a\), and writing \(r=Y-X\beta\), gives the score

Under the stated model, \(r\sim N(0,V)\). For symmetric matrices \(A\) and \(B\), the covariance of the quadratic forms \(r^\mathsf{T}Ar\) and \(r^\mathsf{T}Br\) is \(2\operatorname{tr}(AVBV)\). Applying this identity with \(A=V^{-1}V_aV^{-1}\) and \(B=V^{-1}V_bV^{-1}\) yields

Thus the trace expression is the expected information, not an observed-data calculation using particular measured outcomes. It is evaluated at specified nominal parameter values. A nonsingular information matrix is necessary for local identification of all included parameters, although good numerical conditioning also matters.

Patterns That Cannot Separate the Components

A design can be informative about some combinations of parameters while failing to distinguish the individual parameters. The derivative matrices give a direct way to see this. If two parameters have proportional covariance derivatives across all subjects, their effects on the marginal covariance cannot be separated by the covariance likelihood alone. More generally, linearly dependent derivatives imply a deficient covariance information block.

Reading a Design for Correlation Information

The covariance parameter \(\omega_{01}\) enters through sums of paired times. For two distinct measurements, its derivative is \(t_{ij}+t_{ik}\); for a measurement's own variance, the derivative is \(2t_{ij}\). Consequently, a collection of observations at varied times can generate a covariance derivative matrix that is distinguishable from the intercept-variance and slope-variance derivatives. A single shared time generally cannot do so when those components are all unknown.

A centered design with observations at \(-a\) and \(a\) has zero covariance-parameter contribution to the off-diagonal element for that pair, because \((-a)+a=0\). This does not mean the covariance is wholly uninformative: the two diagonal derivatives are \(-2a\) and \(2a\). The full matrix, including its interaction with the other covariance derivatives and the residual variance, determines the information.

If \(\omega_{00}\) and \(\omega_{11}\) are positive, the correlation scale is often easier to interpret than the covariance scale. However, its information must be obtained with the appropriate parameter transformation. Near a zero variance, \(\rho\) may be poorly defined, and standard local approximations can be unreliable at the positive-semidefinite boundary. Do not interpret a nonzero diagonal entry for \(\omega_{01}\) as proof that correlation is separately estimable when the variance components are also unknown; inspect the full information matrix.

Assumptions, Diagnostics, and Alternatives

The calculations rely on the stated Gaussian model, independent subjects, a common covariance structure as specified, known design values, and residual variance \(\sigma^2 I\). The fixed mean is linear in the same design variable. For a nonlinear mixed-effects model, the same construction is an approximation only when the response is adequately represented by its local linearization at the nominal parameter values. If residuals are serially correlated, heteroscedastic, or otherwise misspecified, the residual covariance must be changed before the information is computed.

Useful warning signs include a zero or nearly zero eigenvalue in the covariance information block, very small information for \(\omega_{01}\), or near-proportional derivative matrices. Such findings indicate that parameter combinations are weakly distinguished by the proposed times. They are design diagnostics, not proof that the model is scientifically wrong. Conversely, an apparently positive information entry alone is not a sufficient identifiability check.

A random-intercept-only model is a simpler alternative when there is no scientifically justified subject-to-subject slope variation. It avoids estimating the slope variance and covariance but cannot represent individual trajectory differences. Another alternative is to measure at more varied times: that may separate the derivative patterns, but it does not guarantee stable estimation if the times are too close or the nominal variances are near a boundary. Adding repeated observations at one time can improve precision for existing combinations, while changing the time pattern is what can create new distinctions.

Common Pitfall

• Wrong reasoning: “There are many subjects, so the random-intercept variance, slope variance, and correlation must all be estimable.”
• Correction: Subjects observed in the same uninformative pattern repeat the same covariance derivative structure. In the one-time example, more subjects multiply a rank-one information matrix; they do not raise its rank. Assess the derivatives and the full information matrix, not just the sample size.