How Random-Effect Variability Changes Population Information

Why random-effect variability matters for design

A population design can contain many subjects and still provide limited precision for a typical value if subjects differ substantially from one another. The reason is that individual observations vary not only because of residual error, but also because each subject has a latent deviation from the population typical value. The approximate population Fisher matrix makes this effect explicit: random-effect variance contributes to the covariance of the observations, and that covariance determines how much information the data carry about typical values.

There is a second, distinct question: how much information do the data carry about the random-effect variance itself? A larger variance changes the sensitivity of the observation distribution to that parameter, but its Fisher information depends on the scale used to represent the parameter. For example, information for a variance and information for its logarithm are not interchangeable. The direction of change must therefore be stated for a specified parameterization and model.

The approximate population Fisher formula

For subject \(i\), let \(y_i\) be the vector of observations at the subject's specified measurement times and covariate values. Let \(\theta\) denote population typical-value parameters, and let \(\eta_i\) be the subject's random effects, with mean zero and covariance matrix \(\Omega\). A first-order approximation linearizes the subject-specific mean response around \(\eta_i=0\):

Here \(f_i(\theta)\) is the response at the typical values, \(J_i\) is the derivative of the response with respect to the random effects, \(\epsilon_i\) is residual error with covariance \(R_i\), and \(V_i\) is the resulting approximate marginal covariance of \(y_i\). Under this approximation, the observations for a subject are treated as multivariate normal. Subjects are assumed independent, the relevant covariance matrices are nonsingular, and the design and nominal parameter values are fixed for the calculation.

Write \(\psi\) for the vector of population parameters being considered. It can include typical values and parameters describing \(\Omega\) or \(R_i\). Let \(m_i\) be the approximate mean of \(y_i\), and use subscripts to denote derivatives with respect to a parameter. The approximate expected Fisher information contributed by subject \(i\) is:

The first term is information carried by changes in the mean; the second is information carried by changes in the covariance. The total population information is the sum of the subject contributions, \(\mathcal{I}=\sum_i\mathcal{I}_i\). This is a local approximation: its value depends on the nominal parameters used to calculate the derivatives and covariance.

Why larger covariance reduces mean information

Suppose a scalar typical-value parameter \(\theta\) affects the approximate mean through a derivative vector \(d\), while the covariance is fixed with respect to \(\theta\). Its information is \(d^\mathsf{T}V^{-1}d\). If increasing a random-effect variance changes \(V\) to \(V+A\), where \(A\) is positive semidefinite, then the new covariance is no smaller in any direction. For positive definite \(V\), the inverse reverses this ordering:

Thus, under these conditions, added covariance cannot increase mean information for that fixed sensitivity direction. The inequality can be equality if the added variability does not affect the direction that informs the parameter. This result concerns the mean-information term with a fixed derivative and a covariance increase; it is not a general claim that every entry or every parameter's information must decline whenever any random-effect variance increases.

A short derivation of the Fisher formula also clarifies its two parts. For a multivariate normal observation with mean \(m\) and covariance \(V\), the score for parameter \(\psi_a\) has a mean component \(m_{,a}^{\mathsf{T}}V^{-1}(y-m)\) and a covariance component formed from the centered quadratic quantity in \(y-m\). Taking expected products of scores gives the mean term \(m_{,a}^{\mathsf{T}}V^{-1}m_{,b}\) and the covariance term \(\tfrac12\operatorname{tr}(V^{-1}V_{,a}V^{-1}V_{,b})\). The cross-product has expectation zero because centered normal observations have zero third central moments. The result assumes the normal approximation and differentiable mean and covariance functions; in a nonlinear mixed-effects model it is approximate because the response has been linearized in the random effects.

Repeated measurements and shared random effects

With repeated measurements, a random effect can induce covariance among observations from the same subject. The information then depends on the full covariance matrix, not only on a single marginal variance. In a random-intercept model, repeated observations share the same subject-specific deviation. This shared component does not disappear by taking more measurements from that subject; its effect on population information differs from that of independent measurement noise.

Which typical values are affected?

When random effects affect different parts of a response differently, increasing one component of \(\Omega\) can primarily reduce information about the typical-value contrast aligned with that component. For example, random-intercept variability most directly affects level information, whereas random-slope variability affects information about change across a covariate or time. Whether other parameter directions are also affected depends on the design and on correlations among random effects.

Reading the result, checking assumptions, and avoiding overclaims

These examples illustrate useful patterns, not a universal monotonicity rule for every element of a population Fisher matrix. Before interpreting a change, check which parameter is varied, which parameter is the target, whether the response sensitivities change with the nominal values, and whether random-effect covariance or correlations also change. If \(J_i\) depends on the parameters, increasing a variance may change both \(V_i\) and its derivatives. If the parameterization changes from a variance to a log-variance, apply the chain rule before comparing information.

A practical diagnostic is to calculate the information at more than one plausible nominal variance, holding the design and all other nominal values fixed. Confirm that the covariance remains positive definite and that the derivative used for the variability parameter matches the parameterization. Check whether repeated observations were represented with their induced covariance. If small changes to the nominal variance cause implausibly large changes in the approximate information, the local linearization may be inadequate or the model may be near a boundary such as zero variance. In those cases, the approximation should be treated cautiously rather than as a reliable ranking of designs.

The Fisher calculation is a local, model-based measure of expected curvature; it does not establish that the fitted model is correct or that an estimator will behave well in a small sample. A larger random-effect variance can make typical values harder to estimate while making a relative variability parameter more informative under some parameterizations. Conversely, information for an absolute variance can decline as the variance grows, as in the examples. Interpret each statement with its target parameter and units.

The main alternative to this approximate calculation is to use a less approximated likelihood-based information calculation, for example one that evaluates the marginal likelihood without linearizing the response in the random effects. That can be more faithful to a nonlinear model, but may require substantially more computation and still depends on the assumed model. The first-order Fisher formula is useful for understanding local mechanisms and comparing candidate designs efficiently; it should not be mistaken for an exact, model-free measure of information.