Separating Random-Effect and Residual Variability

Why Variance Components Can Be Difficult to Separate

A population model may contain both between-subject random-effect variability and within-subject residual variability. These sources describe different mechanisms, but a design can make their effects on the observed data look nearly alike. When that happens, the data may estimate their combined variation reasonably well while estimating either component separately poorly. The central question here is therefore not just how much variation is present, but whether the planned observations make the two variance components leave distinguishable signatures in the covariance matrix.

The diagnostic developed below compares the covariance derivatives for the two components and examines the resulting Fisher information. It is a local, model-based assessment at specified nominal parameter values. It does not replace checks of model adequacy or establish that a model is globally identifiable.

Model and Covariance Derivatives

Definition: Consider subject $i$'s vector of $m$ observations under the Gaussian random-intercept model $$ Y_i=\mu_i+\mathbf{1}_m b_i+\varepsilon_i,\qquad b_i\sim N(0,\tau^2),\qquad \varepsilon_i\sim N(0,\sigma^2 I_m), $$ where $b_i$ and $\varepsilon_i$ are independent, $\mathbf{1}_m$ is a vector of $m$ ones, $\mu_i$ is the mean vector, $\tau^2$ is the random-intercept variance, and $\sigma^2$ is the residual variance. Marginally, $Y_i\sim N(\mu_i,V_i)$, with $V_i=\tau^2\mathbf{1}_m\mathbf{1}_m^\mathsf{T}+\sigma^2 I_m$.

This calculation assumes that the marginal observation vector is Gaussian, that the random effect and residuals are independent, and that the residual variance is common across measurements. For the formulas below, the mean is treated as fixed with respect to $\tau^2$ and $\sigma^2$. Subjects are independent. These assumptions let us focus on whether the covariance patterns alone distinguish the two variance parameters.

Write $D_{\tau^2}=\partial V_i/\partial\tau^2$ and $D_{\sigma^2}=\partial V_i/\partial\sigma^2$. For this model,

D_{\tau^2}=\mathbf{1}_m\mathbf{1}_m^\mathsf{T}, \qquad D_{\sigma^2}=I_m.

The first derivative adds covariance equally to every pair of measurements, including their marginal variances. The second adds variance to each measurement but does not add covariance between distinct measurements. Separation is possible to the extent that these derivative patterns are distinguishable after accounting for the covariance $V_i$ already present in the model.

Fisher Information and a Derivative-Based Diagnostic

For a Gaussian vector whose mean does not depend on covariance parameters, the expected Fisher information between covariance parameters $a$ and $b$ is

\mathcal{I}_{ab} =\frac{1}{2}\operatorname{tr} \left(V_i^{-1}D_aV_i^{-1}D_b\right).

Thus each information entry depends on the covariance derivatives, weighted by the inverse covariance. A useful diagnostic is the normalized cross-information

\rho_{\mathcal{I}} =\frac{\mathcal{I}_{\tau^2,\sigma^2}} {\sqrt{\mathcal{I}_{\tau^2,\tau^2}\mathcal{I}_{\sigma^2,\sigma^2}}}.

For positive variance parameters in this model, values near one indicate that the two derivative patterns are nearly aligned in the Fisher-information geometry. The matrix is then poorly conditioned in the direction that trades one variance against the other. A value of one corresponds to exact linear dependence of the derivative directions and singular information for the pair. This is a local information diagnostic: a high value warns of weak separation at the nominal parameter values, but is not by itself a proof about every possible dataset or parameter value.

Why the Derivative Alignment Matters

The relationship to derivative shapes can be seen directly. Define $A_a=V_i^{-1/2}D_aV_i^{-1/2}$, using the symmetric positive-definite square root of $V_i$. The trace identity gives $\mathcal{I}_{ab}=\tfrac12\operatorname{tr}(A_aA_b)$. Since the matrices are symmetric, this is one half their Frobenius inner product. Consequently, the normalized cross-information is their normalized inner product: it measures the angle between the two covariance-change directions after whitening by $V_i$.

For the random-intercept model, the covariance has one eigenvalue $\sigma^2+m\tau^2$ in the direction $\mathbf{1}_m$, and $m-1$ eigenvalues $\sigma^2$ in directions orthogonal to $\mathbf{1}_m$. The random-intercept derivative acts only in the shared direction; the residual derivative acts in both the shared and orthogonal directions. Substitution into the trace formula gives the per-subject information:

\mathcal{I}_{\tau^2,\tau^2} =\frac{1}{2}\left(\frac{m}{\sigma^2+m\tau^2}\right)^2, \qquad \mathcal{I}_{\tau^2,\sigma^2} =\frac{1}{2}\frac{m}{(\sigma^2+m\tau^2)^2},

\mathcal{I}_{\sigma^2,\sigma^2} =\frac{1}{2}\left[ \frac{m-1}{(\sigma^2)^2} +\frac{1}{(\sigma^2+m\tau^2)^2} \right].

For example, the first information entry follows because $V_i^{-1}D_{\tau^2}$ has eigenvalue $m/(\sigma^2+m\tau^2)$ in the shared direction and zero in the other directions. Its squared trace contribution is therefore $m^2/(\sigma^2+m\tau^2)^2$, multiplied by one half. Independent subjects contribute additively, so $N$ subjects with the same design have total information $N\mathcal{I}$.

Worked Example: One Observation Gives Exact Confounding

Suppose each of 30 hypothetical subjects contributes one observation, with nominal values $\tau^2=4\ \text{units}^2$ and $\sigma^2=9\ \text{units}^2$. Here $m=1$, so the marginal variance of each observation is $\tau^2+\sigma^2=13\ \text{units}^2$. The two covariance derivatives are both the one-by-one matrix $[1]$: changing either variance changes the only observable variance in the same way.

The information per subject is $$ \mathcal{I} =\frac{1}{2(13)^2} \begin{pmatrix}1&1\\1&1\end{pmatrix} = \begin{pmatrix}0.002959&0.002959\\0.002959&0.002959\end{pmatrix} \ \text{units}^{-4}. $$ For 30 independent subjects, the information is $$ 30\mathcal{I} = \begin{pmatrix}0.08876&0.08876\\0.08876&0.08876\end{pmatrix} \ \text{units}^{-4}. $$

The rows are identical and the determinant is zero, so the information has rank one. Increasing the number of such subjects multiplies this matrix but does not make it rank two. The observations can inform the sum $\tau^2+\sigma^2$, but this design cannot separate the two components using the marginal variance alone.

Worked Example: Two Measurements Provide a Separation Direction

Now suppose 50 hypothetical subjects each have two measurements, at the same nominal values $\tau^2=4\ \text{units}^2$ and $\sigma^2=9\ \text{units}^2$. For $m=2$, $\sigma^2+m\tau^2=17\ \text{units}^2$. The per-subject entries are $$ \mathcal{I}_{\tau^2,\tau^2} =\frac12\left(\frac{2}{17}\right)^2=0.006920, \qquad \mathcal{I}_{\tau^2,\sigma^2} =\frac12\frac{2}{17^2}=0.003460, $$ $$ \mathcal{I}_{\sigma^2,\sigma^2} =\frac12\left(\frac{1}{9^2}+\frac{1}{17^2}\right)=0.007903 \quad\text{units}^{-4}. $$

For 50 subjects, the total information matrix is approximately $$ \mathcal{I}_{\mathrm{total}} = \begin{pmatrix} 0.3460&0.1730\\ 0.1730&0.3951 \end{pmatrix} \ \text{units}^{-4}. $$ Its determinant is $0.3460(0.3951)-(0.1730)^2\approx0.1068\ \text{units}^{-8}$, which is positive. The normalized cross-information is $0.003460/\sqrt{0.006920(0.007903)}\approx0.468$. The derivative directions are no longer identical: the contrast between the two measurements has residual variance $\sigma^2$ and cancels the shared random intercept.

Under the local normal approximation based on this Fisher matrix, the marginal standard errors for the two variance estimates are approximately $\sqrt{0.3951/0.1068}=1.923\ \text{units}^2$ for $\tau^2$ and $\sqrt{0.3460/0.1068}=1.800\ \text{units}^2$ for $\sigma^2$. These are local approximations, not guarantees of estimator performance in a small sample or near a parameter boundary.

Worked Example: More Replicates Change the Derivative Geometry

Consider instead four hypothetical measurements per subject at the same nominal variances, $\tau^2=4\ \text{units}^2$ and $\sigma^2=9\ \text{units}^2$. Now $\sigma^2+m\tau^2=25\ \text{units}^2$. The per-subject information entries are $$ \mathcal{I}_{\tau^2,\tau^2} =\frac12\left(\frac{4}{25}\right)^2=0.01280, \qquad \mathcal{I}_{\tau^2,\sigma^2} =\frac12\frac{4}{25^2}=0.003200, $$ $$ \mathcal{I}_{\sigma^2,\sigma^2} =\frac12\left(\frac{3}{9^2}+\frac{1}{25^2}\right) =0.01932 \quad\text{units}^{-4}. $$

The normalized cross-information is $0.003200/\sqrt{0.01280(0.01932)}\approx0.2035$, lower than the approximately $0.468$ for two measurements. The residual derivative now has three orthogonal contrast directions in which it contributes information but the random-intercept derivative does not. This illustrates improved separation per subject under this model. It is not a universal claim that four measurements are optimal: costs, timing, mean-model information, and other design objectives also matter.

Diagnostics, Alternatives, and Limitations

In a proposed design, compute the covariance derivatives at the nominal parameter values, form the relevant Fisher block, and inspect both the derivative patterns and the normalized cross-information. A nearly singular block, large cross-information relative to its diagonal entries, or large uncertainty in the inverse-information approximation warns that variance components may be difficult to distinguish. The derivative comparison helps explain the warning: if two parameter changes produce almost the same weighted covariance change, more observations of the same type may add little separation.

This diagnostic is most interpretable when parameters use a meaningful, stated scale. Reparameterizing a variance as its logarithm changes the derivative by the chain rule and rescales the corresponding information row and column. The rank of a nonsingularly transformed information block is preserved, but raw entries and standard errors depend on parameterization. Compare normalized derivative directions or explicitly state the parameter scale when interpreting magnitudes.

Other approaches answer related but different questions. A likelihood profile can show how the fitted likelihood changes as one variance is fixed across a range, while simulation can assess estimator bias, boundary estimates, and finite-sample uncertainty under a specified design. These can reveal features that a local Fisher calculation misses, but require fitting or simulating the model. The derivative-based Fisher diagnostic is comparatively direct and useful during design evaluation; it is local and depends on the assumed model and nominal values.

The formulas above do not cover every mixed-effects setting. If the mean depends on the variance parameters, its information contribution must also be included. If residual covariance is structured, or random effects have several components, the relevant derivatives and information matrix must be recomputed for that model. A poor separation diagnostic may also indicate model misspecification rather than merely an inadequate design. Finally, positive Fisher information is not proof of global identifiability, and a singular local matrix does not by itself describe every possible source of information in a nonlinear model.

Common Pitfall

Common mistake: “The covariance matrix is positive definite, so the two variance components must be separately estimable.” Positive definiteness of $V_i$ ensures that the Gaussian covariance is valid; it does not ensure that its parameter derivatives are distinct. With one observation, $V_i=[\tau^2+\sigma^2]$ is positive for positive variances, yet both derivatives are $[1]$ and the two-parameter Fisher information is singular. Check the derivative directions and the information block, not just whether $V_i$ is invertible.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

In the random-intercept model, what covariance pattern does $D_{\tau^2}$ represent, and how does it differ from $D_{\sigma^2}$?
Why can adding more subjects with one measurement each fail to separate $\tau^2$ and $\sigma^2$?
For $m=3$, $\tau^2=2\ \text{units}^2$, and $\sigma^2=5\ \text{units}^2$, calculate the per-subject cross-information $\mathcal{I}_{\tau^2,\sigma^2}$.
Does a normalized cross-information of 0.9 prove that the model is globally non-identifiable? Explain.

Solutions. (1) $D_{\tau^2}=\mathbf{1}_m\mathbf{1}_m^\mathsf{T}$ adds covariance to every pair, including the diagonal; $D_{\sigma^2}=I_m$ adds only to the diagonal. (2) With $m=1$, both derivatives equal $[1]$, so each subject informs only $\tau^2+\sigma^2$. Multiplication by the number of subjects cannot increase the rank of that information block. (3) Since $\sigma^2+m\tau^2=5+3(2)=11\ \text{units}^2$, the cross-information is $\tfrac12(3/11^2)=1.5/121\approx0.01240\ \text{units}^{-4}$. (4) No. A value of 0.9 signals strong local alignment and potentially weak separation at the evaluated parameter values; it does not establish global non-identifiability over all parameter values or under all data patterns.