Estimating Variance Components From Replicated Profiles

Why Replication Matters for Variance Components

A repeated profile contains two sources of variation: subjects may differ systematically from one another, and observations may fluctuate within a subject. A design can estimate these sources separately only if it contains enough repeated measurements to distinguish a subject-level shift from observation-level noise. Fisher information makes this distinction quantitative: it describes the local precision available for estimating each variance component under a specified model and design.

Here the focus is a balanced Gaussian random-intercept model. The calculation is deliberately specific: it shows how the number of independent subjects and the number of observations per subject contribute to precision for the between-subject variance and the residual variance. It does not, by itself, establish that the model is adequate for a nonlinear pharmacokinetic profile.

The marginal covariance matrix for one subject's \(m\)-vector is \(V=\tau^2\mathbf{1}\mathbf{1}^{T}+\sigma^2 I_m\). Thus two distinct observations from the same subject have covariance \(\tau^2\), while each observation has variance \(\tau^2+\sigma^2\). Observations from different subjects are independent. This covariance structure is the key link between replication and variance-component information.

Fisher Information for the Two Variances

For a multivariate normal observation with mean \(\mu\mathbf{1}\) and covariance \(V\), the expected Fisher information for covariance parameters \(\theta_a,\theta_b\) is \(\frac{1}{2}\operatorname{tr}(V^{-1}V_aV^{-1}V_b)\), where \(V_a=\partial V/\partial\theta_a\). The covariance parameters here are \(\theta_1=\tau^2\) and \(\theta_2=\sigma^2\). The mean parameter is orthogonal to these covariance parameters in expected Fisher information for this Gaussian model, so estimating the common mean does not add a mean-covariance cross term to this matrix.

The calculation is easiest by separating each subject's profile into its average direction and its within-subject contrasts. Along the average direction, \(V\) has eigenvalue \(\sigma^2+m\tau^2\); each of the \(m-1\) contrast directions has eigenvalue \(\sigma^2\). Write \(A=\sigma^2+m\tau^2\). The eigenvalues show directly that the average contains information about both variance components, while the contrasts contain information about the residual variance.

The terms involving \(A\) arise from the one average direction per subject; the term involving \(m-1\) arises from the within-subject contrasts. The off-diagonal entry is nonzero: the subject average depends on both \(\tau^2\) and \(\sigma^2\), so the two variance estimates are not generally independent.

Derivation of the Joint Precision Bound

For an unbiased estimator under regularity conditions, the Cramér–Rao inequality says its covariance matrix is at least the inverse Fisher information matrix in the positive-semidefinite sense. For a regular maximum-likelihood estimator in a sufficiently informative sample, the inverse information also describes its approximate asymptotic covariance. These are local, model-based results, not guarantees of finite-sample performance.

The determinant calculation shows when the two parameters can be distinguished. Put \(s=\sigma^2\), so \(s^2=\sigma^4\). For one subject, the determinant of the information matrix is

For \(n\) subjects the determinant is multiplied by \(n^2\). It is positive exactly when \(m>1\), given positive variances. Inverting the matrix gives the diagonal entries of the joint-estimation precision bound:

The symbol \(\gtrsim\) indicates an information lower bound for unbiased estimation, or an approximate asymptotic variance for a regular efficient estimator. The corresponding standard errors have units of variance, such as concentration squared—not concentration. These expressions account for estimating both components jointly; treating one component as known would give a different and usually smaller bound.

What Changes When the Design Changes?

For fixed \(m\) and parameter values, the information matrix is proportional to \(n\), so the variance bounds are proportional to \(1/n\), and the standard errors decrease in proportion to \(1/\sqrt{n}\). Increasing \(m\) adds within-subject contrasts, improving information about \(\sigma^2\), and changes the subject-average variance, which informs \(\tau^2\). The effects on the two components are not interchangeable: a new subject contributes a new independent subject-level deviation, whereas another observation on an existing subject does not.

Connection to Replicated-Profile Moment Estimates

The information calculation has a useful counterpart in simple moment estimators. Let \(\bar{Y}_i\) be subject \(i\)'s average, and let \(S_B^2\) be the sample variance of the \(n\) subject averages. The average has variance \(\tau^2+\sigma^2/m\). The pooled within-subject sum of squares estimates residual variation independently of the subject averages under the normal model. Consequently, the moment estimates are the pooled within-subject variance for \(\sigma^2\), and \(S_B^2-\widehat{\sigma^2}/m\) for \(\tau^2\).

The subtraction explains both the strength and the weakness of replication-based variance estimation: the observed spread of subject averages combines between-subject variation with residual noise averaged across \(m\) observations. Under the model, the exact sampling variance of the between-subject moment estimate is

The first term is uncertainty in the sample variance of subject averages; the second is uncertainty in the estimated residual variance used in the subtraction. For large \(n\), this agrees to first order with the inverse-information bound above. The pooled within-subject estimator has variance \(2\sigma^4/[n(m-1)]\), matching its information bound under this model.

Diagnostics, Alternatives, and Limitations

These bounds are most useful as a model-based check on whether a proposed replication pattern contains information for the intended variance components. They should be interpreted alongside checks of the model assumptions and, when possible, simulation from the full planned analysis.

• Check replication and independence. If there are no repeated observations within subjects, the two components are confounded. If subjects are clustered or otherwise dependent, the independent-subject information calculation overstates the effective information.
• Check the covariance pattern. Serial residual correlation, time-varying residual variance, or random slopes change the covariance matrix and hence the information. Applying this random-intercept formula unchanged can give misleading precision.
• Check the Gaussian and interior-parameter conditions. Strong skewness, a variance near zero, or small numbers of subjects can make asymptotic approximations unreliable. Variance estimates can also hit the zero boundary, where ordinary interior-point approximations do not apply.
• Separate model-based precision from actual performance. Fisher information uses the assumed parameter values and model. A poor parameter guess or structural misspecification can make the resulting precision assessment unrepresentative.

Moment estimators are transparent and show directly how within-subject and between-subject variation enter, but subtraction can produce a negative estimate of \(\tau^2\) in a finite sample even though a variance cannot be negative. Constrained likelihood estimation avoids reporting an inadmissible negative variance, while restricted maximum likelihood (REML) accounts for estimation of fixed effects when estimating covariance parameters. Neither alternative removes the need for a sound covariance model. Parametric simulation can assess finite-sample bias, boundary estimates, and interval coverage for a proposed analysis; unlike the local Fisher calculation, it can reproduce the actual estimator and constraints, but its conclusions still depend on the simulated model.

A common mistake is to interpret the number of observations as the only determinant of variance-component precision. For example, 120 observations from 30 subjects with four replicates each do not contain the same covariance information as 120 observations from 120 subjects with one measurement each. The correction is to represent both \(n\) and \(m\) in the covariance model and inspect the full information matrix: replication within subjects supplies contrasts, while independent subjects supply independent realizations of the random effect.