Replication and Information Per Subject

Why Replication Is Not the Same as More Subjects

Repeated observations can make a subject’s trajectory more precisely measured, but they do not necessarily provide the same information as observations on additional independent subjects. The distinction depends on how measurements from the same subject covary. If a subject-specific random effect shifts all that subject’s observations together, repeating a measurement reduces uncertainty about that subject’s average only up to a limit. By contrast, each new independent subject brings a new realization of the population’s random effects.

The relevant unit for population Fisher information is therefore the subject-level observation vector, not an individual row considered in isolation. This tutorial focuses on how changing the number of repeated measurements changes that vector’s covariance matrix and its Fisher contribution. The standard Gaussian information formulas below are stated explicitly; the earlier material on the Gaussian population Fisher matrix provides the broader context.

Information from a Repeated-Measurement Vector

First consider a scalar population mean \(\mu\) that is the same at every measurement occasion, so \(\mu_Y=\mu\mathbf{1}\), where \(\mathbf{1}\) is an \(m\)-vector of ones. If \(\Sigma\) is known and does not depend on \(\mu\), the subject’s expected Fisher information for \(\mu\) is

This formula accounts for correlation: the inverse covariance, rather than the number of recorded values alone, determines the information. If the measurements are independent with common variance \(\sigma^2\), then \(\Sigma=\sigma^2 I_m\) and \(J_{\mu\mu}=m/\sigma^2\). With positive within-subject correlation, the repeated values partly carry overlapping information about the common mean.

When a parameter \(\theta_a\) affects the covariance, the corresponding expected Fisher information entry for a multivariate normal model with parameter-independent mean is

This expression requires a differentiable covariance matrix that is positive definite at the parameter values under consideration. It describes the expected information for the specified subject-level vector. If more than one parameter is estimated, off-diagonal entries can matter: a diagonal entry alone is not generally the effective information after accounting for nuisance parameters.

A Shared Random Effect Changes the Covariance

A useful illustration is a random-intercept model. Let \(Y_j=\mu+b+\epsilon_j\), for \(j=1,\ldots,m\), where \(b\sim N(0,\tau^2)\), the errors \(\epsilon_j\sim N(0,\sigma^2)\) are mutually independent, and \(b\) is independent of all errors. Here \(\tau^2\) is the between-subject random-intercept variance and \(\sigma^2\) is the residual variance. The model implies

Two different measurements from the same subject have covariance \(\tau^2\), not zero: they share the same \(b\). Their marginal variance is \(\tau^2+\sigma^2\). The sample average has variance \(\tau^2+\sigma^2/m\), so the information for \(\mu\) is

The same result follows from \(\mathbf{1}^\mathsf{T}\Sigma^{-1}\mathbf{1}\). The average becomes more precise as \(m\) increases because residual noise is averaged down, while the shared subject effect remains in the average. Thus, for \(\tau^2>0\), \(J_{\mu\mu}(m)\) approaches \(1/\tau^2\) as \(m\) grows. If \(\tau^2=0\), there is no shared random-intercept variation, and \(J_{\mu\mu}(m)=m/\sigma^2\) grows without bound. These are distinct cases; the limit \(1/\tau^2\) applies only when \(\tau^2>0\).

Replication Can Inform Covariance Parameters Too

Replication is not merely repeated measurement of a mean. It can help distinguish residual variation from shared subject variation because the covariance pattern changes with the number of measurements. For the random-intercept model, suppose \(\sigma^2>0\), \(\tau^2>0\), and both are treated as interior covariance parameters. The covariance has one eigenvalue \(\sigma^2+m\tau^2\) in the direction of \(\mathbf{1}\), and \(m-1\) eigenvalues \(\sigma^2\) in directions orthogonal to \(\mathbf{1}\). The contrast directions contain information about residual variation; the common direction contains the shared variation as well.

For example, with \(\sigma^2\) known, the covariance derivative for \(\tau^2\) is \(\partial\Sigma/\partial\tau^2=\mathbf{1}\mathbf{1}^\mathsf{T}\). Substituting this derivative into the Gaussian covariance-information formula gives

For the final equality, \(\Sigma\mathbf{1}=(\sigma^2+m\tau^2)\mathbf{1}\), hence \(\Sigma^{-1}\mathbf{1}=\mathbf{1}/(\sigma^2+m\tau^2)\). The trace of the resulting rank-one product is \(m^2/(\sigma^2+m\tau^2)^2\). This information is for the variance parameter \(\tau^2\), so its units are inverse response-units to the fourth power.

Similarly, when \(\tau^2\) is known, the information for \(\sigma^2\), using \(\partial\Sigma/\partial\sigma^2=I_m\), is

The first term comes from the \(m-1\) contrast directions and the second from the common direction. Both formulas are subject-level expected information components; jointly estimating the two variances also requires their cross-information.

Additional Measurements Versus Independent Subjects

If \(K\) subjects are independent and have the same design and parameter values, their subject-level information matrices add: \(J_{\mathrm{total}}=KJ_{\mathrm{subject}}\). This multiplication applies across independent subject vectors. It does not mean that every repeated observation within one vector contributes the same standalone amount as an independent subject, because their covariance affects the vector’s information.

Assumptions, Diagnostics, and Common Pitfalls

The calculations above assume independent subjects, the stated within-subject covariance model, and a correctly specified mean structure. The Gaussian covariance-information formula additionally assumes a multivariate normal subject-level vector, a differentiable positive-definite \(\Sigma\), and regular parameter values. The random-intercept variance calculation assumes an interior value \(\tau^2>0\); at the boundary \(\tau^2=0\), standard regularity conditions for variance-parameter inference can fail even though the fixed-mean information formula remains well-defined.

A practical diagnostic is to compare observed within-subject patterns with the covariance implied by the model. Under the random-intercept structure, distinct measurements have equal covariance \(\tau^2\); strong decay with time separation, occasion-specific variance, or serial residual dependence would indicate that this structure may be inadequate. Unequal numbers or timings of observations require each subject’s own \(\Sigma\) and information calculation; multiplying one equal-design contribution by the subject count is then inappropriate unless the designs truly match.

• Pitfall: count every row as an independent observation. This uses a diagonal covariance matrix despite a shared random effect. The correction is to construct the joint subject-level \(\Sigma\) and calculate information from it.
• Pitfall: assume replication always makes mean information saturate. Saturation at \(1/\tau^2\) requires \(\tau^2>0\). If \(\tau^2=0\), information is \(m/\sigma^2\) and increases without bound in this model.
• Pitfall: interpret a diagonal Fisher entry as final precision when nuisance parameters are estimated. The full matrix, including cross-information, is needed to account for parameter dependence.

An alternative is to work directly with the likelihood of the full repeated-measurement vector, which is equivalent to using its covariance matrix when the Gaussian model is correctly specified. Treating measurements as independent is a simpler calculation only when independence is justified. Neither approach removes model dependence: misspecified correlation, informative observation patterns, or uncertainty in nominal parameter values can change the design’s actual information.