Covariance Patterns That Distinguish the Two Noise Sources

Why covariance carries information about the noise source

Two sources of variation can make concentrations scatter around a predicted profile, but they do not generally leave the same pattern across time. A process perturbation changes the evolving concentration path. Its effect can persist, so measurements at different times share some of the same perturbations. Independent observation errors, by contrast, affect the recorded values at their own measurement times and do not create covariance between separate measurements.

The distinction is clearest in a model where the deterministic pharmacokinetic trajectory is separated from a stochastic fluctuation around it. We use a mean-reverting fluctuation as a tractable example. The same reasoning extends to linear models with other response kernels, but the precise covariance pattern depends on the dynamics.

A mean-reverting concentration fluctuation

Let \(C(t)=m(t)+X(t)\), where \(m(t)\) is a deterministic reference concentration and \(X(t)\) is a zero-mean fluctuation. Consider

Here \(k\) controls how quickly a fluctuation is pulled back toward zero, \(\sigma\) controls the strength of process perturbations, and \(W(t)\) is standard Brownian motion. The zero initial fluctuation is assumed for the derivation below. A nonzero deterministic initial fluctuation would add a deterministic term, not a covariance contribution. The model is an additive-noise approximation; it can produce negative concentrations near zero and should not be treated as a universally realistic concentration model.

The solution is

For deterministic square-integrable integrands, the covariance of two Itô integrals against the same Brownian motion is the integral of the product of their integrands. This standard consequence of the Itô isometry makes the shared Brownian history explicit: the perturbations before the earlier time can affect both concentrations.

Proof. The deterministic kernel \(u\mapsto \sigma e^{-k(t-u)}\) is square-integrable on \([0,t]\), so the Itô integral has mean zero. For \(s\le t\), the covariance identity for deterministic-kernel Itô integrals gives

Setting \(s=t\) gives the variance formula. The covariance is positive when \(s>0\), because \(t-s<t+s\) and \(k>0\), so \(e^{-k(t-s)}>e^{-k(t+s)}\). The shared perturbations before time \(s\) therefore produce positive covariance in this model. \(\square\)

Adding independent observation errors

Suppose observations at times \(t_1,\ldots,t_n\) are \(Y_i=C(t_i)+\epsilon_i\), where \(E[\epsilon_i]=0\), \(\operatorname{Var}(\epsilon_i)=\tau_i^2\), and the errors are mutually independent and independent of the process. Since \(m(t_i)\) is deterministic, it contributes no covariance. For any \(i,j\),

When \(i\ne j\), error independence makes the second term zero. When \(i=j\), it equals \(\tau_i^2\). Thus the observation covariance matrix has the form \(K+D\), where \(K\) is the latent process covariance matrix and \(D\) is diagonal with entries \(\tau_i^2\).

To verify the result directly, expand the covariance of \(Y_i\) and \(Y_j\). The two cross-covariances between \(X\) and \(\epsilon\) are zero by independence. The remaining terms are the process covariance and the error covariance. For distinct indices, the latter is zero; for equal indices, it is \(\operatorname{Var}(\epsilon_i)=\tau_i^2\). This proves the matrix decomposition without requiring normality.

What the covariance pattern says

In this mean-reverting model, process noise creates positive covariance across distinct times because part of the random history is shared. The covariance depends on both the separation between times and how much process variation has accumulated by the earlier time. Independent observation error instead appears only on the diagonal. The diagonal alone does not identify the sources: a large observed variance could reflect process variability, observation error, or both. Off-diagonal covariance supplies additional information under the model assumptions.

A useful comparison is the stationary version of the same mean-reverting process. If \(X(0)\) is independent of future Brownian increments and has distribution \(N(0,\sigma^2/(2k))\), then the process has variance \(\sigma^2/(2k)\) at every time, and for \(s\le t\),

This follows by writing \(X(t)=e^{-k(t-s)}X(s)+\sigma\int_s^t e^{-k(t-u)}\,dW(u)\). The integral is independent of \(X(s)\) and has mean zero, so the covariance is \(e^{-k(t-s)}\operatorname{Var}(X(s))\). The stationary covariance depends only on the time lag; the zero-initial-condition covariance above does not.

Diagnostics, alternatives, and limitations

In practice, covariance patterns are estimated from repeated information across individuals or replicates under a defined population model; two isolated observations do not provide a reliable covariance estimate. A useful diagnostic is to compare the fitted covariance structure with empirical residual patterns after accounting for the deterministic concentration profile and any modeled between-subject variation. Systematic residual association across times suggests that a diagonal-only error model may be inadequate. Conversely, a fitted process covariance that cannot reproduce observed temporal association may indicate an incorrect process model, an omitted source of shared variation, or insufficient information to separate components.

The main alternative is an observation-error-only model with independent errors. It permits variation at each measured time but predicts zero covariance between distinct residuals, conditional on the modeled mean and other included effects. A process-noise model predicts covariance according to the dynamics. A combined model allows both: process covariance off the diagonal and process-plus-observation variance on the diagonal. Selecting among these is not accomplished by labeling all scatter as one source; it requires checking whether the implied covariance structure is compatible with the data.

Several limitations matter. Correlated assay errors, shared batches, or common sample handling can create off-diagonal covariance that resembles process covariance. Random individual parameters can also induce correlation across a person's concentrations, even without a Brownian process perturbation. Further, not every process model has positive covariance at every lag; the positive pattern derived here belongs to the stated mean-reverting model. The decomposition is consequently a way to reason from assumptions and covariance structure, not a stand-alone proof of causal source.

A common mistake is to say, “measurement error increases variability, so it must increase covariance between any two observations.” The correction is to calculate covariance: independent errors have \(\operatorname{Cov}(\epsilon_i,\epsilon_j)=0\) for \(i\ne j\), so they increase the diagonal variances only. If errors share a batch effect or other common component, that independence premise fails and the error covariance need not be zero.