Combining Process Noise and Measurement Noise

One observation, two sources of variation

A measured concentration is not necessarily the concentration state itself. The state can vary as it evolves between observations, while the assay can also produce a value different from the state present at the sampling time. Combining these sources in one observation model makes explicit which part of the variability belongs to the evolving system and which part belongs to measurement.

Let $C_t$ denote the latent concentration at time $t$, and let $Y_i$ be the recorded concentration at sampling time $t_i$. For an additive measurement model, write $Y_i=C_{t_i}+\epsilon_i$. The state may follow a stochastic differential equation, while $\epsilon_i$ represents measurement error. The central question is how the uncertainty in $C_{t_i}$ and the error in $\epsilon_i$ contribute to the variability of $Y_i$.

Definition: A process-noise contribution is variation in the latent state $C_t$ generated by stochastic state evolution, under the specified model and conditioning information. A measurement-noise contribution is variation in an observed value conditional on the latent state at the sampling time. These are distinct parts of the model; their variances can be added directly only under stated assumptions.

For example, a state model might be $dC_t=a(C_t,t)\,dt+b(C_t,t)\,dW_t$, where $W_t$ is standard Brownian motion. Assume the coefficients and initial condition are such that the model has a well-defined, nonexplosive solution with finite second moments over the times considered. The Brownian-driven term describes stochastic evolution of the state. A separate observation model describes how that state produces a recorded value.

The variance decomposition

The key result is the law of total variance. It applies whether the measurement error is additive or concentration-dependent, and it clarifies exactly what is meant by the two contributions.

Theorem: If $Y$ is square-integrable and $C$ is the latent state at the sampling time, then $$ \operatorname{Var}(Y)=\operatorname{Var}\!\big(E[Y\mid C]\big)+E\!\big[\operatorname{Var}(Y\mid C)\big]. $$ The first term is variation in the conditional mean of the observation across states; the second is the average conditional measurement variance.

Proof. Write $m(C)=E[Y\mid C]$. Since $E[Y-m(C)\mid C]=0$, the cross-product has expectation zero: $$ E\!\left[(m(C)-E[Y])(Y-m(C))\right] =E\!\left[(m(C)-E[Y])E[Y-m(C)\mid C]\right]=0. $$ Now express $Y-E[Y]$ as $(m(C)-E[Y])+(Y-m(C))$, square, and take expectations. The cross-product vanishes, leaving $$ \operatorname{Var}(Y)=E[(m(C)-E[Y])^2]+E[(Y-m(C))^2]. $$ The first term is $\operatorname{Var}(E[Y\mid C])$; by conditioning, the second is $E[\operatorname{Var}(Y\mid C)]$. Thus the stated identity holds. $\square$

In the additive model $Y=C+\epsilon$, suppose $E[\epsilon\mid C]=0$ and $\operatorname{Var}(\epsilon\mid C)=R$, where $R$ is a fixed nonnegative variance. Then $E[Y\mid C]=C$ and $\operatorname{Var}(Y\mid C)=R$. The theorem gives the familiar sum:

\operatorname{Var}(Y)=\operatorname{Var}(C)+R.

This does not require the state $C$ to be normally distributed. It does require the stated conditional mean and variance. In particular, independent, mean-zero measurement error with variance $R$ satisfies these conditions. If the error has a nonzero conditional mean, depends on the state in its mean, or shares a noise source with the state, the simple formula may not apply.

Worked Example: Adding Independent Additive Variances

At a particular sampling time, suppose the latent concentration has distribution $C\sim N(8,1.5)$, and the assay error is independent with $\epsilon\sim N(0,0.64)$. The observation is $Y=C+\epsilon$. Find its mean, variance, and standard deviation.

The mean is $E[Y]=E[C]+E[\epsilon]=8+0=8$. Independence makes $\operatorname{Cov}(C,\epsilon)=0$, so $$ \operatorname{Var}(Y)=\operatorname{Var}(C)+\operatorname{Var}(\epsilon)=1.5+0.64=2.14. $$

Thus the observation standard deviation is $\sqrt{2.14}\approx1.463$, and the sum of independent normal variables gives $Y\sim N(8,2.14)$. The state contributes variance $1.5$; measurement contributes variance $0.64$. Adding standard deviations instead would give $ \sqrt{1.5}+0.8$, which is not the standard deviation of this sum.

Calculating process variance from a state model

The state variance in the decomposition must be obtained from the state model, not assigned to the assay. A useful derivation is available for a linear stochastic model with constant coefficients. Consider $$ dC_t=-kC_t\,dt+\sigma\,dW_t,\qquad C_0=c_0, $$ where $k>0$, $\sigma>0$, $c_0$ is fixed, and $W_t$ is standard Brownian motion. Here the stochastic forcing acts on the latent state. Assume the observation error is added separately.

Result: For the stated linear model, $$ E[C_t]=c_0e^{-kt},\qquad \operatorname{Var}(C_t)=\frac{\sigma^2}{2k}\left(1-e^{-2kt}\right). $$

Derivation. Multiply the SDE by the integrating factor $e^{kt}$. Since this factor is deterministic, the product rule gives $$ d(e^{kt}C_t)=\sigma e^{kt}\,dW_t. $$ Integrating from $0$ to $t$ yields $$ C_t=c_0e^{-kt}+\sigma\int_0^t e^{-k(t-s)}\,dW_s. $$ The Itô integral with a deterministic square-integrable integrand has mean zero and variance equal to the integral of the squared integrand. Therefore $$ E[C_t]=c_0e^{-kt},\qquad \operatorname{Var}(C_t)=\sigma^2\int_0^t e^{-2k(t-s)}\,ds =\frac{\sigma^2}{2k}(1-e^{-2kt}). $$ The integral is finite because $k>0$ and $t$ is finite. This variance is generated in the state model; it is not measurement variance. $\square$

Worked Example: State Variance Plus Assay Variance

Suppose $dC_t=-0.4C_t\,dt+0.6\,dW_t$, $C_0=10$, with time measured in hours. At $t=2$ hours, an additive assay error has mean zero and variance $0.25$, independent of the state. Calculate the observation mean and variance.

The state mean is $10e^{-0.4(2)}=10e^{-0.8}\approx4.493$. The state variance is $$ \frac{0.6^2}{2(0.4)}(1-e^{-2(0.4)(2)}) = \frac{0.36}{0.8}(1-e^{-1.6}) =0.45(1-0.2019)\approx0.3591. $$

For $Y=C_2+\epsilon$, the measurement variance is $0.25$, so $$ E[Y]\approx4.493,\qquad \operatorname{Var}(Y)\approx0.3591+0.25=0.6091. $$ The observation standard deviation is $\sqrt{0.6091}\approx0.7804$. The state-model variance and assay variance are separate quantities even though both contribute to the spread of observed values.

This linear model is an illustrative additive-noise model, not a guarantee that concentration remains positive: a Gaussian state can take negative values. Its suitability therefore depends on the intended approximation and concentration range.

When measurement variance depends on concentration

Assay variability is not always constant across the concentration range. The general variance decomposition still applies, but measurement variance can no longer be taken outside the expectation. For instance, a positive multiplicative error model can be written $Y=C\exp(\epsilon-s^2/2)$, with $C>0$, $\epsilon\sim N(0,s^2)$, and $\epsilon$ independent of $C$. The correction $-s^2/2$ makes the observation conditionally unbiased: $E[Y\mid C]=C$.

Worked Example: A Positive Multiplicative Measurement Model

Suppose the latent state takes the values $8$ and $12$, each with probability $1/2$. Thus $E[C]=(8+12)/2=10$, $\operatorname{Var}(C)=((8-10)^2+(12-10)^2)/2=4$, and $E[C^2]=(64+144)/2=104$. Let the measurement model be $Y=C\exp(\epsilon-s^2/2)$, with $\epsilon\sim N(0,s^2)$, independent of $C$, and $s^2=\ln(1.25)$.

For a normal error, $E[e^\epsilon]=e^{s^2/2}$ and $E[e^{2\epsilon}]=e^{2s^2}$. Hence $E[Y\mid C]=C$, while $$ \operatorname{Var}(Y\mid C) =C^2\left(e^{s^2}-1\right)=0.25C^2. $$ The theorem gives the state contribution $\operatorname{Var}(E[Y\mid C])=\operatorname{Var}(C)=4$, and the average measurement contribution is $$ E[\operatorname{Var}(Y\mid C)]=0.25E[C^2]=0.25(104)=26. $$ Therefore $E[Y]=10$ and $\operatorname{Var}(Y)=4+26=30$. The measurement term depends on the squared concentration, so it cannot be treated as a fixed additive variance. The model also preserves positive observations.

Assumptions, alternatives, and warning signs

The additive constant-variance model $Y_i=C_{t_i}+\epsilon_i$ is simple and interpretable when errors are approximately unbiased, have a stable variance on the concentration scale, and are independent of the state. A proportional or log-scale observation model may be more suitable when assay variability grows with concentration or when positivity is important. The choice changes the conditional observation distribution and can change the variance decomposition; it does not change the distinction between state evolution and observation.

A practical diagnostic is to check whether the observation model implies plausible values and variance across the measured range. Constant additive error can be implausible when low concentrations cannot be negative or when residual spread clearly increases with concentration. A proportional model can be unsuitable near zero if it implies vanishing measurement variance where the assay has a nonzero noise floor. These are signs to examine the observation model, not reasons to move all unexplained variation into the state model.

The simple additive sum also has limits. If measurement errors are correlated with the state or have conditional bias, the conditional mean is not $C$. If variance depends on concentration, the measurement contribution is $E[\operatorname{Var}(Y\mid C)]$, not one fixed number. If parameters or initial conditions are uncertain, the state variance may include that uncertainty as well as stochastic forcing; the source should be stated according to what has been conditioned upon. This tutorial concerns the marginal variance at an observation time; how dependence across sampling times helps distinguish noise sources is a separate question.

Common pitfall: It is incorrect to add standard deviations, or to label every difference between observations and a deterministic prediction as assay error. Variances add in the independent, additive model because the component covariance is zero. First specify the latent state model and conditional measurement model; then derive the observation variance under their assumptions.

The law of total variance is the most flexible way to keep the contributions straight. For an additive unbiased assay with fixed variance, it reduces to state variance plus assay variance. For state-dependent error, it separates variation in the conditional mean from average conditional measurement variance. Neither expression identifies the components from one observed spread alone; the model assumptions and available data must support that distinction.

Check Your Understanding

Try each question, then compare with the solutions below.

If $\operatorname{Var}(C)=2.25$ and independent additive measurement error has variance $0.49$, what is $\operatorname{Var}(Y)$ and the observation standard deviation?
In the linear state model above, what happens to the process variance at $t=0$, and what does that represent when $C_0$ is fixed?
Why is it not valid to use a fixed measurement variance in the multiplicative model of the third worked example?
State the two terms in the law of total variance and interpret them for a concentration observation.

Solutions. (1) The total variance is $2.25+0.49=2.74$, so the standard deviation is $\sqrt{2.74}\approx1.655$; variances, not standard deviations, were added. (2) At $t=0$, $\frac{\sigma^2}{2k}(1-e^0)=0$. With a fixed initial state, no stochastic evolution has yet accumulated. (3) Its conditional variance is $C^2(e^{s^2}-1)$, which changes with $C$; the marginal measurement contribution is its expectation over the state distribution. (4) They are $\operatorname{Var}(E[Y\mid C])$, variation in the conditional observation mean across latent states, and $E[\operatorname{Var}(Y\mid C)]$, average observation variance conditional on the state.