Gaussian Conditioning and Covariance Updates

From an Observation to a Conditional State Estimate

In filtering, the state is not observed directly. Instead, an observation provides partial information about it, and the state estimate is updated by conditioning on the observed data. For Gaussian state and observation vectors, this update has an explicit form: the conditional mean is an affine function of the observation, and the conditional covariance is a deterministic matrix that does not depend on the observed value.

We work on a filtered probability space $(\Omega,\mathcal{F},(\mathcal{F}_t)_{t\ge0},\mathbb{P})$ carrying a Brownian motion $B_t$. At a fixed time, the calculation below concerns finite-dimensional random vectors; it uses no stochastic integrals. In the filtering setting, the state and observation vectors are Gaussian under the linear Gaussian model. Conditioning on a particular observation value is understood through a regular conditional distribution: a single exact value may have probability zero, so it is not necessary that the event of observing that value have positive probability.

Definition: A random vector $Z$ is Gaussian if every scalar linear combination of its coordinates is normally distributed; this definition allows degenerate Gaussian vectors. The mean of $Z$ is $\mathbb{E}[Z]$, and its covariance is $\operatorname{Cov}(Z)=\mathbb{E}[(Z-\mathbb{E}[Z])(Z-\mathbb{E}[Z])^T]$.

Let $X\in\mathbb{R}^n$ be the state vector and $Y\in\mathbb{R}^m$ the observation vector. Write their means and covariance blocks as

\mathbb{E}[X]=m_X,\qquad \mathbb{E}[Y]=m_Y,\qquad \operatorname{Cov}(X)=P,\qquad \operatorname{Cov}(Y)=S,\qquad \operatorname{Cov}(X,Y)=C.

Here $C=\mathbb{E}[(X-m_X)(Y-m_Y)^T]$ is an $n$-by-$m$ matrix. The joint covariance of $(X,Y)$ is the block matrix with rows $(P,C)$ and $(C^T,S)$. The key condition for the formula below is that $S$ is invertible; in particular, it is positive definite.

The Gaussian Conditioning Formula

Theorem: Suppose $(X,Y)$ is jointly Gaussian, with the means and covariance blocks above, and $S$ is invertible. Then a version of the conditional distribution of $X$ given $Y=y$, for every $y\in\mathbb{R}^m$, is Gaussian with mean and covariance $$ \mathbb{E}[X\mid Y=y]=m_X+CS^{-1}(y-m_Y),\qquad \operatorname{Cov}(X\mid Y=y)=P-CS^{-1}C^T. $$ The conditional covariance is positive semidefinite and is independent of $y$.

Proof. Define the residual vector $U=X-m_X-CS^{-1}(Y-m_Y)$. We first compute its covariance with $Y$: $\operatorname{Cov}(U,Y)=C-CS^{-1}S=0$. The pair $(U,Y)$ is jointly Gaussian because it is an affine transformation of the jointly Gaussian vector $(X,Y)$. Its cross-covariance is zero, so its joint Gaussian characteristic function factors into the product of the characteristic functions of $U$ and $Y$. This factorization proves that $U$ and $Y$ are independent, including when the covariance of $U$ is singular.

Expanding the covariance of $U$ gives $\operatorname{Cov}(U)=P-CS^{-1}C^T$: the two cross terms each contribute $-CS^{-1}C^T$, and the covariance of $CS^{-1}(Y-m_Y)$ contributes $+CS^{-1}C^T$. Since $U$ is independent of $Y$, conditioning on $Y=y$ leaves the distribution of $U$ unchanged. The identity $X=m_X+CS^{-1}(Y-m_Y)+U$ therefore gives the stated conditional mean and covariance. Finally, a covariance matrix is positive semidefinite because for every vector $a$, $a^T\operatorname{Cov}(U)a=\mathbb{E}[(a^T(U-\mathbb{E}[U]))^2]\ge0$. This proves both the formula and the covariance claim. $\square$

This proof also explains why the update is affine: after subtracting the part of $X$ linearly predicted from $Y$, the remaining Gaussian error is independent of $Y$. Conditioning changes the predictable part but not the residual distribution.

A Noisy Scalar Observation and Its Gain

A common observation model has a vector state $X$ and one scalar measurement $Y=HX+V$, where $H$ is a fixed row vector and $V$ is a mean-zero Gaussian measurement error independent of $X$, with variance $R>0$. Suppose $X$ has mean $m$ and covariance $P$. The observation mean is $Hm$, its variance is $HPH^T+R$, and the state-observation covariance is $PH^T$. Applying the theorem gives the update below.

Result: For the model $Y=HX+V$, the gain $K$ and the conditional state moments are $$ K=PH^T(HPH^T+R)^{-1},\qquad \mathbb{E}[X\mid Y=y]=m+K(y-Hm),\qquad \operatorname{Cov}(X\mid Y=y)=P-PH^T(HPH^T+R)^{-1}HP. $$ Since $R>0$, the scalar denominator $HPH^T+R$ is positive, so the inverse is defined.

The gain converts the observation residual $y-Hm$—the difference between the observed value and its prior predicted mean—into a correction to the state estimate. For a scalar state with $H=1$, prior variance $P>0$, and noise variance $R>0$, the gain is $P/(P+R)$, which lies strictly between zero and one. More uncertain prior states give the observation more influence; noisier measurements give it less. In a vector state, $K$ is a matrix, so its entries also reflect how each state coordinate covaries with the measured quantity.

Worked Updates

Worked Example: Updating a Scalar State

Let $X\sim N(2,4)$, let $V\sim N(0,1)$ be independent of $X$, and observe $Y=X+V=y$. Find the conditional mean and variance when $y=5$.

Here $m=2$, $P=4$, $H=1$, and $R=1$. The prior predicted observation mean is $Hm=1\cdot2=2$, and its variance is $HPH^T+R=1\cdot4\cdot1+1=5$. Thus the gain is $K=4/5$. The residual is $y-Hm=5-2=3$, so the conditional mean is $2+(4/5)3=2+12/5=22/5$. The conditional variance is $4-(4\cdot4)/5=4-16/5=4/5$.

Check: The prior precision is $1/4$, and the observation noise precision is $1$. Their sum is $5/4$, whose reciprocal is $4/5$, the calculated posterior variance. The precision-weighted mean is $\big(2(1/4)+5(1)\big)/(5/4)=(1/2+5)(4/5)=22/5$, agreeing with the update.

Worked Example: A Two-Coordinate State with One Measurement

Let the prior state have mean $m=(0,1)^T$ and covariance $P=\begin{pmatrix}4&1\\1&2\end{pmatrix}$. Observe $Y=X_1+X_2+V$, where $V$ is independent Gaussian noise with mean zero and variance $2$. Find the conditional moments when $y=4$.

Here $H=(1,1)$. First, $Hm=0+1=1$. Multiplying the covariance matrix by $H^T$ gives $PH^T=(4+1,1+2)^T=(5,3)^T$. The state contribution to the observation variance is $HPH^T=(1,1)(5,3)^T=5+3=8$, so the total observation variance is $8+2=10$. The gain is therefore $K=(5/10,3/10)^T=(1/2,3/10)^T$.

The observation residual is $4-1=3$. The conditional mean is $(0,1)^T+3(1/2,3/10)^T=(3/2,19/10)^T$. For the covariance, subtract $\frac1{10}(5,3)^T(5,3)$ from $P$: $$ \begin{pmatrix}4&1\\1&2\end{pmatrix} -\frac1{10}\begin{pmatrix}25&15\\15&9\end{pmatrix} = \begin{pmatrix}3/2&-1/2\\-1/2&11/10\end{pmatrix}. $$

Check: The conditional variance in the measured direction is $(1,1)\begin{pmatrix}3/2&-1/2\\-1/2&11/10\end{pmatrix}(1,1)^T =3/2-1/2-1/2+11/10=8/5$. The prior variance in that direction was $8$; the update reduces it by $8-8/5=32/5$, which equals the squared state-observation covariance contribution $ (5^2+2\cdot5\cdot3+3^2)/10=64/10=32/5$. The updated covariance is positive definite: its determinant is $(3/2)(11/10)-(1/2)^2=33/20-1/4=28/20=7/5>0$, and its top-left entry is positive.

Worked Example: Conditioning on Two Noisy Measurements

Let $X\sim N(0,1)$, and let $Y_1=X+V_1$ and $Y_2=2X+V_2$, where $V_1,V_2$ are independent $N(0,1)$ variables, independent of $X$. Find the conditional mean and variance of $X$ given $Y_1=1$ and $Y_2=0$.

Set $Y=(Y_1,Y_2)^T$. Its mean is zero. Its covariance entries are $\operatorname{Var}(Y_1)=1+1=2$, $\operatorname{Cov}(Y_1,Y_2)=\operatorname{Cov}(X+V_1,2X+V_2)=2$, and $\operatorname{Var}(Y_2)=4+1=5$. Thus $S=\begin{pmatrix}2&2\\2&5\end{pmatrix}$. Also $C=\operatorname{Cov}(X,Y)=(1,2)$. The determinant of $S$ is $2\cdot5-2\cdot2=6$, so $S^{-1}=\frac16\begin{pmatrix}5&-2\\-2&2\end{pmatrix}$. Multiplication gives $CS^{-1}=(1,2)\frac16\begin{pmatrix}5&-2\\-2&2\end{pmatrix}=(1/6,1/3)$.

At $y=(1,0)^T$, the conditional mean is $0+(1/6,1/3)(1,0)^T=1/6$. The conditional variance is $1-(1/6,1/3)(1,2)^T=1-(1/6+2/3)=1-5/6=1/6$.

Check: Combining the two independent measurement likelihoods adds precisions $1$ and $2^2=4$ to the prior precision $1$, giving posterior variance $1/(1+1+4)=1/6$. Their precision-weighted observation values contribute $1\cdot1+2\cdot0=1$, so the posterior mean is $1/6$, as calculated by the covariance formula.

Why the Covariance Update Matters

The covariance reduction $P-PH^T(HPH^T+R)^{-1}HP$ quantifies how much uncertainty the measurement removes. It is always positive semidefinite as a reduction: for any vector $a$, the variance of $a^TX$ after conditioning cannot exceed its prior variance. One way to see the matrix form is to use the residual $U=X-m-K(Y-Hm)$. Its covariance is the conditional covariance, and the difference between the prior covariance and this residual covariance is $PH^T(HPH^T+R)^{-1}HP$, a positive semidefinite matrix.

The conditional mean depends on the realized observation, but the conditional covariance does not. This separation is useful computationally: for a fixed model, the covariance update can be calculated before the observation arrives, while the mean update uses the observed residual. In a sequential filtering problem, these finite-dimensional updates are the basic Gaussian calculation behind state estimation; continuous-time evolution and the dynamics of the gain require additional arguments.

A Common Mistake

Omitting observation uncertainty gives the wrong denominator. In the scalar example with prior variance $4$, unit observation coefficient, and noise variance $1$, an incorrect calculation might use $K=P=4$, giving the claimed mean $2+4(5-2)=14$ and variance $4-4^2=-12$. A negative variance already signals an error. The observation has variance $P+R=4+1=5$, not $1$; the correct gain is $P/(P+R)=4/5$, giving mean $22/5$ and variance $4/5$. The noise variance belongs in the observation covariance because the measured value contains both state uncertainty and measurement noise.

Check Your Understanding

For each question, assume the stated Gaussian vectors and positive observation-noise variances, so the observation covariance matrices used in conditioning are invertible.

For $X\sim N(3,2)$ and $Y=X+V$, with independent $V\sim N(0,2)$, calculate the conditional mean and variance given $Y=1$.
In the model $Y=HX+V$, identify the prior predicted observation mean, the observation variance, and the state-observation covariance.
Why does zero cross-covariance between the Gaussian residual $U$ and $Y$ imply their independence?
For a scalar state with $H=1$, explain how the gain changes as the observation-noise variance increases while the prior variance stays fixed and positive.
For the two-coordinate example, verify directly that the conditional covariance is symmetric and has nonnegative diagonal entries.