From Observations to a Mean Equation
In a filtering problem, the signal is not observed directly. The observation record contains both information about the signal and independent measurement noise. The conditional mean summarizes the information in that record at each time. For a linear Gaussian model, it evolves as a linear drift plus a correction driven by the innovations: the part of the observation not predicted by the current conditional mean.
Fix a finite horizon \(T>0\) and a filtered probability space \((\Omega,\mathcal{F},(\mathcal{F}_t)_{t\ge0},\mathbb{P})\) carrying a standard \((n+m)\)-dimensional Brownian motion \(\mathbf{B}_t=(\mathbf{B}^s_t,\mathbf{B}^o_t)\). Its \(n\)-dimensional signal-noise component \(\mathbf{B}^s\) and \(m\)-dimensional observation-noise component \(\mathbf{B}^o\) are independent. Assume \(X_0\) is independent of \(\mathbf{B}\) and Gaussian with mean \(m_0\) and finite covariance \(P_0\). Take constant matrices \(A\in\mathbb{R}^{n\times n}\), \(G\in\mathbb{R}^{n\times n}\), \(C\in\mathbb{R}^{m\times n}\), and an invertible \(S\in\mathbb{R}^{m\times m}\). Write \(R=SS^{\mathsf T}\), which is positive definite.
All stochastic integrals below are Ito integrals. Their integrands are progressively measurable and square-integrable on \([0,T]\): in vector or matrix notation, each component \(H\) satisfies \(\mathbb{E}\int_0^T\|H_s\|^2\,ds<\infty\). The bounded linear coefficients and finite second moment of \(X_0\) ensure that \(X\) has finite second moments on every finite horizon. Joint Gaussianity of the signal and observations, together with Gaussian projection, implies that \(P_t\) is deterministic; it is not random even though \(m_t\) is random.
Innovations and the Conditional-Mean Equation
Define the innovations process by $$ I_t=Y_t-\int_0^t C m_s\,ds. $$ The innovations theorem for this linear Gaussian model says that \(J_t=S^{-1}I_t\) is an \(m\)-dimensional standard Brownian motion in the observation filtration, and that this filtration is generated by \(J\). This is the innovations Brownian result established in the earlier material on the innovations process. In particular, \(dI_t=C(X_t-m_t)\,dt+S\,d\mathbf{B}^o_t\), and the quadratic covariation is \(d[I]_t=R\,dt\).
Proof. We first identify the drift and the form of the remaining term. For \(s\le t\), the signal equation and independence of future signal noise from the observation history give \(\mathbb{E}[X_t\mid\mathcal{Y}_s]=e^{A(t-s)}m_s\). Thus \(e^{-At}m_t\) is a martingale in \((\mathcal{Y}_t)\): its conditional expectation at time \(s\) is \(e^{-At}e^{A(t-s)}m_s=e^{-As}m_s\). It is square-integrable on \([0,T]\), since conditional expectation contracts the \(L^2\) norm and the signal has finite second moments. The product rule for the deterministic matrix \(e^{At}\) therefore gives \(m_t=m_0+\int_0^t A m_s\,ds+N_t\), where \(N\) is a square-integrable \(\mathcal{Y}_t\)-martingale with \(N_0=0\).
Every centered variable in this model is jointly Gaussian with the observations. Gaussian projection places \(m_t-m_0\), and hence \(N_t\), in the closed linear span of centered observations up to time \(t\). The innovations span the same observation space. Consequently, the Brownian martingale representation theorem in the filtration generated by \(J\) gives \(N_t=\int_0^t H_s\,dJ_s\). Here \(H\) can be taken deterministic: deterministic-integrand Wiener integrals form the closed Gaussian linear span of Brownian increments, and the representation is unique by the Ito isometry. Square-integrability of \(N_T\) gives \(\int_0^T\|H_s\|^2ds<\infty\). Since \(dI=S\,dJ\), put \(K_s=H_sS^{-1}\) to obtain \(dm_s=A m_s\,ds+K_s\,dI_s\). The martingale representation theorem is used here in its standard form; its general proof is not needed for the covariance identification that follows.
Let \(e_t=X_t-m_t\). Conditional-mean projection gives \(\mathbb{E}[e_t\mid\mathcal{Y}_t]=0\), so \(\mathbb{E}[e_t I_t^{\mathsf T}]=0\), since \(I_t\) is square-integrable and \(\mathcal{Y}_t\)-measurable. Also \(\mathbb{E}[e_te_t^{\mathsf T}]=P_t\), because the conditional covariance is deterministic. The error and innovations satisfy $$ de_t=Ae_t\,dt+G\,d\mathbf{B}^s_t-K_t\,dI_t,\qquad dI_t=Ce_t\,dt+S\,d\mathbf{B}^o_t. $$ Apply the product rule to \(e_tI_t^{\mathsf T}\) and take expectations. The drift contribution from \(A e_t I_t^{\mathsf T}\) is zero because \(\mathbb{E}[e_tI_t^{\mathsf T}]=0\). The term \(e_t\,dI_t^{\mathsf T}\) contributes \(P_tC^{\mathsf T}dt\); its Brownian-integral part has zero expectation. Independence of the two Brownian components and \(d[I]_t=R\,dt\) give the cross-variation contribution \(-K_tR\,dt\). All stochastic-integral expectations are justified by square-integrability on \([0,T]\). Integrating the resulting identity, whose left side is zero at every time, yields \(P_tC^{\mathsf T}-K_tR=0\) for almost every \(t\). Since \(R\) is invertible, \(K_t=P_tC^{\mathsf T}R^{-1}\) almost everywhere, which is sufficient for equality of the Ito integrals. Finally, \(P_t\preceq\mathbb{E}[X_tX_t^{\mathsf T}]\) in the sense of bounded second moments, so \(P_t\) is bounded on finite horizons; thus \(K\) is square-integrable. \(\square\)
The drift \(A m_t\) is the signal's deterministic linear evolution applied to the current estimate. The gain multiplies the innovation \(dY_t-Cm_t\,dt\), not the raw observation increment alone. The factor \(R^{-1}\) accounts for observation-noise covariance: when measurement noise is large in a direction, that direction receives less weight.
Worked Gain Calculations
Worked Example: A Two-State Signal and One Observation
At a fixed time, suppose $$ A=\begin{pmatrix}0&1\\-1&0\end{pmatrix},\quad C=\begin{pmatrix}1&2\end{pmatrix},\quad R=(4),\quad P_t=\begin{pmatrix}2&1\\1&3\end{pmatrix},\quad m_t=\begin{pmatrix}1\\0\end{pmatrix}. $$ Find the gain and write the mean equation at that time.
First \(C^{\mathsf T}=(1,2)^{\mathsf T}\), so $$ P_tC^{\mathsf T} =\begin{pmatrix}2&1\\1&3\end{pmatrix} \begin{pmatrix}1\\2\end{pmatrix} =\begin{pmatrix}2+2\\1+6\end{pmatrix} =\begin{pmatrix}4\\7\end{pmatrix}. $$ Because \(R^{-1}=1/4\), \(K_t=(1,7/4)^{\mathsf T}\). Also \(Am_t=(0,-1)^{\mathsf T}\) and \(Cm_t=1\). Therefore $$ dm_t=\begin{pmatrix}0\\-1\end{pmatrix}dt+ \begin{pmatrix}1\\7/4\end{pmatrix}(dY_t-dt). $$ Check: \(K_tR=(4,7)^{\mathsf T}=P_tC^{\mathsf T}\), as required.
Worked Example: Gain at the Start of a Scalar Filter
Let \(X_t=X_0\), where \(X_0\sim N(2,9)\), and let \(dY_t=2X_t\,dt+3\,d\mathbf{B}^o_t\). At time zero, \(m_0=2\), \(P_0=9\), \(C=2\), and \(R=3^2=9\). The gain is $$ K_0=P_0CR^{-1}=9\cdot2\cdot\frac{1}{9}=2. $$ Since \(Cm_0=2\cdot2=4\), the mean equation at the initial time is \(dm_t=2(dY_t-4\,dt)\), evaluated at \(t=0\). The gain later changes as the conditional covariance changes; this calculation does not assume it stays equal to \(2\).
Check: the covariance-weighted observation sensitivity is \(P_0C=9\cdot2=18\), while gain times observation variance is \(K_0R=2\cdot9=18\). The two agree.
Worked Example: Correlated Noise in Two Observation Channels
Suppose \(n=m=2\), and at a fixed time $$ P_t=\begin{pmatrix}1&0\\0&4\end{pmatrix},\quad C=\begin{pmatrix}1&1\\1&-1\end{pmatrix},\quad R=\begin{pmatrix}2&1\\1&2\end{pmatrix}. $$ Compute the gain. First, $$ C^{\mathsf T}=C,\qquad P_tC^{\mathsf T}=\begin{pmatrix}1&1\\4&-4\end{pmatrix},\qquad R^{-1}=\frac13\begin{pmatrix}2&-1\\-1&2\end{pmatrix}. $$ Multiplication gives $$ K_t=\frac13\begin{pmatrix}1&1\\4&-4\end{pmatrix} \begin{pmatrix}2&-1\\-1&2\end{pmatrix} =\begin{pmatrix}1/3&1/3\\4&-4\end{pmatrix}. $$ For \(m_t=(1,2)^{\mathsf T}\), \(Am_t\) is the drift term and \(Cm_t=(3,-1)^{\mathsf T}\), so the correction uses \(dI_t=dY_t-(3,-1)^{\mathsf T}dt\). The mean equation is \(dm_t=Am_t\,dt+K_t\,dI_t\).
Check: $$ K_tR= \begin{pmatrix}1/3&1/3\\4&-4\end{pmatrix} \begin{pmatrix}2&1\\1&2\end{pmatrix} =\begin{pmatrix}1&1\\4&-4\end{pmatrix} =P_tC^{\mathsf T}. $$ Thus the computed matrix satisfies the defining gain identity, including the off-diagonal observation-noise covariance.
Why Projection Determines the Gain
The error \(e_t\) is orthogonal to every square-integrable quantity measurable from observations through time \(t\). In particular, it is orthogonal to the accumulated innovations. The mean equation says how the estimate responds to new innovations; the cross-variation calculation in the proof forces that response to be \(P_tC^{\mathsf T}R^{-1}\). This is the matrix form of a signal-observation covariance divided by observation-noise covariance.
A useful limiting check is \(C=0\). Then \(P_tC^{\mathsf T}R^{-1}=0\), so observations contain no signal term and the conditional mean follows \(dm_t=A m_t\,dt\). Another check is that a positive definite \(R\) is essential in this formula: if \(R\) is singular, \(R^{-1}\) does not exist, and one must reformulate the observation model or work on the noise-supported subspace rather than apply this expression unchanged.
Common mistake. Using \(K_t=P_tC^{\mathsf T}\) omits the observation-noise scaling. In the first example this incorrect calculation gives \((4,7)^{\mathsf T}\), whereas the correct gain is \((1,7/4)^{\mathsf T}\), because \(R=4\). Substitution exposes the error: the incorrect gain gives \(K_tR=(16,28)^{\mathsf T}\), which is not \(P_tC^{\mathsf T}=(4,7)^{\mathsf T}\). The correction is to multiply by \(R^{-1}\), yielding \(K_tR=(4,7)^{\mathsf T}\).
Check Your Understanding
Use the gain identity and the innovation definition to check each answer.
- For scalar \(P_t=5\), \(C=3\), and \(R=6\), compute \(K_t\) and verify \(K_tR=P_tC\).
- Explain why the conditional covariance \(P_t\) is deterministic under the linear Gaussian assumptions.
- For \(C=0\), write the conditional-mean equation and interpret the missing innovation term.
- In the multidimensional example, compute \(Cm_t\) for \(m_t=(1,2)^{\mathsf T}\) and state the resulting innovation increment.
- Why does the gain formula in this tutorial require \(R\) to be invertible?