The Innovations Process and Its Brownian Character

Observed Data After Removing Its Prediction

In filtering, observations contain both the part predicted from the information available so far and a residual. The residual is called the innovation. Its importance is structural: under a linear Gaussian model, the innovation, after normalization for the observation-noise covariance, is Brownian motion relative to the information generated by the observations. Thus the residual at each time represents genuinely new observation noise, even though its definition uses a prediction based on the past.

Work on a filtered probability space \((\Omega,\mathcal{F},(\mathcal{F}_t)_{t\ge0},\mathbb{P})\). Let \(V_t\) and \(B_t\) be independent standard Brownian motions, of dimensions \(q\) and \(d\), respectively. Consider the linear model \(dX_t=A(t)X_t\,dt+G(t)\,dV_t\) and \(dY_t=C(t)X_t\,dt+L\,dB_t\), where \(X_t\in\mathbb{R}^n\), \(Y_t\in\mathbb{R}^d\), and \(L\) is a constant invertible \(d\times d\) matrix. Assume \(X_0\) is Gaussian with finite second moment and independent of \(V\) and \(B\); assume the deterministic coefficient matrices are locally bounded. The observation filtration \(\mathcal{F}^Y_t\) is the usual completion and right-continuous augmentation of the filtration generated by \((Y_s)_{0\le s\le t}\). The observation covariance matrix per unit time is \(R=LL^{\mathsf T}\), which is positive definite.

Here \(\hat X_t=\mathbb{E}[X_t\mid\mathcal{F}^Y_t]\) is the conditional mean of the state. The innovation subtracts the integrated predicted observation drift from the observed data. All stochastic integrals below are Ito integrals; whenever an integrand \(H_s\) is used in such an integral on \([0,T]\), it is adapted and progressively measurable and satisfies \(\mathbb{E}\int_0^T\lVert H_s\rVert^2\,ds<\infty\). In this model the noise integrand \(L\) is constant, and the finite-second-moment assumptions ensure the required integrability on every finite time interval.

The prediction in this definition is conditional on the observation filtration, not on the full information that includes the hidden state. The distinction matters: \(C(t)\hat X_t\) is the best conditional prediction of the instantaneous observation drift, whereas \(C(t)X_t\) is the unobserved drift itself.

Why the Innovation Is a Martingale

We first verify the martingale property directly. For \(0\le s\le t\), the observation increment is \(\int_s^t C(u)X_u\,du+L(B_t-B_s)\). The observation-noise increment has conditional mean zero given \(\mathcal{F}^Y_s\), since \(\mathcal{F}^Y_s\) is contained in the full information at time \(s\), and Brownian increments after \(s\) are independent of that information. For the drift term, conditional expectation and the tower property give \(\mathbb{E}[C(u)X_u\mid\mathcal{F}^Y_s]=\mathbb{E}[C(u)\hat X_u\mid\mathcal{F}^Y_s]\) for \(u\ge s\). Integrating this equality shows that the conditional expectation of the increment of \(I\) is zero.

These conditional expectations are legitimate: on every finite horizon the linear model has finite second moments, and conditional expectation does not increase the \(L^2\) norm. The Cauchy–Schwarz inequality then makes the time integrals of the drift terms integrable. The same bounds give \(\mathbb{E}\lVert I_t\rVert^2<\infty\) for each finite \(t\). Thus \(I_t\) is a square-integrable martingale in the observation filtration, not merely a local martingale.

Proof. The martingale property and square integrability were verified above. The paths of \(I\) are continuous because \(Y\) is continuous and its subtracted time integral is continuous. Also, \(I_t=L B_t+\int_0^t C(s)(X_s-\hat X_s)\,ds\). The second term has continuous finite-variation paths on every finite interval. A continuous finite-variation process has zero quadratic variation and zero covariation with a continuous semimartingale. Therefore, entry by entry, \([I]_t=[LB]_t=LL^{\mathsf T}t=Rt\). It follows that \(J\) is a continuous martingale starting at zero and \([J]_t=R^{-1/2}R(R^{-1/2})^{\mathsf T}t=t\,\mathrm{Id}_d\).

For completeness, this bracket identity implies the Brownian property as follows. Fix \(u\in\mathbb{R}^d\) and define \(Z_t=\exp(iu^{\mathsf T}J_t+\tfrac12\lVert u\rVert^2t)\). Ito's formula applied to the function \(f(t,x)=\exp(iu^{\mathsf T}x+\tfrac12\lVert u\rVert^2t)\) gives \(f_t=\tfrac12\lVert u\rVert^2f\), gradient \(iu f\), and Hessian \(-uu^{\mathsf T}f\). The time derivative and the \(\tfrac12\) Hessian correction cancel because \(d[J]_t=\mathrm{Id}_d\,dt\), so \(Z\) is a local martingale. On any fixed interval \([0,T]\), \(|Z_t|=\exp(\tfrac12\lVert u\rVert^2t)\le\exp(\tfrac12\lVert u\rVert^2T)\); a bounded local martingale is a true martingale. Its conditional-expectation identity yields \(\mathbb{E}[\exp(iu^{\mathsf T}(J_t-J_s))\mid\mathcal{F}^Y_s]=\exp(-\tfrac12\lVert u\rVert^2(t-s))\). This is the characteristic function of a centered Gaussian vector with covariance \((t-s)\mathrm{Id}_d\), and it is deterministic. Uniqueness of characteristic functions therefore gives Gaussian increments independent of \(\mathcal{F}^Y_s\). Together with continuity and \(J_0=0\), these are the defining properties of standard Brownian motion in the observation filtration. \(\square\)

The proof uses the Brownian characterization through its conditional characteristic functions, rather than assuming that the innovation is Brownian because it has the right covariance. A covariance calculation alone would not establish independence of future increments from the observation past.

Worked Models

Interpretation and a Common Pitfall

The innovation is a martingale because, once the observation past is fixed, the expected future observation drift equals the expected future predicted drift. Its quadratic variation, in contrast, is determined by the observation noise: subtracting a continuous time integral changes neither quadratic variation nor covariation. These two facts together explain why normalization produces Brownian motion. The signal may be uncertain and random; it is the conditional prediction that makes the residual a martingale in the observation filtration.

A common error is to call any whitening the normalized innovation without checking the definition. For positive definite \(R\), the convention here is specifically \(J=R^{-1/2}I\), with the symmetric positive definite inverse square root. If \(I=L\mathbf{B}\), it is true that \(L^{-1}I=\mathbf{B}\); it does not follow that \(R^{-1/2}I=\mathbf{B}\). The two agree only when the corresponding matrices agree. In the vector example, \(R^{-1/2}I\) is a rotation of \(\mathbf{B}\), while \(L^{-1}I\) is \(\mathbf{B}\) itself. Both are Brownian motions, but they are distinct processes.

Another error is to infer the Brownian property from the bracket alone without first establishing the martingale property in the observation filtration. A continuous process can have the right quadratic variation and still fail to be Brownian. Here the conditional-expectation argument establishes the martingale property, and the bracket plus the Brownian characterization establishes the required increment distributions and independence.