Checking Ornstein-Uhlenbeck Fits with Residual Diagnostics

What One-Step Residuals Should Reveal

An Ornstein–Uhlenbeck (OU) model specifies how a process changes over each observation interval. If its parameters are correct, the change from the conditional mean, divided by the conditional standard deviation, should behave like a standard normal innovation. Residual diagnostics check whether observed transitions are consistent with those predictions: they examine location, scale, distribution, and serial dependence.

These checks are informative, but they are not automatic proofs that a model is correct. In particular, when parameters are estimated from the same observations being diagnosed, the standardized residuals need not be exactly independent standard normal variables in a finite sample. The model’s predictions provide a reference against which to look for systematic discrepancies.

The Exact Transition and Its Residual

Work on a filtered probability space $(\Omega,\mathcal{F},(\mathcal{F}_t)_{t\ge0},\mathbb{P})$ carrying a Brownian motion $B_t$ relative to the filtration. Consider the OU equation $dX_t=\kappa(\theta-X_t)\,dt+\sigma\,dB_t$, where $\kappa>0$ is the rate of mean reversion, $\theta\in\mathbb{R}$ is the long-run mean, and $\sigma>0$ is the diffusion coefficient. The drift is $\kappa(\theta-X_t)$; the diffusion coefficient is $\sigma$. The drift pulls the process toward $\theta$, while the diffusion creates random variation.

For completeness, the exact transition follows by applying Ito’s formula to $f(t,x)=e^{\kappa t}(x-\theta)$. Its derivatives are $f_t=\kappa e^{\kappa t}(x-\theta)$, $f_x=e^{\kappa t}$, and $f_{xx}=0$, so the Ito correction $\tfrac12 f_{xx}\sigma^2\,dt$ is zero. The resulting process satisfies $d f(t,X_t)=\sigma e^{\kappa t}\,dB_t$. This Ito integral has integrand $H_s=\sigma e^{\kappa s}$, which is progressively measurable and belongs to $\mathcal{L}^2$ on every finite time interval because $\mathbb{E}\int_0^T H_s^2\,ds=\sigma^2\int_0^T e^{2\kappa s}\,ds<\infty$. Therefore, for $h>0$,

X_{t+h}=\theta+e^{-\kappa h}(X_t-\theta) +\sigma\int_t^{t+h}e^{-\kappa(t+h-s)}\,dB_s.

Since the integrand in the last term is deterministic, the integral is independent of $\mathcal{F}_t$ and normally distributed with mean zero and variance $\sigma^2\int_t^{t+h}e^{-2\kappa(t+h-s)}\,ds$. Evaluating that integral gives the exact conditional transition:

X_{t+h}\mid\mathcal{F}_t\sim N\left(\theta+e^{-\kappa h}(X_t-\theta), \frac{\sigma^2}{2\kappa}(1-e^{-2\kappa h})\right).

Definition: For observations $x_0,x_1,\ldots,x_n$ at equally spaced times $t_i=ih$, and fitted parameters $\widehat\kappa>0$, $\widehat\theta\in\mathbb{R}$, and $\widehat\sigma>0$, the standardized one-step residual is $$ \widehat z_i= \frac{x_i-\widehat\theta-e^{-\widehat\kappa h}(x_{i-1}-\widehat\theta)} {\widehat\sigma\sqrt{(1-e^{-2\widehat\kappa h})/(2\widehat\kappa)}}, \qquad i=1,\ldots,n. $$ The denominator is the fitted conditional standard deviation, not the short-time approximation $\widehat\sigma\sqrt h$. For unequal observation gaps $h_i=t_i-t_{i-1}>0$, use $h_i$ in the corresponding numerator and denominator.

Why the Ideal Innovations Are Independent Standard Normals

Theorem: Suppose the OU equation holds with fixed parameters $\kappa>0$, $\theta\in\mathbb{R}$, and $\sigma>0$. For deterministic observation times $0=t_0<t_1<\cdots<t_n$, define $$ Z_i= \frac{X_{t_i}-\theta-e^{-\kappa h_i}(X_{t_{i-1}}-\theta)} {\sigma\sqrt{(1-e^{-2\kappa h_i})/(2\kappa)}}, \qquad h_i=t_i-t_{i-1}>0. $$ Then $Z_1,\ldots,Z_n$ are independent $N(0,1)$ random variables.

Proof. The exact transition formula shows that the numerator of $Z_i$ is the Ito integral $\sigma\int_{t_{i-1}}^{t_i}e^{-\kappa(t_i-s)}\,dB_s$. Its integrand is deterministic, progressively measurable, and square-integrable on the finite interval. The integral is therefore normal with mean zero and variance $\sigma^2(1-e^{-2\kappa h_i})/(2\kappa)$; division by its positive standard deviation gives $Z_i\sim N(0,1)$. The integral over $[t_{i-1},t_i]$ is independent of $\mathcal{F}_{t_{i-1}}$, and all earlier residuals $Z_1,\ldots,Z_{i-1}$ are measurable with respect to $\mathcal{F}_{t_{i-1}}$. Thus $Z_i$ is independent of the earlier residuals. Applying this argument successively for $i=1,\ldots,n$ proves mutual independence. The assumptions $h_i>0$, $\kappa>0$, and $\sigma>0$ ensure that each denominator is positive. $\square$

This theorem concerns residuals formed with the true parameters. If the parameters are fitted from the observed data, the fitted values depend on those data, so the theorem does not directly imply that the fitted residuals are exactly independent standard normals. Instead, the theorem motivates the diagnostic benchmarks: a residual sequence should not show substantial systematic departures from zero mean, unit variance, normality, or serial independence.

Worked Diagnostics

Worked Example: Computing One Exact Standardized Residual

Use fitted values $\widehat\kappa=\ln 2$, $\widehat\theta=3$, $\widehat\sigma=\sqrt{8\ln 2/3}$, and $h=1$. Given $x_0=2$ and $x_1=9/2$, calculate the standardized residual.

First, $e^{-\widehat\kappa h}=e^{-\ln 2}=1/2$. The fitted conditional mean is $3+\tfrac12(2-3)=3-\tfrac12=\tfrac52$. The fitted conditional variance is

\widehat\sigma^2\frac{1-e^{-2\widehat\kappa h}}{2\widehat\kappa} =\frac{8\ln 2}{3}\cdot\frac{1-1/4}{2\ln 2} =\frac{8\ln 2}{3}\cdot\frac{3}{8\ln 2} =1.

The fitted standard deviation is therefore $1$, and the observed value $x_1=9/2$ is $9/2-5/2=2$ above the fitted conditional mean. Thus $\widehat z_1=2$. Check: substituting this residual back into the one-step prediction gives $\tfrac52+1\cdot2=\tfrac92=x_1$, as required.

Worked Example: Mean, Variance, and a Strict Threshold

For the residuals $(-1,0,1,0)$, calculate the empirical mean, the variance using divisor $n$, the sample variance using divisor $n-1$, and the count satisfying the strict condition $|\widehat z_i|>1$.

The sum is $-1+0+1+0=0$, so the empirical mean is $0/4=0$. The sum of squared deviations from that mean is $1+0+1+0=2$. The variance with divisor $n=4$ is $2/4=1/2$; the sample variance with divisor $n-1=3$ is $2/3$. For the strict threshold, the absolute residuals are $1,0,1,0$. None is strictly greater than $1$, so the count is $0$. All four satisfy the non-strict condition $|\widehat z_i|\le1$.

Check: the sum of the four residuals is zero, and multiplying the variance with divisor $n$ by $n$ recovers the sum of squared deviations: $4\cdot\tfrac12=2$. Under an ideal standard normal model, the probability of $|Z|\le1$ is $2\Phi(1)-1$, approximately $0.6827$, so the expected count among four independent ideal innovations is approximately $4(0.6827)=2.7308$. Observing four in such a small sample is not, by itself, evidence of model failure.

Worked Example: Detecting Lag-One Patterning

For the ordered residuals $(1,-1,1,-1)$, calculate the empirical mean, variance with divisor $n$, and the lag-one sample autocorrelation defined by $$ \widehat\rho_1= \frac{\sum_{i=2}^n(\widehat z_i-\overline z)(\widehat z_{i-1}-\overline z)} {\sum_{i=1}^n(\widehat z_i-\overline z)^2}. $$

The sum is $1-1+1-1=0$, so $\overline z=0$. The sum of squared deviations is $1+1+1+1=4$, giving variance $4/4=1$. The lag-one products are $(-1)(1)=-1$, $(1)(-1)=-1$, and $(-1)(1)=-1$; their sum is $-3$. Hence $\widehat\rho_1=-3/4$.

Check: the denominator is $4$, the sum of the three lag-one products is $-3$, and their ratio is $-3/4$. The residuals alternate in sign, so the negative autocorrelation has the direction suggested by the data. It is a warning to investigate serial structure, not a stand-alone test conclusion.

Reading the Four Diagnostics

For $n$ ideal independent $N(0,1)$ innovations, the mean and variance have useful reference properties. The following statements concern true-parameter innovations $Z_i$, not automatically residuals computed after fitting.

Proposition: If $Z_1,\ldots,Z_n$ are independent $N(0,1)$ variables and $\overline Z=n^{-1}\sum_{i=1}^n Z_i$, then $\mathbb{E}[\overline Z]=0$, $\operatorname{Var}(\overline Z)=1/n$, and $\mathbb{E}[n^{-1}\sum_{i=1}^n Z_i^2]=1$.

Proof. Each $Z_i$ has mean zero and variance one. Linearity of expectation gives $\mathbb{E}[\overline Z]=n^{-1}\sum_i\mathbb{E}[Z_i]=0$. Independence makes all cross-covariances zero, so $\operatorname{Var}(\overline Z)=n^{-2}\sum_i\operatorname{Var}(Z_i)=n^{-2}n=1/n$. Also, $\mathbb{E}[n^{-1}\sum_i Z_i^2]=n^{-1}\sum_i\mathbb{E}[Z_i^2]=n^{-1}n=1$. These calculations hold for every positive integer $n$, including $n=1$. $\square$

A residual mean far from zero can indicate a mismatch in the fitted conditional means. A variance far from one can point to a scale mismatch, such as an inappropriate diffusion estimate. Neither discrepancy has a universal cutoff: sampling variability matters, and fitting parameters changes the reference distribution.

To inspect distribution, use a normal quantile–quantile plot: order the residuals and compare them with corresponding standard normal quantiles. Strong curvature, skewness, or unusually extreme points can indicate departures from the Gaussian innovation assumption. A histogram can provide a quick visual summary, but with few observations it is coarse. Threshold counts, such as the number satisfying $|\widehat z_i|>2$, should use a stated strict inequality; compare them with the appropriate normal tail probability, while recognizing that a small sample can differ substantially by chance.

For serial dependence, plot residuals in observation order and inspect sample autocorrelations at several lags. Under the ideal model, distinct innovations are independent, so their cross-products have mean zero. A visible pattern or sustained autocorrelation may indicate a misspecified drift, omitted structure, incorrect observation spacing, or parameter-estimation effects. One lag alone is not decisive, especially when many lags are inspected.

Common Pitfall: Using the Short-Time Scale

A frequent error is to standardize by $\widehat\sigma\sqrt h$ even when the observation interval is not sufficiently short for that approximation. The exact conditional variance is $\widehat\sigma^2(1-e^{-2\widehat\kappa h})/(2\widehat\kappa)$. In the first worked example, the short-time denominator would be $\widehat\sigma\sqrt h=\sqrt{8\ln 2/3}$, rather than the exact denominator $1$. Its reciprocal is approximately $0.7355$: indeed, $8\ln 2/3\approx1.8484$, and $1/\sqrt{1.8484}\approx0.7355$. Using this approximation would turn the correct residual $2$ into $2/\sqrt{8\ln 2/3}$, about $1.4711$. The discrepancy is material, so use the exact transition variance for diagnostics.

Residual checks matter because they examine distinct aspects of a stochastic model rather than reducing model assessment to a single fitted parameter or summary statistic. Report how residuals were standardized, whether the conditional variance was exact, which variance convention was used, and which thresholds were strict. Interpret plots and summaries as diagnostic evidence, not as proof that the OU assumptions hold or fail.

Check Your Understanding

Use the exact-transition definitions and distinguish theoretical innovations from fitted residuals.

For $\kappa>0$ and a gap $h>0$, write the exact conditional mean and variance of $X_{t+h}$ given $\mathcal{F}_t$.
Why does the exact-parameter innovation have a standard normal distribution, and what makes innovations from disjoint observation intervals independent?
For residuals $(0,1,1)$, calculate the mean and variance using divisor $n$. How many satisfy the strict condition $|\widehat z_i|>1$?
Give one reason a fitted residual sequence need not be exactly independent standard normal, even if the OU model is appropriate.
What additional information does an ordered residual plot or a lag-one autocorrelation provide beyond a residual mean and variance?