From Gaussian Vectors to Gaussian Processes
A stochastic process is a family of random variables indexed by time. To describe its distribution fully at a finite collection of times, one must specify their joint law, not just the separate law at each time. Gaussian processes make this description especially structured: every finite vector of values is multivariate Gaussian, and its law is determined by its mean vector and covariance matrix.
Let \((\Omega,\mathcal{F},\mathbb{P})\) be a probability space, let the time set be \(T=[0,\infty)\), and let \(S=\mathbb{R}\). A real-valued process \((X_t)_{t\in T}\) is a collection of real-valued random variables on this space. Its finite-dimensional distributions are the joint distributions of vectors \((X_{t_1},\ldots,X_{t_d})\), for every positive integer \(d\) and every choice of times \(t_1,\ldots,t_d\in T\). The times need not be distinct.
For a Gaussian process, define its mean function \(m:T\to\mathbb{R}\) and covariance function \(K:T\times T\to\mathbb{R}\) by \(m(t)=\mathbb{E}[X_t]\) and \(K(s,t)=\operatorname{Cov}(X_s,X_t)\). For selected times \(t_1,\ldots,t_d\), the mean vector is \(\boldsymbol{m}=(m(t_1),\ldots,m(t_d))^\mathsf{T}\), and the covariance matrix is \(\boldsymbol{\Sigma}\) with entries \(\Sigma_{ab}=K(t_a,t_b)\). The finite vector at those times has the multivariate Gaussian law with those parameters. This statement includes singular covariance matrices: one cannot assume a density exists.
The following result explains why these two functions are enough. The Gaussian-vector fact used in its proof is standard: a multivariate Gaussian vector with mean \(\boldsymbol{m}\) and covariance matrix \(\boldsymbol{\Sigma}\), including a singular one, has characteristic function \(\exp(i\boldsymbol{u}^{\mathsf T}\boldsymbol{m}-\tfrac12\boldsymbol{u}^{\mathsf T}\boldsymbol{\Sigma}\boldsymbol{u})\). Uniqueness of characteristic functions is also a standard result.
Proof. Fix \(d\ge1\) and times \(t_1,\ldots,t_d\). By the definition of a Gaussian process, the vector \(\boldsymbol{X}=(X_{t_1},\ldots,X_{t_d})^\mathsf{T}\) is multivariate Gaussian. Its mean vector has entries \(m(t_a)\), and its covariance matrix has entries \(K(t_a,t_b)\). Thus its characteristic function at any \(\boldsymbol{u}\in\mathbb{R}^d\) is \(\exp(i\boldsymbol{u}^{\mathsf T}\boldsymbol{m}-\tfrac12\boldsymbol{u}^{\mathsf T}\boldsymbol{\Sigma}\boldsymbol{u})\). If two Gaussian processes have the same \(m\) and \(K\), this expression is identical for their vectors at the chosen times. Uniqueness of characteristic functions implies that the vectors have the same distribution. Since the finite list of times was arbitrary, all their finite-dimensional distributions agree. \(\square\)
Restrictions on the Mean and Covariance
Not every pair of functions can serve as the mean and covariance of a Gaussian process. A covariance function is symmetric, since covariance is symmetric, and it must satisfy a positivity condition. For any finite list of times, its covariance matrix must be positive semidefinite: for every real column vector \(\boldsymbol{c}\), the quadratic form is the variance of a linear combination of process values, and variances cannot be negative.
Proof. Covariance is symmetric, so \(K(s,t)=K(t,s)\). For the positivity claim, set \(Y=\sum_{a=1}^d c_a X_{t_a}\). Linearity of expectation gives \(\mathbb{E}[Y]=\sum_a c_a m(t_a)\). Expanding the variance using the definition of covariance gives \(\operatorname{Var}(Y)=\sum_{a=1}^d\sum_{b=1}^d c_a c_b\operatorname{Cov}(X_{t_a},X_{t_b})=\sum_{a=1}^d\sum_{b=1}^d c_a c_bK(t_a,t_b)\). Because \(Y\) has a finite second moment, its variance is nonnegative. This proves the stated inequality. \(\square\)
Conversely, if a function \(m\) is real-valued and \(K\) is symmetric and positive semidefinite in this sense, then the Gaussian vectors specified by \(m\) and \(K\) are consistent when times are reordered or coordinates are omitted. The standard Kolmogorov extension theorem then gives a process with those finite-dimensional distributions on the product sigma-field; the theorem’s proof is not given here. This existence statement concerns the coordinate distributions. It does not, by itself, assert that the resulting process has continuous paths.
Worked Example: Recovering a Bivariate Gaussian Law
Suppose \(Z_1\) and \(Z_2\) are independent standard normal random variables, and define \(X=1+2Z_1\) and \(Y=-1+Z_1+\sqrt{2}Z_2\). Find the mean vector and covariance matrix, then verify the covariance matrix is positive definite.
First, \(\mathbb{E}[Z_1]=\mathbb{E}[Z_2]=0\), so \(\mathbb{E}[X]=1+2(0)=1\) and \(\mathbb{E}[Y]=-1+0+\sqrt{2}(0)=-1\). Independence gives \(\operatorname{Cov}(Z_1,Z_2)=0\), while each standard normal has variance \(1\). Therefore, \(\operatorname{Var}(X)=4\operatorname{Var}(Z_1)=4\), and \(\operatorname{Var}(Y)=\operatorname{Var}(Z_1)+2\operatorname{Var}(Z_2)+2\sqrt{2}\operatorname{Cov}(Z_1,Z_2)=1+2+0=3\).
For the covariance between the variables, constants do not affect covariance, so \(\operatorname{Cov}(X,Y)=\operatorname{Cov}(2Z_1,Z_1+\sqrt{2}Z_2)=2\operatorname{Var}(Z_1)+2\sqrt{2}\operatorname{Cov}(Z_1,Z_2)=2\). Thus the mean vector and covariance matrix are \(\boldsymbol{m}=(1,-1)^\mathsf{T}\) and \(\boldsymbol{\Sigma}=\begin{bmatrix}4&2\\2&3\end{bmatrix}\).
The determinant is \(4\cdot3-2\cdot2=8\), and the upper-left entry is \(4>0\), so the symmetric matrix is positive definite. As a direct check, for any real \(a,b\), \(4a^2+4ab+3b^2=4(a+\tfrac12b)^2+2b^2\), which is positive unless \(a=b=0\). Finally, \((X,Y)\) is Gaussian because it is an affine transformation of the Gaussian vector \((Z_1,Z_2)\). Its joint law is therefore specified by the mean and covariance just computed.
Reading a Process Through Its Kernel
For a process, covariance entries are evaluations of one function at pairs of times. The kernel must produce a valid covariance matrix no matter which finite list of times is chosen. When a process is built from a small number of Gaussian random variables, this can be checked directly: each value is a linear combination of those variables, so every finite vector is an affine transformation of a Gaussian vector.
Worked Example: A Linear Gaussian Process
Let \(Z\) and \(W\) be independent standard normal random variables on a probability space, and define \(X_t=Z+tW\) for \(t\ge0\). Find \(m(t)\), \(K(s,t)\), and the joint law at times \(0,1,2\).
For each \(t\), \(m(t)=\mathbb{E}[Z+tW]=0\). For \(s,t\ge0\), \(K(s,t)=\operatorname{Cov}(Z+sW,Z+tW)=\operatorname{Var}(Z)+st\operatorname{Var}(W)+(s+t)\operatorname{Cov}(Z,W)=1+st\). The process is Gaussian: for any finite times \(t_1,\ldots,t_d\), its value vector equals \((Z,\ldots,Z)^\mathsf{T}+(t_1,\ldots,t_d)^\mathsf{T}W\), a linear transformation of the Gaussian vector \((Z,W)^\mathsf{T}\).
At times \(0,1,2\), the mean vector is \((0,0,0)^\mathsf{T}\). Evaluating \(K(s,t)=1+st\) at each pair gives \(\boldsymbol{\Sigma}=\begin{bmatrix}1&1&1\\1&2&3\\1&3&5\end{bmatrix}\). For example, the entry in row 2, column 3 is \(K(1,2)=1+(1)(2)=3\). The vector is singular: \(X_2-2X_1+X_0=(Z+2W)-2(Z+W)+Z=0\). This dependence is consistent with the covariance matrix being positive semidefinite rather than positive definite. The matrix also has the required quadratic form: for real \(c_0,c_1,c_2\), its quadratic form equals \((c_0+c_1+c_2)^2+(c_1+2c_2)^2\), which is nonnegative.
These calculations also show why the covariance function need not be strictly positive definite. A process can have exact linear relations among its values, and then some finite covariance matrices are singular. The correct general condition is positive semidefiniteness, not invertibility.
Worked Example: Same Finite-Dimensional Laws, Different Paths
Let \(T=[0,\infty)\), take \(\Omega=[0,1]\) with its uniform probability distribution, and let \(U(\omega)=\omega\). Define \(X_t(\omega)=0\) for all \(t,\omega\), and \(Y_t(\omega)=\mathbf{1}_{\{U=t\}}\). Compare their finite-dimensional laws and their paths.
For each fixed \(t\), the event \(\{U=t\}\) has probability \(0\). Therefore \(\mathbb{P}(Y_t=0)=1\), so \(\mathbb{E}[Y_t]=0\) and \(\operatorname{Var}(Y_t)=0\). For any fixed finite list \(t_1,\ldots,t_d\), the probability that \(U\) equals at least one of these times is at most the sum of \(d\) zero probabilities, hence is zero. Thus \((Y_{t_1},\ldots,Y_{t_d})=(0,\ldots,0)\) almost surely. The same is true for \(X\), so both processes have identical finite-dimensional distributions. In particular, both are Gaussian processes with mean function \(0\) and covariance function \(K(s,t)=0\).
Nevertheless, for every \(\omega\in[0,1]\), the path \(t\mapsto Y_t(\omega)\) equals \(1\) at \(t=\omega\) and \(0\) at every other time. It is discontinuous at \(t=\omega\), whereas every path of \(X\) is constant and continuous. This example does not contradict equality of finite-dimensional laws: a fixed finite list of times almost surely misses the random exceptional time. Extra conditions are needed to draw conclusions about path regularity.
Common Pitfalls and What the Parameters Do Not Say
A frequent mistake is to infer a joint law from the one-dimensional marginals alone. Even when every \(X_t\) is a standard normal random variable, the dependence between different times still matters. For instance, if \(Z\) and \(W\) are independent standard normal variables, the pair \((Z,Z)\) has standard normal marginals and covariance \(1\), while \((Z,W)\) has the same marginals and covariance \(0\). These are different bivariate Gaussian laws. The covariance function records precisely this cross-time dependence.
Another mistake is to use a proposed kernel without checking positivity. For example, \(K(s,t)=-1\) for all \(s,t\) cannot be a covariance function: already at a single time it gives variance \(K(t,t)=-1\), which is impossible. More generally, it is not enough for each proposed variance \(K(t,t)\) to be nonnegative; all finite covariance matrices must be positive semidefinite.
Keep the conclusions in the right scope. The mean and covariance functions of a Gaussian process determine every finite-dimensional distribution, and matching those functions gives matching laws for every finite list of coordinates. They do not automatically provide continuity, differentiability, or any other pathwise property. Nor should the term “Gaussian” be applied merely because each individual \(X_t\) is normal: the definition requires joint Gaussianity for every finite collection of times.
Check Your Understanding
For each question, distinguish statements about finite-dimensional distributions from statements about sample paths.
- State the definition of a Gaussian process, including what is meant by a finite-dimensional distribution.
- For \(X_t=Z+tW\), where \(Z,W\) are independent standard normal variables, calculate \(K(1,3)\) and \(K(3,3)\).
- Why must every finite covariance matrix generated by a covariance function be positive semidefinite?
- Can two Gaussian processes with the same mean and covariance functions have different finite-dimensional distributions? Explain.
- What does the example with \(Y_t=\mathbf{1}_{\{U=t\}}\) show about inferring path continuity from finite-dimensional laws?