From Two Functions to Every Finite-Dimensional Law
A Gaussian process may be specified by many different constructions: one may use a collection of normal random variables, while another uses a different set of random inputs. To decide whether these constructions describe the same process law, it helps to have a criterion that does not depend on the particular representation. For Gaussian processes, the criterion is exact: the mean and covariance functions determine the distribution of every finite collection of process values.
Let \(T\) be an index set, and let \((X_t)_{t\in T}\) be real-valued random variables on a probability space \((\Omega,\mathcal{F},\mathbb{P})\). The process is Gaussian if, for every finite list \(t_1,\ldots,t_d\) from \(T\), the vector \((X_{t_1},\ldots,X_{t_d})\) has a multivariate Gaussian distribution. Repeated indices are allowed. Define its mean function \(m:T\to\mathbb{R}\) and covariance function \(C:T\times T\to\mathbb{R}\) by \[ m(t)=\mathbb{E}[X_t],\qquad C(s,t)=\operatorname{Cov}(X_s,X_t) =\mathbb{E}[(X_s-m(s))(X_t-m(t))]. \] These quantities are finite because each coordinate of a Gaussian vector has finite second moment.
The finite-dimensional distribution at \(t_1,\ldots,t_d\) is the joint probability distribution of \((X_{t_1},\ldots,X_{t_d})\). Write \(m_{\boldsymbol{t}}=(m(t_1),\ldots,m(t_d))^\mathsf{T}\), and let \(\Sigma_{\boldsymbol{t}}\) be the \(d\times d\) matrix whose \((a,b)\) entry is \(C(t_a,t_b)\). The Gaussian-vector characteristic-function formula is \[ \mathbb{E}\!\left[e^{i u^\mathsf{T}(X_{t_1},\ldots,X_{t_d})^\mathsf{T}}\right] =\exp\!\left(iu^\mathsf{T}m_{\boldsymbol{t}}-\tfrac12u^\mathsf{T}\Sigma_{\boldsymbol{t}}u\right), \qquad u\in\mathbb{R}^d. \] It holds whether or not \(\Sigma_{\boldsymbol{t}}\) is invertible. Indeed, the scalar \(u^\mathsf{T}(X_{t_1},\ldots,X_{t_d})^\mathsf{T}\) is normal with mean \(u^\mathsf{T}m_{\boldsymbol{t}}\) and variance \(u^\mathsf{T}\Sigma_{\boldsymbol{t}}u\), which gives the displayed formula from the characteristic function of a normal variable.
Proof. First suppose the mean functions and covariance functions agree. Fix any finite list \(t_1,\ldots,t_d\). The two vectors formed by evaluating the processes at this list are Gaussian, and agreement of the functions gives them the same mean vector \(m_{\boldsymbol{t}}\) and covariance matrix \(\Sigma_{\boldsymbol{t}}\). Their characteristic functions therefore agree for every \(u\in\mathbb{R}^d\), by the formula above. The standard Fourier uniqueness theorem for probability measures on \(\mathbb{R}^d\) says that equal characteristic functions imply equal probability distributions. Thus the two vectors have the same distribution. Since the finite list was arbitrary, all finite-dimensional distributions agree. This argument does not require an inverse of \(\Sigma_{\boldsymbol{t}}\), so it also covers singular matrices. Conversely, suppose all finite-dimensional distributions agree. The distributions at a single time agree, so their expectations agree, giving equality of the mean functions. For any \(s,t\in T\), the two-coordinate distributions at \((s,t)\) agree. Their covariances, computed from those distributions and their already equal means, agree as well. Hence the covariance functions agree everywhere. \(\square\)
The theorem explains why specifying a Gaussian process through a construction is often more detail than needed to compare its law with another process. For each finite set of indices, the relevant information is just a vector of means and a covariance matrix. The theorem is a statement about distributions, not equality of the random variables on their probability spaces: the processes being compared need not even be defined on the same space.
Comparing Processes in Practice
Worked Example: Matching Marginals Does Not Match a Joint Law
Let \(T=\{1,2\}\). Let \(Z,Z_1,Z_2\) be standard normal random variables, with \(Z_1\) and \(Z_2\) independent. Define \(X_1=Z,\ X_2=Z\), and \(Y_1=Z_1,\ Y_2=Z_2\). Each coordinate in both processes is standard normal, but do the processes have the same finite-dimensional distributions?
Both mean vectors are \((0,0)^\mathsf{T}\). For \(X\), each variance is \(1\), and \(\operatorname{Cov}(X_1,X_2)=\mathbb{E}[Z^2]=1\). For \(Y\), each variance is \(1\), while independence gives \(\operatorname{Cov}(Y_1,Y_2)=0\). Thus \[ \Sigma_X=\begin{bmatrix}1&1\\1&1\end{bmatrix}, \qquad \Sigma_Y=\begin{bmatrix}1&0\\0&1\end{bmatrix}. \] The covariance matrices differ, so the Gaussian-vector theorem shows the joint distributions differ.
Check by a linear combination. For \(X\), \(X_1-X_2=0\) with probability \(1\), so \(\operatorname{Var}(X_1-X_2)=0\). For \(Y\), independence gives \(\operatorname{Var}(Y_1-Y_2)=\operatorname{Var}(Y_1)+\operatorname{Var}(Y_2)=1+1=2\). The joint laws cannot agree. The single-time distributions were identical, but they did not capture the dependence between the two times.
Worked Example: Two Constructions with the Same Process Law
Let \(T=\{1,2,3\}\), and let \(Z,W\) be independent standard normal random variables. Define \[ (X_1,X_2,X_3)=(Z,W,Z+W),\qquad (Y_1,Y_2,Y_3)=(W,-Z,-Z+W). \] Check whether the two Gaussian processes have the same finite-dimensional distributions.
Each mean is zero. For \(X\), the variances are \(1,1,2\): the last is \(\operatorname{Var}(Z+W)=1+1+2\operatorname{Cov}(Z,W)=2\). The covariances are \(\operatorname{Cov}(X_1,X_2)=0\), \(\operatorname{Cov}(X_1,X_3)=\operatorname{Cov}(Z,Z+W)=1\), and \(\operatorname{Cov}(X_2,X_3)=\operatorname{Cov}(W,Z+W)=1\). Therefore \[ \Sigma_X=\begin{bmatrix}1&0&1\\0&1&1\\1&1&2\end{bmatrix}. \] For \(Y\), the variances are \(1,1,2\). Its covariances are \(\operatorname{Cov}(W,-Z)=0\), \(\operatorname{Cov}(W,-Z+W)=1\), and \(\operatorname{Cov}(-Z,-Z+W)=1\). Hence \[ \Sigma_Y=\begin{bmatrix}1&0&1\\0&1&1\\1&1&2\end{bmatrix}=\Sigma_X. \] Both vectors are Gaussian, since each is a linear transformation of the Gaussian vector \((Z,W)^\mathsf{T}\). They have the same mean vector and covariance matrix, so the theorem gives equality of their joint distributions. On this three-point index set, that joint distribution is the full finite-dimensional law.
Check by expressing the coordinates. The vector \(Y\) is \((X_2,-X_1,-X_1+X_2)\). Its covariance entries computed from \(X_1=Z,\ X_2=W\) are \(1,1,2\) on the diagonal and \(0,1,1\) above it, confirming the matrix calculation. This example does not assert that corresponding coordinates are equal; it asserts that the resulting vectors have the same distribution.
What “The Same Process Law” Means
For an arbitrary index set, one can regard a process as a random element of the product space \(\mathbb{R}^T\), equipped with the sigma-field generated by coordinate evaluations \(x\mapsto x(t)\). A cylinder event specifies that finitely many coordinates lie in a chosen Borel set. The law of a process on this sigma-field is determined by its finite-dimensional distributions. To see the measure-theoretic step, cylinder events form a collection closed under finite intersections, and equality of finite-dimensional distributions gives equality of the two process laws on every cylinder event. The standard \(\pi\)-\(\lambda\) theorem then gives equality on the sigma-field generated by those events. Here the \(\pi\)-\(\lambda\) theorem is the standard measure-uniqueness result: two probability measures agreeing on a generating collection closed under finite intersections agree on its generated sigma-field.
This meaning of process law should be distinguished from a claim about every sample path. With an uncountable index set, a property involving all times need not be determined by the finite-dimensional distributions alone. Additional assumptions about path regularity, and an appropriate path-space sigma-field, matter when such claims are needed.
Worked Example: Identical Finite-Dimensional Laws, Different Paths
Take \(T=[0,1]\). On one probability space let \(X_t=0\) for every \(t\). On another, let \(U\) be uniform on \([0,1]\) and define \(Y_t=\mathbf{1}_{\{U=t\}}\). For each fixed \(t\), \(\mathbb{P}(U=t)=0\), so \(Y_t=0\) almost surely. For any finite list \(t_1,\ldots,t_d\), \[ \mathbb{P}\big((Y_{t_1},\ldots,Y_{t_d})\ne(0,\ldots,0)\big) \leq \sum_{a=1}^{d}\mathbb{P}(U=t_a)=0. \] Thus this vector equals zero almost surely, just like \((X_{t_1},\ldots,X_{t_d})\). In particular, both processes are Gaussian: every finite vector has a degenerate Gaussian distribution. Both have mean function zero and covariance function zero.
Nevertheless, every path of \(X\) is identically zero, while for each outcome \(U=u\), the path \(t\mapsto Y_t\) is \(1\) at \(u\) and \(0\) at all other times. It is discontinuous at \(u\). The process \((\omega,t)\mapsto\mathbf{1}_{\{U(\omega)=t\}}\) is measurable, since the equality set is the inverse image of the closed diagonal under the measurable map \((\omega,t)\mapsto(U(\omega),t)\). The example shows precisely what equality of finite-dimensional distributions does not say: it does not by itself guarantee matching path properties for processes indexed by an uncountable set.
Assumptions, Limits, and a Common Mistake
The Gaussian assumption is essential. Means and covariances alone do not generally identify a distribution. For example, a standard normal variable and a variable that equals \(1\) and \(-1\) with probabilities \(\tfrac12\) each both have mean \(0\) and variance \(1\), but they do not have the same distribution: the second variable takes values only in \(\{-1,1\}\), whereas a standard normal variable assigns probability \(0\) to that set. Thus the theorem applies to Gaussian processes because their finite vectors are Gaussian, not merely because means and covariances happen to be available.
Common mistake. In the first worked example, it would be wrong to argue that the processes have the same law because \(X_1,Y_1\) have the same distribution and \(X_2,Y_2\) have the same distribution. That calculation checks only the two one-dimensional marginals. The omitted calculation is the cross-covariance: it is \(1\) for \(X\) and \(0\) for \(Y\). For a Gaussian process, compare the entire mean vector and covariance matrix for each finite list, not just the separate marginal distributions.
For a practical comparison, fix any finite list of indices, compute the corresponding means and all pairwise covariances, and compare the resulting vector and matrix. If the functions agree at every index and every pair of indices, the theorem covers all finite lists at once. No nonsingularity condition is needed, and a singular covariance matrix simply means that some linear combination of the coordinates has zero variance and is therefore constant almost surely.
Check Your Understanding
Use the Gaussian-process characterization to answer these questions.
- Two Gaussian processes have the same mean function and the same covariance function. What can you conclude about their vectors at any chosen finite list of indices?
- Two Gaussian processes have identical distributions at each single time, but their covariance functions differ at a pair of times. Can their finite-dimensional distributions all agree? Explain.
- Why does the characterization still apply when the covariance matrix of a finite vector is singular?
- In the rotated-construction example, what are the variances of \(X_3\) and \(Y_3\), and what calculation verifies them?
- What feature of the random-spike example is not determined by its finite-dimensional distributions?