Linear Transformations of Gaussian Processes

How Gaussian Processes Change Under Linear Operations

A Gaussian process is often specified through its mean function and covariance kernel. In applications, however, the process of interest may be a filtered, differenced, rescaled, or shifted version of another process. The key fact is that a finite deterministic linear operation preserves Gaussianity. Once this is established, the transformed mean and covariance follow by linearity of expectation and bilinearity of covariance.

Work on a probability space \((\Omega,\mathcal{F},\mathbb{P})\), and let \(T\) be an arbitrary index set. A real-valued process \((X_t)_{t\in T}\) is a Gaussian process if, for every finite selection \(t_1,\ldots,t_d\in T\), the vector \((X_{t_1},\ldots,X_{t_d})\) has a multivariate Gaussian distribution. Singular Gaussian distributions are allowed: the vector may be supported on a lower-dimensional affine subspace.

Write \(m(t)=\mathbb{E}[X_t]\) and \(K(s,t)=\operatorname{Cov}(X_s,X_t)\), assuming these means and covariances are finite. A deterministic linear transformation uses fixed coefficients and a finite number of process values. The coefficients may depend on the output index, and the input indices need not equal the output indices.

Proof. Choose any finite output indices \(v_1,\ldots,v_d\in U\). The vector of corresponding outputs is a deterministic affine transformation of the finite vector consisting of all input values \(X_{u_\ell(v_j)}\), for \(1\le j\le d\) and \(1\le\ell\le r\). That input vector is multivariate Gaussian by the definition of a Gaussian process; repeated input indices cause no problem and may make its covariance singular. A deterministic affine transformation of a Gaussian vector is Gaussian, so \((Y_{v_1},\ldots,Y_{v_d})\) is Gaussian. Since the output indices were arbitrary, \(Y\) is a Gaussian process. Taking expectations term by term gives the stated mean. For the covariance, deterministic constants contribute zero, and bilinearity gives \(\operatorname{Cov}(Y_v,Y_w)=\sum_{\ell,q}a_\ell(v)a_q(w)\operatorname{Cov}(X_{u_\ell(v)},X_{u_q(w)})\), which is the displayed formula. \(\square\)

The theorem applies to finite moving averages, differences, rescalings, and linear combinations at different input locations. For example, a difference uses coefficients \(1\) and \(-1\); its covariance includes the cross-covariance terms between the two input values. Those terms should not be discarded unless they are known to vanish.

Sums of Independent Gaussian Processes

When processes share an index set, they can be added point by point. Independence is a useful sufficient condition for the sum to be Gaussian, because it makes the joint vector of input values Gaussian with a block-diagonal covariance matrix. Independence here means independence of the sigma-fields generated by the processes, not merely zero covariance at matching indices.

Proof. Fix \(t_1,\ldots,t_d\in T\). For each \(\ell\), the vector \((X^{(\ell)}_{t_1},\ldots,X^{(\ell)}_{t_d})\) is Gaussian. Independence of the processes makes these vectors independent across \(\ell\). Their concatenation is therefore Gaussian: its characteristic function is the product of the Gaussian characteristic functions, which is the Gaussian characteristic function with the concatenated mean and block-diagonal covariance. The vector \((Z_{t_1},\ldots,Z_{t_d})\) is a deterministic affine transformation of that concatenated vector, so it is Gaussian. This holds for every finite selection of indices. For the mean, take expectations term by term. For the covariance, expand into all pairs of summands. If \(\ell\ne q\), independence gives \(\operatorname{Cov}(X^{(\ell)}_s,X^{(q)}_t)=0\); the terms with \(\ell=q\) give exactly the displayed sum. The deterministic shift contributes no covariance. \(\square\)

For a single process shifted by a deterministic function \(b\), the rule is especially simple: the new mean is \(m(t)+b(t)\), while the covariance kernel remains \(K(s,t)\). For a weighted sum, each covariance kernel is multiplied by the coefficient at each of its two indices. In particular, multiplying a process by a constant \(c\) multiplies its covariance by \(c^2\), not by \(c\).

Conditions and Common Mistakes

The preservation results rely on finite deterministic linear operations. A random coefficient is not covered: even if \(X\) is Gaussian, multiplying it by an independent random scale need not give a Gaussian variable. For example, if \(X\) is standard normal and \(R\) independently equals \(1\) or \(2\) with probability \(\tfrac12\) each, then \(RX\) is a mixture of two different normal distributions, not a single Gaussian distribution. Infinite sums also require additional convergence and integrability conditions; the finite-sum theorem alone does not justify them.

Independence in the sum theorem is sufficient, not necessary. If two processes are dependent, their sum may still be Gaussian when they are jointly Gaussian, but Gaussian marginal distributions alone do not guarantee this. For instance, let \(G\) be standard normal and let \(R\) be an independent random sign taking values \(1\) and \(-1\) with equal probabilities. Both \(G\) and \(RG\) are standard normal, but \(G+RG\) equals \(0\) with probability \(\tfrac12\) and equals \(2G\) with probability \(\tfrac12\); this mixture is not Gaussian.

Common mistake. For independent processes, a coefficient is applied once to each covariance factor. Thus \(\operatorname{Var}(U+2V)=\operatorname{Var}(U)+4\operatorname{Var}(V)\), not \(\operatorname{Var}(U)+2\operatorname{Var}(V)\). For example, if \(U\) and \(V\) are independent with variances \(1\) and \(4\), respectively, the incorrect calculation gives \(1+2\cdot4=9\). The correct calculation is \(1+2^2\cdot4=17\). The general covariance formula makes the reason explicit: the coefficient at the first index and the coefficient at the second index are both included.

These rules are useful because they let a transformed model be characterized without deriving its full finite-dimensional density. Once its mean and covariance are known, its finite-dimensional Gaussian distributions are determined, including in singular cases. The main checks are to include all cross-covariances among input values, square a constant scale factor for a variance, and distinguish deterministic shifts from random ones.