Worked Problems on Gaussian Linear Combinations

From a Gaussian Specification to a Probability

A Gaussian vector is often specified by its mean vector and covariance matrix, while a question concerns a particular weighted sum of its components. The practical work is to translate between those descriptions without losing a covariance term or confusing a variance with a standard deviation. Once the linear combination's mean and variance are known, its probability calculations reduce to standard normal probabilities.

The key fact is that a linear combination of a jointly Gaussian vector is Gaussian. The entries of the covariance matrix determine the variance of that combination: diagonal entries contribute squared-coefficient terms, while off-diagonal entries contribute cross terms. The examples below emphasize both the matrix calculation and the final probability calculation.

Here and below, the second argument of \(N(\cdot,\cdot)\) is a variance. For scalar coefficients \(a_1,\ldots,a_p\), the variance can be written componentwise as \(\sum_{i=1}^p a_i^2\operatorname{Var}(X_i)+2\sum_{i<j}a_i a_j\operatorname{Cov}(X_i,X_j)\). The factor of two appears because a symmetric covariance matrix contains each off-diagonal covariance twice in the matrix product.

Why the Formula Works

The affine-image property of Gaussian vectors gives the Gaussian distribution of \(c+a^\mathsf{T}X\). The mean and variance formulas can also be checked directly from expectation and covariance. This direct calculation is useful because it makes the signs of the cross terms explicit.

To verify the scalar formulas, linearity of expectation gives \(E[c+a^\mathsf{T}X]=c+a^\mathsf{T}E[X]=c+a^\mathsf{T}\mu\). Since subtracting a constant does not affect variance, \(\operatorname{Var}(Y)=E[(a^\mathsf{T}(X-\mu))^2]\). Expanding the product and using \(E[(X-\mu)(X-\mu)^\mathsf{T}]=\Sigma\) yields \(\operatorname{Var}(Y)=a^\mathsf{T}\Sigma a\).

For the cross-covariance, constants again cancel: \(\operatorname{Cov}(Y,Z)=E[(a^\mathsf{T}(X-\mu))(b^\mathsf{T}(X-\mu))]=a^\mathsf{T}\Sigma b\). The pair \((Y,Z)\) is an affine image of \(X\), so it is jointly Gaussian by the affine-image property. These calculations establish the stated means and covariance entries. \(\square\)

When \(a^\mathsf{T}\Sigma a>0\), standardization gives \(P(Y\leq t)=\Phi((t-c-a^\mathsf{T}\mu)/\sqrt{a^\mathsf{T}\Sigma a})\), where \(\Phi\) is the standard normal cumulative distribution function. For \(r<s\), \(P(r<Y<s)=\Phi((s-E[Y])/\sqrt{\operatorname{Var}(Y)})-\Phi((r-E[Y])/\sqrt{\operatorname{Var}(Y)})\). These expressions follow by applying the increasing transformation \(y\mapsto (y-E[Y])/\sqrt{\operatorname{Var}(Y)}\) to the event.

Worked Probability Problems

Two Linear Combinations at Once

Some questions specify more than one linear combination. The covariance formula for two combinations supplies their joint Gaussian parameters, even if the question asks only for their separate distributions. For example, with \(Y=c+a^\mathsf{T}X\) and \(Z=d+b^\mathsf{T}X\), compute \(a^\mathsf{T}\Sigma b\) to determine their covariance. This value is important for describing their joint distribution; marginal means and variances alone do not generally specify their joint law.

The joint covariance matrix is \(\begin{pmatrix}a^\mathsf{T}\Sigma a&a^\mathsf{T}\Sigma b\\b^\mathsf{T}\Sigma a&b^\mathsf{T}\Sigma b\end{pmatrix}\). It may be singular, for example if one combination is an exact linear function of the other. A zero variance for an individual combination means that combination is constant with probability one; it is not a nondegenerate normal variable to be standardized. These cases are consistent with the support properties of Gaussian laws.

Common Mistakes and a Reliable Calculation Order

A common error is to calculate the variance of a weighted sum by adding only the squared-coefficient times variance terms. That is correct only when all relevant covariances are zero. For example, \(\operatorname{Var}(X_1-X_2)=\operatorname{Var}(X_1)+\operatorname{Var}(X_2)-2\operatorname{Cov}(X_1,X_2)\); the minus sign comes from multiplying the coefficients \(1\) and \(-1\). Ignoring that term can change both the distribution and the resulting probability.

A second frequent error is to use the variance where the standard deviation is required. If \(Y\sim N(m,v)\) with \(v>0\), the standardized value of \(t\) is \((t-m)/\sqrt{v}\), not \((t-m)/v\). A third is to subtract cumulative probabilities in the wrong order for an interval: the probability between \(r\) and \(s\), where \(r<s\), is the CDF at \(s\) minus the CDF at \(r\).