Affine Maps Preserve Gaussianity
Linear combinations of Gaussian variables occur throughout probability and statistics: weighted averages, contrasts, sums of measurements, and changes of coordinates all have this form. Treating several such quantities at once is efficient because a single matrix calculation gives both their means and their covariance structure. The key fact is that applying any affine map to a Gaussian vector produces another Gaussian vector, even when the map changes the dimension.
Work on a probability space \((\Omega,\mathcal{F},\mathbb{P})\). Throughout, vectors are column vectors. A random vector \(X\) in \(\mathbb{R}^d\) is Gaussian if every scalar linear combination \(u^T X\), for \(u\in\mathbb{R}^d\), has a univariate normal distribution. Its mean vector is \(m=\mathbb{E}[X]\), and its covariance matrix is \(\Sigma=\mathbb{E}[(X-m)(X-m)^T]\). We write \(X\sim N_d(m,\Sigma)\). The covariance matrix may be singular; the definition via linear combinations still applies.
For a fixed \(r\times d\) matrix \(A\) and fixed vector \(b\in\mathbb{R}^r\), the affine transformation of \(X\) is \(Y=AX+b\). The matrix \(A\) can combine, rescale, or discard coordinates, and can also produce more coordinates than \(X\) has. The shift \(b\) changes the mean but not the covariance.
Proof. Linearity of expectation gives \(\mathbb{E}[Y]=A\mathbb{E}[X]+b=Am+b\). Subtract this mean to obtain \(Y-\mathbb{E}[Y]=A(X-m)\). Therefore, $$ \operatorname{Cov}(Y) =\mathbb{E}\!\left[A(X-m)(X-m)^TA^T\right] =A\mathbb{E}\!\left[(X-m)(X-m)^T\right]A^T =A\Sigma A^T. $$ To prove that \(Y\) is Gaussian, take any \(c\in\mathbb{R}^r\). Then \(c^TY=(A^Tc)^TX+c^Tb\), which is a scalar linear combination of the coordinates of \(X\), followed by a constant shift. It is normal by the definition of a Gaussian vector. Its mean and variance are $$ \mathbb{E}[c^TY]=c^T(Am+b),\qquad \operatorname{Var}(c^TY)=c^TA\Sigma A^Tc, $$ using the mean and covariance just computed. Thus every scalar linear combination of \(Y\) is normal, so \(Y\) is Gaussian. If the displayed variance is zero, then \(\mathbb{E}[(c^TY-\mathbb{E}[c^TY])^2]=0\). A nonnegative random variable with expectation zero is zero almost surely, so \(c^TY=c^T(Am+b)\) almost surely. \(\square\)
The theorem gives a practical route to distributions: first find the transformed mean and covariance, then use the fact that each scalar projection is normal. For a particular scalar linear combination, it is often useful to compute its coefficients in the original vector as a separate check.
Proof. By the definition of a Gaussian vector, \(a^TX\) is univariate normal. Its expectation is \(a^T\mathbb{E}[X]=a^Tm\), and its variance is $$ \mathbb{E}\!\left[(a^T(X-m))^2\right] =a^T\mathbb{E}\!\left[(X-m)(X-m)^T\right]a =a^T\Sigma a. $$ When this variance is zero, the expectation of the nonnegative square \((a^TX-a^Tm)^2\) is zero, so that square is zero almost surely. This proves the stated constant case as well. \(\square\)
Worked Calculations
Worked Example: Two Transformed Coordinates
Let \(X=(X_1,X_2)^T\sim N_2(m,\Sigma)\), where $$ m=\begin{bmatrix}1\\-2\end{bmatrix},\qquad \Sigma=\begin{bmatrix}4&1\\1&9\end{bmatrix}. $$ Find the distribution of \(Y=AX+b\) for $$ A=\begin{bmatrix}1&2\\-1&1\end{bmatrix},\qquad b=\begin{bmatrix}2\\-1\end{bmatrix}. $$
First compute the mean. The matrix product is $$ Am= \begin{bmatrix}1&2\\-1&1\end{bmatrix} \begin{bmatrix}1\\-2\end{bmatrix} =\begin{bmatrix}1-4\\-1-2\end{bmatrix} =\begin{bmatrix}-3\\-3\end{bmatrix}, $$ so \(\mathbb{E}[Y]=Am+b=(-1,-4)^T\).
For the covariance, first multiply on the left: $$ A\Sigma= \begin{bmatrix}1&2\\-1&1\end{bmatrix} \begin{bmatrix}4&1\\1&9\end{bmatrix} =\begin{bmatrix}6&19\\-3&8\end{bmatrix}. $$ Then multiply by \(A^T\): $$ A\Sigma A^T =\begin{bmatrix}6&19\\-3&8\end{bmatrix} \begin{bmatrix}1&-1\\2&1\end{bmatrix} =\begin{bmatrix}44&13\\13&11\end{bmatrix}. $$ Thus \(Y\sim N_2\!\left((-1,-4)^T,\begin{bmatrix}44&13\\13&11\end{bmatrix}\right)\).
Check by expanding the covariance. Here \(Y_1=X_1+2X_2+2\) and \(Y_2=-X_1+X_2-1\). Their variances are \(\operatorname{Var}(Y_1)=4+4(1)+4(9)=44\) and \(\operatorname{Var}(Y_2)=4+9-2(1)=11\). For the cross-covariance, expand all four terms: $$ \operatorname{Cov}(Y_1,Y_2) =-\operatorname{Var}(X_1)+\operatorname{Cov}(X_1,X_2) -2\operatorname{Cov}(X_2,X_1)+2\operatorname{Var}(X_2) =-4+1-2+18=13. $$ This agrees with the off-diagonal entries from the matrix product.
For example, \(3Y_1-2Y_2\) has mean \(3(-1)-2(-4)=5\) and variance $$ 9(44)+4(11)+2(3)(-2)(13)=396+44-156=284. $$ Directly, \(3Y_1-2Y_2=5X_1+4X_2+8\), whose mean is \(5(1)+4(-2)+8=5\) and variance \(25(4)+16(9)+2(5)(4)(1)=284\). Therefore \(3Y_1-2Y_2\sim N(5,284)\); both calculations agree.
Worked Example: An Average and a Contrast
Suppose \(X=(X_1,X_2,X_3)^T\sim N_3(m,\Sigma)\), with $$ m=\begin{bmatrix}2\\0\\-1\end{bmatrix},\qquad \Sigma=\begin{bmatrix}4&1&0\\1&1&\tfrac12\\0&\tfrac12&1\end{bmatrix}. $$ Let \(U=(X_1+X_2+X_3)/3\) and \(V=X_1-X_2\). Find their joint Gaussian distribution and their marginal distributions.
The covariance matrix is valid: its leading principal minors are \(4\), \(4(1)-1=3\), and \(4(1-\tfrac14)-1=2\), all positive. Write \((U,V)^T=AX\), where $$ A=\begin{bmatrix}\tfrac13&\tfrac13&\tfrac13\\1&-1&0\end{bmatrix}. $$ The mean is $$ Am=\begin{bmatrix}(2+0-1)/3\\2-0\end{bmatrix} =\begin{bmatrix}\tfrac13\\2\end{bmatrix}. $$
For the first variance, the sum \(X_1+X_2+X_3\) has variance equal to the sum of all entries of \(\Sigma\): \(4+1+1+2(1+0+\tfrac12)=9\). Hence \(\operatorname{Var}(U)=9/9=1\). For the second, \(\operatorname{Var}(V)=4+1-2(1)=3\). For their covariance, expand the terms: $$ \operatorname{Cov}(U,V) =\tfrac13\bigl(4-1+1-1+0-\tfrac12\bigr) =\tfrac13\cdot\tfrac52 =\tfrac56. $$ Thus $$ \begin{bmatrix}U\\V\end{bmatrix} \sim N_2\!\left( \begin{bmatrix}\tfrac13\\2\end{bmatrix}, \begin{bmatrix}1&\tfrac56\\\tfrac56&3\end{bmatrix} \right). $$
In particular, \(U\sim N(\tfrac13,1)\) and \(V\sim N(2,3)\). As a check on the joint covariance calculation, its determinant is \(1\cdot3-(\tfrac56)^2=\tfrac{83}{36}\), consistent with a positive definite covariance matrix for this pair.
Worked Example: A Rotation and a Weighted Sum
Let \(X_1\sim N(1,1)\) and \(X_2\sim N(2,4)\) be independent, and set \(Y_1=X_1+X_2\), \(Y_2=X_1-X_2\). Find the joint mean and covariance of \(Y=(Y_1,Y_2)^T\), then find the distribution of \(2Y_1+Y_2\).
Independence gives \(\operatorname{Cov}(X_1,X_2)=0\), so \(m=(1,2)^T\) and \(\Sigma=\begin{bmatrix}1&0\\0&4\end{bmatrix}\). The transformation matrix is \(A=\begin{bmatrix}1&1\\1&-1\end{bmatrix}\). Therefore $$ Am=\begin{bmatrix}3\\-1\end{bmatrix},\qquad A\Sigma=\begin{bmatrix}1&4\\1&-4\end{bmatrix},\qquad A\Sigma A^T=\begin{bmatrix}5&-3\\-3&5\end{bmatrix}. $$ Thus \(Y\) is Gaussian with mean \((3,-1)^T\) and the displayed covariance matrix.
Now \(2Y_1+Y_2\) has mean \(2(3)+(-1)=5\) and variance \(4(5)+5+2(2)(1)(-3)=20+5-12=13\). Directly, \(2Y_1+Y_2=3X_1+X_2\), so its mean is \(3(1)+2=5\), and, by independence, its variance is \(9(1)+1(4)=13\). Hence \(2Y_1+Y_2\sim N(5,13)\), with the direct calculation verifying the matrix result.
Reading and Checking the Result
The covariance transformation \(A\Sigma A^T\) has a useful coordinate interpretation. Its \((i,j)\) entry is the covariance between row \(i\) of \(A\) applied to \(X\) and row \(j\) of \(A\) applied to \(X\). In particular, the diagonal entries are variances and the off-diagonal entries are covariances. A shift \(b\) does not appear in this formula because subtracting the mean removes the shift.
For any \(c\in\mathbb{R}^r\), the scalar variable \(c^TY\) can be computed either from the transformed vector or from the original vector: $$ c^TY=c^TAX+c^Tb=(A^Tc)^TX+c^Tb. $$ The two routes must give the same mean and variance. This identity is often the fastest way to check a calculation involving one or two linear combinations.
Common mistake. When expanding a covariance, preserve both the signs and the coefficients in every term. In the first worked example, a wrong calculation might write the cross-covariance as \(-4+2-18+1=-19\). That expression has incorrect coefficients and signs. The four terms are \(-\operatorname{Var}(X_1)=-4\), \(\operatorname{Cov}(X_1,X_2)=1\), \(-2\operatorname{Cov}(X_2,X_1)=-2\), and \(2\operatorname{Var}(X_2)=18\). Their sum is \(-4+1-2+18=13\), matching the matrix product. A reliable check is to expand the covariance term by term and compare it with the corresponding entry of \(A\Sigma A^T\).
Finally, a linear combination can have variance zero even when the original vector has positive variances. In that case it is constant almost surely, as shown in the proposition; it is not a nondegenerate normal variable with a density. This distinction matters whenever coefficients cancel random variation. Apart from this degenerate case, the mean and variance completely specify the univariate normal distribution of each linear combination.
Check Your Understanding
Use the affine transformation formulas, and verify each scalar variance by expanding the coefficients in the original vector.
- If \(X\sim N_2((1,0)^T,\begin{bmatrix}1&0\\0&4\end{bmatrix})\), find the distribution of \(2X_1-X_2\).
- For \(Y=AX+b\), explain why \(b\) changes the mean but not the covariance.
- If \(X\sim N_2(m,\Sigma)\) and \(Y=(X_1+X_2,X_1-X_2)^T\), write the matrix \(A\) and express \(\operatorname{Cov}(Y)\) as \(A\Sigma A^T\).
- What does it mean for \(a^T\Sigma a=0\), and what is the distribution of \(a^TX\) in that case?
- For the average and contrast example, verify \(\operatorname{Var}(X_1-X_2)=3\) from the entries of \(\Sigma\).