Affine Images of Gaussian Vectors

Why Affine Transformations Matter

Many operations on a Gaussian vector amount to taking weighted sums of its coordinates, changing units, adding constants, or keeping only certain linear combinations. These operations are all affine transformations. Their importance is not merely computational: if the original vector is Gaussian, the transformed vector remains Gaussian, even when some coordinates of the result are exact linear combinations of others.

The rank-deficient case deserves care. A Gaussian vector with a singular covariance matrix is still a well-defined probability distribution, but it need not have a density with respect to ordinary volume in its ambient space. The right way to establish normality for every affine image is therefore to use the characteristic function or the scalar-projection definition, not to rely on an invertible change of variables.

Affine Images and the Main Result

Definition: Let \(X\) be a random vector in \(\mathbb{R}^p\), let \(A\) be a fixed \(m\times p\) real matrix, and let \(b\in\mathbb{R}^m\) be fixed. The random vector \(Y=AX+b\) is an affine image of \(X\). The matrix \(A\) need not be square or invertible.

Write \(X\sim N_p(\mu,\Sigma)\), where \(\mu\in\mathbb{R}^p\) and \(\Sigma\) is a symmetric positive semidefinite \(p\times p\) matrix. Positive semidefiniteness, rather than positive definiteness, allows the Gaussian law of \(X\) itself to be singular. In this notation the characteristic function is \(\varphi_X(s)=\exp\{i s^{\mathsf T}\mu-\tfrac12s^{\mathsf T}\Sigma s\}\), for \(s\in\mathbb{R}^p\).

Theorem: If \(X\sim N_p(\mu,\Sigma)\), \(A\) is any fixed \(m\times p\) real matrix, and \(b\in\mathbb{R}^m\) is fixed, then \(Y=AX+b\sim N_m(A\mu+b,A\Sigma A^{\mathsf T})\). This conclusion holds whether or not \(\Sigma\) is singular and whether or not \(A\) has full rank.

The matrix \(A\Sigma A^{\mathsf T}\) is an allowable Gaussian covariance parameter: it is symmetric, and for every \(t\in\mathbb{R}^m\), \(t^{\mathsf T}A\Sigma A^{\mathsf T}t=(A^{\mathsf T}t)^{\mathsf T}\Sigma(A^{\mathsf T}t)\geq 0\). Here the parameter formula identifies the resulting Gaussian law; deriving means and covariances of affine transformations is a separate calculation.

Proof by Characteristic Functions

The characteristic function of an \(m\)-dimensional random vector \(Y\) at \(t\in\mathbb{R}^m\) is \(E[\exp(i t^{\mathsf T}Y)]\). Since \(t^{\mathsf T}(AX+b)=(A^{\mathsf T}t)^{\mathsf T}X+t^{\mathsf T}b\), the affine transformation gives the following calculation:

\begin{aligned} \varphi_Y(t) &=E\!\left[\exp\{i t^{\mathsf T}(AX+b)\}\right]\\ &=\exp(i t^{\mathsf T}b)\, E\!\left[\exp\{i(A^{\mathsf T}t)^{\mathsf T}X\}\right]\\ &=\exp(i t^{\mathsf T}b)\, \exp\!\left\{i(A^{\mathsf T}t)^{\mathsf T}\mu -\frac12(A^{\mathsf T}t)^{\mathsf T}\Sigma(A^{\mathsf T}t)\right\}\\ &=\exp\!\left\{i t^{\mathsf T}(A\mu+b) -\frac12t^{\mathsf T}(A\Sigma A^{\mathsf T})t\right\}. \end{aligned}

This is the characteristic function of the \(m\)-dimensional Gaussian law with parameter vector \(A\mu+b\) and parameter matrix \(A\Sigma A^{\mathsf T}\). The latter is positive semidefinite by the quadratic-form calculation above, so this Gaussian law is defined even if its matrix is singular. Characteristic functions determine probability distributions uniquely. Therefore \(Y\) has the stated multivariate normal distribution. \(\square\)

The Scalar-Projection View

There is also a direct interpretation using the characterization of multivariate normality by scalar projections: a vector is multivariate normal if every fixed real linear combination of its coordinates is univariate normal. For \(t\in\mathbb{R}^m\), the projection of the affine image is \(t^{\mathsf T}Y=(A^{\mathsf T}t)^{\mathsf T}X+t^{\mathsf T}b\). The first term is a linear combination of the coordinates of the Gaussian vector \(X\), hence is univariate normal; adding the constant \(t^{\mathsf T}b\) preserves univariate normality. This holds for every \(t\), including \(t=0\), whose projection is the constant zero. Thus all scalar projections of \(Y\) are normal, which proves that \(Y\) is multivariate normal. In particular, the argument does not require \(A^{\mathsf T}t\) to be nonzero or \(A\) to be invertible.

The characteristic-function proof also supplies the parameters of the resulting law directly. The projection argument emphasizes the underlying reason for closure: every linear combination after transformation is still a linear combination of the original Gaussian coordinates, plus a constant.

Worked Example: A Scalar Linear Combination

Suppose \(X\sim N_2(\mu,\Sigma)\), where \(\mu=(1,-1)^{\mathsf T}\) and \(\Sigma=\begin{pmatrix}4&1\\1&2\end{pmatrix}\). Let \(Y=X_1+2X_2+3\). Identify its distribution.

Write \(A=\begin{pmatrix}1&2\end{pmatrix}\) and \(b=3\). The image has dimension one. Its Gaussian location parameter is \(A\mu+b=1(1)+2(-1)+3=2\).

For the variance parameter, first calculate \(\Sigma A^{\mathsf T}=\begin{pmatrix}4&1\\1&2\end{pmatrix}\begin{pmatrix}1\\2\end{pmatrix}=\begin{pmatrix}6\\5\end{pmatrix}\). Consequently, \(A\Sigma A^{\mathsf T}=\begin{pmatrix}1&2\end{pmatrix}\begin{pmatrix}6\\5\end{pmatrix}=6+10=16\). Thus \(Y\sim N(2,16)\). The constant 3 shifts the location parameter, while the weighted combination determines the variance parameter.

Worked Example: A Rank-Deficient Image

Let \(X\sim N_2\!\left(\begin{pmatrix}0\\0\end{pmatrix},I_2\right)\) and define \(Y=AX+b\), with \(A=\begin{pmatrix}1&1\\2&2\end{pmatrix}\) and \(b=\begin{pmatrix}1\\-1\end{pmatrix}\). Find the distributional parameters and describe the degeneracy.

The two rows of \(A\) are multiples, so \(A\) has rank one, not two. The transformed location parameter is \(A\begin{pmatrix}0\\0\end{pmatrix}+b=\begin{pmatrix}1\\-1\end{pmatrix}\). Because \(\Sigma=I_2\), the transformed covariance parameter is \[ A I_2 A^{\mathsf T}=AA^{\mathsf T} =\begin{pmatrix}2&4\\4&8\end{pmatrix}. \]

Therefore \(Y\sim N_2\!\left(\begin{pmatrix}1\\-1\end{pmatrix}, \begin{pmatrix}2&4\\4&8\end{pmatrix}\right)\). The second coordinate is exactly twice the first minus three: \(Y_1=X_1+X_2+1\) and \(Y_2=2(X_1+X_2)-1=2Y_1-3\). Thus the resulting vector is concentrated on a line and its covariance parameter is singular. It is nevertheless a multivariate normal vector by the theorem.

Worked Example: Mapping Three Coordinates to Two

Let \(X=(X_1,X_2,X_3)^{\mathsf T}\sim N_3(0,I_3)\), and set \[ A=\begin{pmatrix}1&0&1\\0&2&-1\end{pmatrix}, \qquad b=\begin{pmatrix}0\\3\end{pmatrix}, \qquad Y=AX+b. \] Find the law of \(Y\), and check whether its covariance parameter is singular.

The location parameter is \(A0+b=(0,3)^{\mathsf T}\). Since the coordinates of \(X\) have identity covariance, multiplication gives \[ AA^{\mathsf T} =\begin{pmatrix}1&0&1\\0&2&-1\end{pmatrix} \begin{pmatrix}1&0\\0&2\\1&-1\end{pmatrix} =\begin{pmatrix}2&-1\\-1&5\end{pmatrix}. \]

It follows that \(Y\sim N_2\!\left(\begin{pmatrix}0\\3\end{pmatrix}, \begin{pmatrix}2&-1\\-1&5\end{pmatrix}\right)\). The determinant of the parameter matrix is \(2(5)-(-1)(-1)=9>0\), so it is nonsingular. This example contrasts with the preceding one: a non-square transformation can produce a nonsingular two-dimensional law when its two rows are linearly independent.

Successive Affine Transformations

An affine operation may be followed by another affine operation, for example when a statistical procedure first forms contrasts and then rescales or shifts them. Such a sequence still has the form of one affine map.

Result: Let \(U=AX+b\), and then let \(V=CU+d\), with fixed compatible matrices and vectors. If \(X\) is multivariate normal, then \(V\) is multivariate normal, and its affine representation in terms of \(X\) is \(V=(CA)X+(Cb+d)\).

To verify the representation, substitute the expression for \(U\): \[ V=CU+d=C(AX+b)+d=(CA)X+(Cb+d). \] The matrix \(CA\) and vector \(Cb+d\) are fixed and have the required dimensions. The affine-image theorem therefore applies to \(X\) and this single combined map, proving that \(V\) is multivariate normal. The argument works regardless of the ranks of \(A\) and \(C\). \(\square\)

Worked Example: Combining Two Transformations

Let \(X\sim N_2\!\left(\begin{pmatrix}0\\1\end{pmatrix},I_2\right)\). First set \(U=AX+b\), where \(A=\begin{pmatrix}1&1\\1&-1\end{pmatrix}\) and \(b=\begin{pmatrix}1\\0\end{pmatrix}\). Then set \(V=CU+d\), where \(C=\begin{pmatrix}2&0\end{pmatrix}\) and \(d=-1\). Find \(V\)'s distribution by combining the maps.

The combined matrix is \(CA=\begin{pmatrix}2&0\end{pmatrix}\begin{pmatrix}1&1\\1&-1\end{pmatrix}=\begin{pmatrix}2&2\end{pmatrix}\). The combined constant is \(Cb+d=\begin{pmatrix}2&0\end{pmatrix}\begin{pmatrix}1\\0\end{pmatrix}-1=2-1=1\). Thus \(V=2X_1+2X_2+1\).

Its location parameter is \(\begin{pmatrix}2&2\end{pmatrix}\begin{pmatrix}0\\1\end{pmatrix}+1=2+1=3\). Its variance parameter is \(\begin{pmatrix}2&2\end{pmatrix}I_2\begin{pmatrix}2\\2\end{pmatrix}=4+4=8\). Hence \(V\sim N(3,8)\), as follows from the single combined affine map.

Common Mistake: Requiring an Inverse

A tempting but incorrect argument is to treat \(Y=AX+b\) as a change of variables and write a transformed density using a determinant and \(A^{-1}\). This reasoning requires a square invertible matrix \(A\), so it does not apply to rectangular or rank-deficient maps. In the rank-deficient example, for instance, the two coordinates of \(Y\) lie on a line: there is no ordinary two-dimensional density to obtain by that inverse-matrix formula.

The correction is to use the characteristic function or scalar projections. Both arguments establish normality without an inverse and remain valid when the image has lower dimension than its ambient space. Singular Gaussian laws are not exceptions to the affine-image result; they are one of the reasons the result must be stated without a full-rank condition.

Key takeaway: Every fixed affine transformation of a multivariate normal vector is multivariate normal. The matrix and vector in the transformation determine the resulting Gaussian parameters, and no invertibility assumption is needed.

Check Your Understanding

Use the affine-image theorem and the calculations in this tutorial to answer the following.

If \(X\sim N_p(\mu,\Sigma)\), what are the dimensions and positive-semidefinite condition required of \(A\Sigma A^{\mathsf T}\) when \(A\) is \(m\times p\)?
Why does the characteristic-function proof not require \(A\) to be invertible?
For a rank-deficient matrix \(A\), can \(AX+b\) still be multivariate normal? Explain why singularity does not contradict the definition.
If \(U=AX+b\) and \(V=CU+d\), write the single affine representation of \(V\) in terms of \(X\).
Why is an inverse-matrix density transformation an unsuitable general proof of the affine-image theorem?