Means and Covariances Under Affine Maps

Why Affine Transformations Need Their Own Moment Calculation

A vector of measurements is often recoded, combined, or shifted before it is analyzed. For example, a matrix may convert measurements into weighted summaries, change units, or produce fewer outputs than inputs. To describe the resulting vector, we need to know how its location and variability change.

Let \(X\) be a random vector in \(\mathbb{R}^p\), with mean vector \(\mu\) and covariance matrix \(\Sigma\). Let \(A\) be a fixed \(m\times p\) matrix and \(b\) a fixed vector in \(\mathbb{R}^m\). The transformed vector is \(Y=AX+b\). The formulas for its mean and covariance follow from the definitions; they do not require \(X\) to be Gaussian. When \(X\) is Gaussian, the affine-image result gives the distributional context, and the calculations here identify its transformed mean and covariance.

Finite second moments mean \(E[\|X\|^2]<\infty\), where \(\|\cdot\|\) is the Euclidean norm. This condition ensures that the means and covariance entries used below exist and are finite. Because \(A\) and \(b\) are fixed and finite-dimensional, \(Y=AX+b\) also has finite second moments.

The Affine Moment Formulas

Proof. Since \(A\) is deterministic, expectation distributes across its rows and across addition of the fixed vector \(b\). Therefore $$ E[Y]=E[AX+b]=AE[X]+b=A\mu+b. $$

To calculate covariance, first center \(Y\) using its mean: $$ Y-E[Y]=AX+b-(A\mu+b)=A(X-\mu). $$ The shift cancels. Applying the definition of covariance and pulling the fixed matrices outside the expectation gives $$ \begin{aligned} \operatorname{Cov}(Y) &=E\left[(Y-E[Y])(Y-E[Y])^{\mathsf T}\right]\\ &=E\left[A(X-\mu)(X-\mu)^{\mathsf T}A^{\mathsf T}\right]\\ &=A\,E\left[(X-\mu)(X-\mu)^{\mathsf T}\right]A^{\mathsf T}\\ &=A\Sigma A^{\mathsf T}. \end{aligned} $$ These expectations are well-defined entry by entry because \(X\) has finite second moments. This proves both formulas. \(\square\)

The dimensions provide a useful check: \(A\mu+b\) is an \(m\)-vector, while \(A\Sigma A^{\mathsf T}\) is an \(m\times m\) matrix. The transpose is essential: it makes the right-hand factor \(p\times m\), so the product is dimensionally valid. The result also shows that adding \(b\) changes the mean but does not change covariance.

Reading the Covariance Formula Entry by Entry

Write \(a_i^{\mathsf T}\) for row \(i\) of \(A\). The \(i\)th transformed coordinate is \(Y_i=a_i^{\mathsf T}X+b_i\). For output coordinates \(i\) and \(j\), the corresponding covariance entry is $$ \operatorname{Cov}(Y_i,Y_j)=a_i^{\mathsf T}\Sigma a_j. $$ If \(a_{ik}\) is entry \(k\) of row \(i\), expansion gives $$ \operatorname{Cov}(Y_i,Y_j)=\sum_{k=1}^{p}\sum_{\ell=1}^{p}a_{ik}\Sigma_{k\ell}a_{j\ell}. $$ Thus each output covariance depends both on the weights in the two rows and on the covariances among the input coordinates. Keeping only the input variances would generally omit important terms.

Two Further Consequences

The formula handles a transformation applied in stages just as consistently as one applied at once. This provides a useful independent check on calculations involving several fixed linear recodings.

Derivation. Substitution gives \(W=C(AX+b)+d=(CA)X+(Cb+d)\). Applying the formulas directly yields $$ E[W]=(CA)\mu+(Cb+d)=C(A\mu+b)+d. $$ Applying them in two stages instead gives \(E[Z]=A\mu+b\), so \(E[W]=CE[Z]+d=C(A\mu+b)+d\), the same mean. For covariance, the direct calculation is $$ \operatorname{Cov}(W)=(CA)\Sigma(CA)^{\mathsf T} =C A\Sigma A^{\mathsf T}C^{\mathsf T}. $$ The two-stage calculation gives \(\operatorname{Cov}(Z)=A\Sigma A^{\mathsf T}\), followed by \(\operatorname{Cov}(W)=C\operatorname{Cov}(Z)C^{\mathsf T}\), which is the same matrix. \(\square\)

A second useful implication is that the output covariance must be symmetric and positive semidefinite. Symmetry follows from \(\Sigma^{\mathsf T}=\Sigma\): $$ (A\Sigma A^{\mathsf T})^{\mathsf T}=A\Sigma A^{\mathsf T}. $$ For any \(u\in\mathbb{R}^m\), $$ u^{\mathsf T}A\Sigma A^{\mathsf T}u=(A^{\mathsf T}u)^{\mathsf T}\Sigma(A^{\mathsf T}u)\geq 0, $$ because a covariance matrix is positive semidefinite. This check works whether or not \(A\) is square or invertible.

Common Mistakes and a Practical Check

A common incorrect calculation is to write \(\operatorname{Cov}(AX+b)=A\Sigma+b\), treating the shift as though it were a covariance contribution and omitting the right-hand matrix factor. The correction is to center the transformed vector first: \(AX+b-E[AX+b]=A(X-\mu)\). The shift disappears, and multiplying this centered vector by its transpose produces both \(A\) and \(A^{\mathsf T}\).

Another tempting shortcut is to transform only the diagonal entries of \(\Sigma\). That is generally wrong: when a row of \(A\) combines multiple coordinates, their covariances contribute to the variance of that output. Use the full product \(A\Sigma A^{\mathsf T}\), or verify its entries with the double-sum formula.