Affine Images and Cross-Covariances of Gaussian Blocks

Why affine images of Gaussian blocks matter

Gaussian models are often expressed in terms of several related random vectors, but calculations rarely stop at the original coordinates. A measurement may combine several components, a contrast may subtract one block from another, or a reported quantity may be shifted by a fixed baseline. Each operation changes the mean and covariance in a systematic way. When two transformed quantities are reported together, their cross-covariance is just as important as either marginal covariance.

The key point is that the transformations need not be square or invertible. A matrix can map a long vector to a shorter one, or to a longer one, and different output vectors can have different dimensions. The formulas below apply to all such affine maps. They also include Gaussian vectors with singular covariance matrices; no density or inverse covariance matrix is needed.

Block notation and affine transformations

Let \(X\) be a \(p\)-dimensional random vector and \(Y\) a \(q\)-dimensional random vector. Suppose they are jointly Gaussian, with finite means \(\mu_X=E[X]\) and \(\mu_Y=E[Y]\). Write their covariance matrix in blocks as follows.

Here \(\Sigma_{XX}\) is \(p\times p\), \(\Sigma_{YY}\) is \(q\times q\), and \(\Sigma_{XY}\) is \(p\times q\). Consider affine transformations \(U=AX+a\) and \(V=BY+b\), where \(A\) is \(r\times p\), \(B\) is \(s\times q\), \(a\) is a fixed \(r\)-vector, and \(b\) is a fixed \(s\)-vector.

The constants \(a\) and \(b\) shift the outputs but are not random. The matrix \(A\) determines how coordinates of \(X\) are combined; \(B\) plays the corresponding role for \(Y\). In particular, the cross-covariance of \(U\) and \(V\) has \(r\) rows and \(s\) columns.

The affine transformation theorem

The diagonal blocks are the covariance matrices of the transformed vectors. The upper-right block is \(\operatorname{Cov}(U,V)\), and the lower-left block is its transpose. Notice the order in \(A\Sigma_{XY}B^T\): it yields an \(r\times s\) matrix, as required.

A more general version allows both outputs to use both original blocks. If \(U=AX+CY+a\) and \(V=DX+EY+b\), define \(Z=(X^T,Y^T)^T\), \(H=(A\ \ C)\), and \(K=(D\ \ E)\). Then \(U=HZ+a\), \(V=KZ+b\), and, writing \(\Sigma=\operatorname{Cov}(Z)\),

This form is useful when the outputs are different combinations of the same blocks. The block-specific formulas follow by taking \(H=(A\ \ 0)\) and \(K=(0\ \ B)\).

Proof and derivation

First, the affine image is jointly Gaussian. By the definition of joint Gaussianity, every scalar linear combination of the entries of the output vector must be normally distributed, with a constant allowed as a degenerate normal variable. For arbitrary fixed vectors \(c\in\mathbb{R}^r\) and \(d\in\mathbb{R}^s\),

The first term on the right is a scalar linear combination of a jointly Gaussian vector, so it is normal. Adding the constant \(c^Ta+d^Tb\) preserves normality. Thus every scalar linear combination of \((U^T,V^T)^T\) is normal, proving that the output vector is jointly Gaussian.

Linearity of expectation gives \(E[U]=A\mu_X+a\) and \(E[V]=B\mu_Y+b\). To derive the covariance blocks, subtract these means. Since fixed constants cancel, \(U-E[U]=A(X-\mu_X)\) and \(V-E[V]=B(Y-\mu_Y)\). Therefore,

The matrices \(A\) and \(B\) are fixed, so they can be taken outside the expectation. Applying the same calculation with \(V=U\) gives \(\operatorname{Cov}(U)=A\Sigma_{XX}A^T\); applying it to \(V\) gives \(\operatorname{Cov}(V)=B\Sigma_{YY}B^T\). Finally, covariance symmetry gives \(\operatorname{Cov}(V,U)=\operatorname{Cov}(U,V)^T=B\Sigma_{YX}A^T\). These blocks give the stated joint covariance. \(\square\)

Worked calculations

Common mistake: offsets and matrix orientation

A frequent error is to carry an offset into a covariance formula, as if \(\operatorname{Cov}(AX+a,BY+b)\) included a term involving \(a\) or \(b\). Covariance is computed after subtracting the means, and the offsets cancel: \((AX+a)-E[AX+a]=A(X-E[X])\). Another common error is to write \(A\Sigma_{XY}B\) instead of \(A\Sigma_{XY}B^T\). The latter transpose is essential both for the covariance definition and for the result to have the required \(r\times s\) dimensions.

It is also a mistake to infer that two transformed Gaussian vectors are independent merely because they are constructed separately or because their formulas look different. Their cross-covariance must be examined; independence is not a consequence of separate notation. The calculation of cross-covariance is the reliable first step, while the broader relationship between zero cross-covariance and Gaussian independence requires its own result.

Why the formulas are useful

These formulas let a model be moved into whichever coordinates make a statistical question convenient. A contrast matrix can encode comparisons, a rectangular map can compress measurements, and an offset can place the result on a chosen baseline. The transformed covariance quantifies the variability in those coordinates, while the cross-covariance describes how two resulting summaries co-vary.

The calculations do not require the map to be invertible or the covariance matrix to have an inverse. That matters whenever a transformation discards coordinates or creates redundant outputs. The current formulas still give the correct means and covariances in those settings; questions about the resulting rank, support, and degeneracy are separate.