Tutorials › Multivariate Statistics (Graduate) › Coordinate Changes in Gaussian Conditional Laws

Home  /  Tutorials  /  Gaussian Geometry and Conditioning

Gaussian Geometry and Conditioning · Tutorial 111 of 975

Coordinate Changes in Gaussian Conditional Laws

Track Gaussian conditional laws through invertible changes of predictor and response coordinates, and verify that the underlying predictions do not change.

Intermediate 10 min read

What You'll Learn

  • Express the Gaussian conditional law using its mean, regression coefficient, and conditional covariance
  • Transform predictor and response means, covariance blocks, and conditional regression coefficients
  • Derive the transformed conditional covariance using the Schur complement
  • Check predictions by mapping transformed conditional means back to the original response coordinates
  • Identify common errors involving inverse transformations and covariance scaling

Why Coordinate Changes Need Care

A Gaussian prediction can be expressed in different coordinate systems. Predictors might be reordered, measured in different units, or represented by linear combinations; the response can also be rescaled or recoded. The numerical entries in a regression coefficient and covariance matrix then change. The prediction itself, however, describes the same conditional distribution once it is translated back to the original response coordinates.

The key is to transform each part of the conditional law consistently. In particular, changing predictor coordinates changes the coefficient that maps predictor deviations to response deviations, while changing response coordinates transforms both the conditional mean and conditional covariance.

Setup and Conditional-Law Notation

Let \(X\) be a \(p\)-dimensional predictor vector and \(Y\) a \(q\)-dimensional response vector, jointly Gaussian with means \(\mu_X\) and \(\mu_Y\). Write their covariance matrix in blocks as follows. Assume \(\Sigma_{XX}\) is positive definite; the conditional covariance below may be positive semidefinite.

$$ \operatorname{Cov}\begin{pmatrix}X\\Y\end{pmatrix} = \begin{pmatrix} \Sigma_{XX} & \Sigma_{XY}\\ \Sigma_{YX} & \Sigma_{YY} \end{pmatrix}, \qquad \Sigma_{YX}=\Sigma_{XY}^{T}. $$

The block Gaussian conditional formula gives a conditional mean that is affine in the observed predictor value, and a conditional covariance that does not depend on that value. Use the following notation for its coefficient and covariance:

$$ B=\Sigma_{YX}\Sigma_{XX}^{-1}, \qquad S=\Sigma_{YY}-\Sigma_{YX}\Sigma_{XX}^{-1}\Sigma_{XY}. $$
Definition: Under the assumptions above, the conditional law of \(Y\) given \(X=x\) is Gaussian with mean \(\mu_Y+B(x-\mu_X)\) and covariance \(S\). The matrix \(B\) maps predictor deviations to conditional response-mean deviations; \(S\) is the conditional covariance matrix.

Now change coordinates using invertible matrices \(A\) and \(C\), and allow shifts \(a\) and \(c\) to represent changes of origin:

$$ X^{*}=AX+a,\qquad Y^{*}=CY+c. $$

The shifts are included because centering variables is common. Setting \(a=0\) and \(c=0\) gives the purely linear reparameterizations in the main focus.

How the Conditional Law Transforms

The transformed means and covariance blocks follow from linearity of expectation and covariance. In particular, \(\mu_{X^{*}}=A\mu_X+a\), \(\mu_{Y^{*}}=C\mu_Y+c\), and the predictor and response covariance blocks become \(A\Sigma_{XX}A^T\), \(C\Sigma_{YY}C^T\), and \(C\Sigma_{YX}A^T\). Because \(A\) and \(C\) are invertible, \(A\Sigma_{XX}A^T\) is positive definite.

Theorem: The conditional law of \(Y^{*}\) given \(X^{*}=x^{*}\) is Gaussian. Its regression coefficient and conditional covariance are $$ B^{*}=CBA^{-1}, \qquad S^{*}=CSC^T. $$ Its conditional mean is $$ E[Y^{*}\mid X^{*}=x^{*}] = C\mu_Y+c+B^{*}(x^{*}-\mu_{X^{*}}). $$

Proof. Since \(A\) is invertible, observing \(X^{*}=x^{*}\) is equivalent to observing \(X=A^{-1}(x^{*}-a)\). Substitute this value into the original conditional mean and then transform the response:

$$ \begin{aligned} E[Y^{*}\mid X^{*}=x^{*}] &=C E[Y\mid X=A^{-1}(x^{*}-a)]+c\\ &=C\mu_Y+c+CB\{A^{-1}(x^{*}-a)-\mu_X\}\\ &=C\mu_Y+c+CBA^{-1}(x^{*}-A\mu_X-a)\\ &=C\mu_Y+c+CBA^{-1}(x^{*}-\mu_{X^{*}}). \end{aligned} $$

For the covariance, conditioning on \(X^{*}\) is equivalent to conditioning on \(X\), because the transformation is invertible. Given \(X\), the response covariance is \(S\); applying \(Y^{*}=CY+c\) transforms it to \(CSC^T\). This establishes both the conditional Gaussian law and the stated covariance. To check the coefficient using the block formula, the transformed predictor covariance has inverse \(A^{-T}\Sigma_{XX}^{-1}A^{-1}\), so

$$ C\Sigma_{YX}A^T(A\Sigma_{XX}A^T)^{-1} =C\Sigma_{YX}\Sigma_{XX}^{-1}A^{-1} =CBA^{-1}. $$

Likewise, the transformed Schur complement is

$$ \begin{aligned} C\Sigma_{YY}C^T &-C\Sigma_{YX}A^T(A\Sigma_{XX}A^T)^{-1}A\Sigma_{XY}C^T\\ &=C\{\Sigma_{YY}-\Sigma_{YX}\Sigma_{XX}^{-1}\Sigma_{XY}\}C^T =CSC^T. \end{aligned} $$

Thus the block conditional formula and the direct change of variables give the same transformed law. \(\square\)

Prediction Is Invariant After Mapping Back

A change of coordinates changes the numerical representation of a conditional mean, not the response value it represents. The inverse response transformation is \(Y=C^{-1}(Y^{*}-c)\). Therefore, the transformed conditional mean, mapped back, must equal the original conditional mean at the corresponding predictor value.

Result: For \(x^{*}=Ax+a\), the conditional mean predictions satisfy $$ C^{-1}\{E[Y^{*}\mid X^{*}=x^{*}]-c\} =E[Y\mid X=x]. $$ The transformed conditional covariance similarly maps back as \(C^{-1}S^{*}C^{-T}=S\).

Proof. By the theorem, \(x^{*}-\mu_{X^{*}}=A(x-\mu_X)\), and \(B^{*}=CBA^{-1}\). Hence

$$ \begin{aligned} C^{-1}\{E[Y^{*}\mid X^{*}=x^{*}]-c\} &=C^{-1}\{C\mu_Y+CBA^{-1}A(x-\mu_X)\}\\ &=\mu_Y+B(x-\mu_X)\\ &=E[Y\mid X=x]. \end{aligned} $$

The covariance identity follows by substituting \(S^{*}=CSC^T\): \(C^{-1}S^{*}C^{-T}=C^{-1}CSC^TC^{-T}=S\). \(\square\)

Worked Examples

Worked Example: Rescaling a Predictor and Response

Suppose \(X\sim N(1,4)\), and \(Y=2+\frac12(X-1)+\varepsilon\), where \(\varepsilon\sim N(0,9)\) is independent of \(X\). Find the conditional law at \(X=5\), then express it using \(X^{*}=(X-1)/2\) and \(Y^{*}=(Y-2)/3\).

At \(X=5\), the original conditional mean is \(2+\frac12(5-1)=4\), and the conditional variance is \(9\). The predictor transform has \(A=\frac12\), \(a=-\frac12\), and the response transform has \(C=\frac13\), \(c=-\frac23\). The original slope is \(B=\frac12\), so the transformed slope is \(B^{*}=CBA^{-1}=\frac13\cdot\frac12\cdot2=\frac13\).

Both transformed means are zero: \(\mu_{X^{*}}=0\) and \(\mu_{Y^{*}}=0\). The observed transformed predictor is \(x^{*}=(5-1)/2=2\), so the transformed conditional mean is \(\frac13(2)=\frac23\). Its conditional variance is \(S^{*}=C^2S=(\frac13)^2(9)=1\). Mapping the mean back gives \(3(\frac23)+2=4\), and mapping the variance back gives \(3^2(1)=9\), as required.

Worked Example: Changing Both Multivariate Coordinate Systems

Let \(X\) have mean \((1,-1)^T\) and covariance \(D=\begin{pmatrix}4&0\\0&1\end{pmatrix}\). Specify the response by \(Y=\mu_Y+B(X-\mu_X)+\varepsilon\), where \(\mu_Y=(2,0)^T\), \(B=\begin{pmatrix}1&2\\-1&1\end{pmatrix}\), and \(\varepsilon\) is independent Gaussian noise with covariance \(S=\begin{pmatrix}1&0\\0&4\end{pmatrix}\). Use \(X^{*}=AX\), \(Y^{*}=CY\), where \(A=\begin{pmatrix}1&1\\0&1\end{pmatrix}\) and \(C=\begin{pmatrix}1&1\\1&-1\end{pmatrix}\). Find the transformed conditional law and compare predictions at \(X=(3,0)^T\).

Both coordinate matrices are invertible. In particular, \(A^{-1}=\begin{pmatrix}1&-1\\0&1\end{pmatrix}\). The transformed means are \(\mu_{X^{*}}=A\mu_X=(0,-1)^T\) and \(\mu_{Y^{*}}=C\mu_Y=(2,2)^T\). The transformed slope is

$$ B^{*}=CBA^{-1} = \begin{pmatrix}1&1\\1&-1\end{pmatrix} \begin{pmatrix}1&1\\-1&2\end{pmatrix} = \begin{pmatrix}0&3\\2&-1\end{pmatrix}. $$

The transformed conditional covariance is

$$ S^{*}=CSC^T = \begin{pmatrix}1&4\\1&-4\end{pmatrix} \begin{pmatrix}1&1\\1&-1\end{pmatrix} = \begin{pmatrix}5&-3\\-3&5\end{pmatrix}. $$

In the original coordinates, \(X-\mu_X=(2,1)^T\), so the conditional mean is \((2,0)^T+B(2,1)^T=(6,-1)^T\). The transformed observation is \(X^{*}=(3,0)^T\), with deviation \((3,1)^T\) from \(\mu_{X^{*}}\). Its transformed conditional mean is \((2,2)^T+B^{*}(3,1)^T=(5,7)^T\). Since \(C^{-1}=\frac12\begin{pmatrix}1&1\\1&-1\end{pmatrix}\), mapping this prediction back gives \(C^{-1}(5,7)^T=(6,-1)^T\). The two coordinate systems agree on the prediction.

Worked Example: Reordering Predictors

Let \(X=(X_1,X_2)^T\) have mean \((0,1)^T\) and covariance \(\Sigma_{XX}=\begin{pmatrix}4&1\\1&2\end{pmatrix}\). Let \(Y\) have mean \(5\), regression coefficient \(B=(1,-2)\), and conditional variance \(S=3\). Reorder the predictors as \(X^{*}=(X_2,X_1)^T\), and calculate the prediction at \(X=(1,2)^T\).

The reordering matrix is \(A=\begin{pmatrix}0&1\\1&0\end{pmatrix}\), and \(A^{-1}=A\). With no response transformation, \(C=1\), so \(B^{*}=BA^{-1}=(-2,1)\). As a block-matrix check, \(\Sigma_{X^{*}X^{*}}=\begin{pmatrix}2&1\\1&4\end{pmatrix}\) has inverse \(\frac17\begin{pmatrix}4&-1\\-1&2\end{pmatrix}\). Also, \(\Sigma_{YX}=B\Sigma_{XX}=(2,-3)\), so \(\Sigma_{YX^{*}}=(-3,2)\). Multiplication gives \((-3,2)\frac17\begin{pmatrix}4&-1\\-1&2\end{pmatrix}=(-2,1)\), as predicted.

The original predictor deviation is \((1,1)^T\), giving conditional mean \(5+(1,-2)(1,1)^T=4\) and variance \(3\). In the reordered coordinates, the deviation from \(\mu_{X^{*}}=(1,0)^T\) is also \((1,1)^T\); the mean is \(5+(-2,1)(1,1)^T=4\). The response was not changed, so its conditional variance remains \(3\).

Common Mistakes and a Practical Check

A common error is to multiply the slope by \(A\) instead of \(A^{-1}\). The slope takes a predictor-coordinate deviation and produces a response-coordinate deviation. Since \(x-\mu_X=A^{-1}(x^{*}-\mu_{X^{*}})\), the transformed slope must contain \(A^{-1}\) on its right. Another error is to leave the conditional covariance unchanged when the response is rescaled. The correct transformation is \(CSC^T\); it is unchanged only for response transformations that preserve that covariance.

These formulas are useful whenever model variables are standardized, predictors are rotated or reordered, or responses are reported in new units. They also provide a diagnostic: a transformed conditional mean mapped back to the original response coordinates must match the original prediction, and the covariance must map back to \(S\). A mismatch usually indicates an incorrect inverse, a missing transpose, or inconsistent handling of the change of origin.

1
Transform the predictor coordinates.
Use \(\mu_{X^{*}}=A\mu_X+a\) and \(\Sigma_{X^{*}X^{*}}=A\Sigma_{XX}A^T\).
2
Transform the conditional mean map.
Use \(B^{*}=CBA^{-1}\), with transformed response mean \(C\mu_Y+c\).
3
Transform and verify the uncertainty.
Use \(S^{*}=CSC^T\), then map the covariance back to check that it equals \(S\).

Check Your Understanding

Use the transformation rules to answer each question.

  1. If \(X^{*}=AX+a\), why does the transformed slope involve \(A^{-1}\) rather than \(A\)?
  2. For \(Y^{*}=CY+c\), how does the conditional covariance change, and what does it become when \(C=2I\)?
  3. If only the predictor coordinates change, what happens to the conditional covariance of the unchanged response?
  4. State how to map a conditional mean prediction in transformed response coordinates back to the original response coordinates.
  5. Suppose the transformed predictor and response are both invertible reparameterizations. Does the conditional law remain Gaussian? Give the conditional mean and covariance in the new coordinates.

Previous

Next

Back to Tutorials
Clinical Biostats