Affine Transformations of a Bivariate Normal Pair

Why Transform the Pair Together?

In a bivariate normal model, new variables often describe combined measurements, such as adjusted totals or contrasts. If \(X\) and \(Y\) are jointly normal, then two affine transformations of them are jointly normal as well. Their means, variances, and covariance can be calculated from the original pair without starting over with a new density.

The important point is to transform the pair together. Calculating the two means and variances separately does not determine the joint distribution: the covariance between the transformed variables must also be found. A matrix formulation keeps all of these quantities organized.

Definition: Let \(\mathbf{Z}=(X,Y)^\mathsf{T}\) be a jointly normal pair with mean vector \(\boldsymbol{\mu}\) and covariance matrix \(\boldsymbol{\Sigma}\). An affine transformation of the pair is \(\mathbf{W}=A\mathbf{Z}+\mathbf{b}\), where \(A\) is a fixed \(2\times2\) matrix and \(\mathbf{b}\) is a fixed two-component vector. Then \(\mathbf{W}\) is jointly normal, with mean vector \(A\boldsymbol{\mu}+\mathbf{b}\) and covariance matrix \(A\boldsymbol{\Sigma}A^\mathsf{T}\). The constant vector shifts the means but does not change the covariance matrix.

Write the original parameters as

\boldsymbol{\mu}= \begin{pmatrix}\mu_X\\\mu_Y\end{pmatrix}, \qquad \boldsymbol{\Sigma}= \begin{pmatrix} \sigma_X^2 & \sigma_{XY}\\ \sigma_{XY} & \sigma_Y^2 \end{pmatrix}, \qquad \sigma_{XY}=\operatorname{Cov}(X,Y).

If the original model is given using a correlation \(\rho\), first calculate \(\sigma_{XY}=\rho\sigma_X\sigma_Y\), where \(\sigma_X\) and \(\sigma_Y\) are standard deviations. The diagonal entries of \(\boldsymbol{\Sigma}\) are variances, not standard deviations.

Computing the Transformed Parameters

Suppose the transformations are \(U=aX+bY+c\) and \(V=dX+eY+f\). The coefficient matrix contains the coefficients of \(X\) and \(Y\); the constants \(c\) and \(f\) go into the shift vector:

\begin{pmatrix}U\\V\end{pmatrix} = \begin{pmatrix}a&b\\d&e\end{pmatrix} \begin{pmatrix}X\\Y\end{pmatrix} + \begin{pmatrix}c\\f\end{pmatrix}.

The transformed mean vector is obtained by applying the same coefficients to the original means and then adding the constants. For the covariance calculations, the constants disappear. In particular,

\begin{aligned} \operatorname{Var}(U) &=a^2\sigma_X^2+b^2\sigma_Y^2+2ab\sigma_{XY},\\ \operatorname{Var}(V) &=d^2\sigma_X^2+e^2\sigma_Y^2+2de\sigma_{XY},\\ \operatorname{Cov}(U,V) &=ad\sigma_X^2+(ae+bd)\sigma_{XY}+be\sigma_Y^2. \end{aligned}

These formulas are the entries of \(A\boldsymbol{\Sigma}A^\mathsf{T}\). In the covariance formula, the two cross terms combine to \((ae+bd)\sigma_{XY}\); neither should be omitted.

Why the covariance formula holds: Center the original variables by writing \(\mathbf{Z}-\boldsymbol{\mu}\). Then \(\mathbf{W}-E[\mathbf{W}]=A(\mathbf{Z}-\boldsymbol{\mu})\), because the fixed shift cancels. By the definition of covariance, \[ \operatorname{Cov}(\mathbf{W}) =E\!\left[(\mathbf{W}-E[\mathbf{W}])(\mathbf{W}-E[\mathbf{W}])^\mathsf{T}\right]. \] Substituting \(A(\mathbf{Z}-\boldsymbol{\mu})\) and taking the fixed matrices outside the expectation gives \(A E[(\mathbf{Z}-\boldsymbol{\mu})(\mathbf{Z}-\boldsymbol{\mu})^\mathsf{T}]A^\mathsf{T} =A\boldsymbol{\Sigma}A^\mathsf{T}\).

Worked Examples

Worked Example: Two Mixed Measurements With a Shift

Suppose \((X,Y)\) is jointly normal with \(E[X]=2\), \(E[Y]=-1\), \(\operatorname{Var}(X)=4\), \(\operatorname{Var}(Y)=9\), and \(\operatorname{Corr}(X,Y)=1/2\). Let \(U=2X-Y+1\) and \(V=X+2Y-3\). Find the transformed mean vector and covariance matrix, and state whether the transformed pair has a nonsingular joint density.

Method. Convert the correlation to a covariance, then use the affine-transformation formulas. This avoids treating the constant shifts as random contributions to the variances.

First, \(\sigma_X=2\), \(\sigma_Y=3\), so \(\sigma_{XY}=(1/2)(2)(3)=3\). Thus \[ \boldsymbol{\mu}=\begin{pmatrix}2\\-1\end{pmatrix}, \qquad \boldsymbol{\Sigma}=\begin{pmatrix}4&3\\3&9\end{pmatrix}, \quad A=\begin{pmatrix}2&-1\\1&2\end{pmatrix}, \quad \mathbf b=\begin{pmatrix}1\\-3\end{pmatrix}. \] The means are \[ E[U]=2(2)-(-1)+1=6,\qquad E[V]=2+2(-1)-3=-3. \] Therefore \(E[\mathbf W]=(6,-3)^\mathsf{T}\).

For the variances and covariance, \[ \begin{aligned} \operatorname{Var}(U)&=2^2(4)+(-1)^2(9)+2(2)(-1)(3) =16+9-12=13,\\ \operatorname{Var}(V)&=1^2(4)+2^2(9)+2(1)(2)(3) =4+36+12=52,\\ \operatorname{Cov}(U,V)&=(2)(1)(4)+\big((2)(2)+(-1)(1)\big)(3)+(-1)(2)(9)\\ &=8+(4-1)(3)-18=8+9-18=-1. \end{aligned} \] Hence \[ \boldsymbol{\Sigma}_{W}= \begin{pmatrix}13&-1\\-1&52\end{pmatrix}. \] Its determinant is \(13(52)-(-1)^2=676-1=675>0\). The covariance matrix is positive definite, so this transformed pair has a nonsingular bivariate normal density.

Check and trap avoided. The covariance calculation gives \(-1\), not \(+1\): the final term is \((-1)(2)(9)=-18\). Also, the shifts \(+1\) and \(-3\) affect the means but not these covariance entries.

Worked Example: A Negative Original Covariance

Suppose \((X,Y)\) is jointly normal with means \(1,2\), variances \(4,1\), and correlation \(-1/2\). For \(U=2X-Y\) and \(V=X+Y\), find the mean vector, covariance matrix, and correlation of the transformed pair.

Method. Find the original covariance first, then calculate all three entries of the transformed covariance matrix. This avoids losing the sign of the cross terms.

The original standard deviations are \(2\) and \(1\), so \(\sigma_{XY}=(-1/2)(2)(1)=-1\). The transformed means are \[ E[U]=2(1)-2=0,\qquad E[V]=1+2=3. \] The variances are \[ \begin{aligned} \operatorname{Var}(U)&=2^2(4)+(-1)^2(1)+2(2)(-1)(-1) =16+1+4=21,\\ \operatorname{Var}(V)&=1^2(4)+1^2(1)+2(1)(1)(-1) =4+1-2=3. \end{aligned} \] The covariance is \[ \begin{aligned} \operatorname{Cov}(U,V) &=(2)(1)(4)+\big((2)(1)+(-1)(1)\big)(-1)+(-1)(1)(1)\\ &=8+(2-1)(-1)-1=8-1-1=6. \end{aligned} \] Thus \[ \boldsymbol{\Sigma}_{W}= \begin{pmatrix}21&6\\6&3\end{pmatrix}, \qquad \operatorname{Corr}(U,V)=\frac{6}{\sqrt{21}\sqrt{3}} =\frac{6}{\sqrt{63}}=\frac{2}{\sqrt{7}}\approx 0.7559. \] The determinant is \(21(3)-6^2=63-36=27>0\), consistent with a nonsingular transformed pair.

Check and trap avoided. The variance of \(V=X+Y\) is \(4+1+2(-1)=3\). The arithmetic is \(5-2=3\), not \(0\); a covariance matrix with \(3\) in this diagonal entry must use \(3\) when calculating the correlation.

Worked Example: A Singular Affine Transformation

Let \(X\) and \(Y\) be independent normal variables with means \(1\) and \(-2\), and variances \(1\) and \(4\). Define \(U=X+Y\) and \(V=2X+2Y+1\). Find the transformed mean and covariance matrix, and describe the joint distribution.

Method. Apply the same matrix formulas, then check whether the two transformed coordinates are linearly dependent. This identifies when a bivariate normal pair does not have an ordinary two-dimensional density.

Independence gives \(\sigma_{XY}=0\), so \[ \boldsymbol{\mu}=\begin{pmatrix}1\\-2\end{pmatrix}, \quad \boldsymbol{\Sigma}=\begin{pmatrix}1&0\\0&4\end{pmatrix}, \quad A=\begin{pmatrix}1&1\\2&2\end{pmatrix}, \quad \mathbf b=\begin{pmatrix}0\\1\end{pmatrix}. \] The means are \(E[U]=1-2=-1\) and \(E[V]=2(1)+2(-2)+1=-1\). The covariance entries are \[ \begin{aligned} \operatorname{Var}(U)&=1+4=5,\\ \operatorname{Var}(V)&=2^2(1)+2^2(4)=4+16=20,\\ \operatorname{Cov}(U,V)&=(1)(2)(1)+(1)(2)(4)=2+8=10. \end{aligned} \] Thus the covariance matrix is \[ \boldsymbol{\Sigma}_{W}=\begin{pmatrix}5&10\\10&20\end{pmatrix}. \] Its determinant is \(5(20)-10^2=100-100=0\). Indeed, directly from the definitions, \(V=2U+1\). The transformed pair is jointly normal but degenerate: it lies on a line and has no ordinary two-dimensional density.

Check and trap avoided. A normal transformation need not have a nonsingular covariance matrix. Here the rows of \(A\) are proportional, so the transformation cannot produce two independent directions of variation.

Joint Distribution and Nonsingularity

The transformed mean vector and covariance matrix specify the joint normal model. If \(\boldsymbol{\Sigma}_{W}=\begin{pmatrix}s_U^2&s_{UV}\\s_{UV}&s_V^2\end{pmatrix}\) is positive definite, the transformed pair has the bivariate normal density

f_{U,V}(\mathbf w) = \frac{1}{2\pi\sqrt{\det(\boldsymbol{\Sigma}_{W})}} \exp\!\left[ -\frac12(\mathbf w-\boldsymbol{\mu}_{W})^\mathsf{T} \boldsymbol{\Sigma}_{W}^{-1} (\mathbf w-\boldsymbol{\mu}_{W}) \right], \qquad \mathbf w=\begin{pmatrix}u\\v\end{pmatrix}.

This density expression applies only when the covariance matrix is positive definite. If it is singular, the pair is still jointly normal but concentrated on a line or a point, and the displayed two-dimensional density is not valid.

When the original covariance matrix is positive definite, a nonsingular \(A\) (that is, \(\det(A)\ne0\)) produces a positive-definite transformed covariance matrix. If \(A\) is singular, at least one transformed coordinate is a linear combination of the other, possibly with a constant shift, and the transformed covariance matrix is singular. A singular original model also remains singular under any affine transformation.

Common Mistakes and Exam Traps

Using correlation where covariance is required. Convert using \(\operatorname{Cov}(X,Y)=\rho\sigma_X\sigma_Y\) before applying matrix formulas.
Forgetting the factor of two in a variance. The cross term in \(\operatorname{Var}(aX+bY)\) is \(2ab\operatorname{Cov}(X,Y)\), not \(ab\operatorname{Cov}(X,Y)\).
Dropping one cross term in a transformed covariance. Both \(\operatorname{Cov}(aX+bY,dX+eY)\) cross terms contribute, combining to \((ae+bd)\sigma_{XY}\).
Including constants in variances or covariances. Adding a fixed number shifts a mean but leaves variance and covariance unchanged.
Assuming every transformed pair has a density. Check positive definiteness of the transformed covariance matrix before using a two-dimensional density formula.

Key takeaway: For a jointly normal pair, write the affine transformation as \(\mathbf W=A\mathbf Z+\mathbf b\). Then \(E[\mathbf W]=A\boldsymbol{\mu}+\mathbf b\) and \(\operatorname{Cov}(\mathbf W)=A\boldsymbol{\Sigma}A^\mathsf{T}\). Check the resulting covariance matrix before deciding whether the joint distribution has a nonsingular density.

Check Your Understanding

For each problem, find the requested transformed parameters. The pairs are jointly normal.

\(E[X]=1\), \(E[Y]=2\), \(\operatorname{Var}(X)=4\), \(\operatorname{Var}(Y)=9\), and \(\operatorname{Corr}(X,Y)=1/3\). Let \(U=X+Y\) and \(V=X-Y+4\). Find the mean vector and covariance matrix. (Answer: mean vector \((3,3)^\mathsf{T}\); covariance matrix \(\begin{pmatrix}17&-5\\-5&9\end{pmatrix}\).)
\(E[X]=0\), \(E[Y]=1\), variances \(1,4\), and correlation \(-1/2\). Let \(U=3X+Y-2\) and \(V=X-2Y\). Find the mean vector and covariance matrix. (Answer: mean vector \((-1,-2)^\mathsf{T}\); covariance matrix \(\begin{pmatrix}7&0\\0&21\end{pmatrix}\).)
The original mean vector is \((2,-1)^\mathsf{T}\) and covariance matrix is \(\begin{pmatrix}4&2\\2&9\end{pmatrix}\). Let \(U=X+2Y\) and \(V=3X-Y+1\). Find the transformed mean vector and covariance matrix. (Answer: mean vector \((0,8)^\mathsf{T}\); covariance matrix \(\begin{pmatrix}48&4\\4&33\end{pmatrix}\).)
\(X,Y\) are independent, with means \(1,-2\) and variances \(1,4\). Let \(U=X+Y\) and \(V=2X+2Y+1\). Find the transformed covariance matrix and determine whether a nonsingular joint density exists. (Answer: covariance matrix \(\begin{pmatrix}5&10\\10&20\end{pmatrix}\); no, since \(V=2U+1\) and the matrix is singular.)
The original mean vector is \((1,0)^\mathsf{T}\) and covariance matrix is \(\begin{pmatrix}4&-1\\-1&1\end{pmatrix}\). Let \(U=X+Y\) and \(V=2X-Y+3\). Find the transformed mean vector, covariance matrix, and correlation. (Answer: mean vector \((1,5)^\mathsf{T}\); covariance matrix \(\begin{pmatrix}3&6\\6&21\end{pmatrix}\); correlation \(2/\sqrt{7}\approx0.7559\).)

What You'll Learn