Affine Equivariance Of The Matrix-Normal Law

Why affine transformations matter

Matrix-valued observations are often re-expressed by changing coordinates, combining rows or columns, or adding a fixed offset. The matrix-normal model has a particularly simple response to these operations: a fixed left transformation acts on the row scale, and a fixed right transformation acts on the column scale. The details matter, because the transformed matrix may have a nonsingular density or may instead be constrained to a lower-dimensional set.

We use $X$ with $p$ rows and $q$ columns, mean $M$, row scale $U$, and column scale $V$. For the nonsingular case, $U$ and $V$ are symmetric positive definite. The matrix-normal convention is that $\operatorname{vec}(X)$, which stacks the columns of $X$, has covariance $V\otimes U$. Equivalently, $\operatorname{Cov}(X_{ij},X_{kl})=U_{ik}V_{jl}$.

Definition: $X\sim MN_{p,q}(M,U,V)$ means that $\operatorname{vec}(X)$ is multivariate normal with mean $\operatorname{vec}(M)$ and covariance $V\otimes U$. In the usual nonsingular matrix-normal density, $U$ and $V$ are symmetric positive definite. We also use the same notation for a possibly degenerate Gaussian matrix when $U$ and $V$ are positive semidefinite; in that case a density with respect to full-dimensional Lebesgue measure need not exist.

The affine transformation rule

Let $A$ be an $r\times p$ fixed matrix, $B$ a $q\times s$ fixed matrix, and $C$ a fixed $r\times s$ matrix. Then $Y=AXB+C$ is $r\times s$. The transpose in the transformed column scale is important: right multiplication by $B$ produces $B^TVB$, not $BVB^T$.

Theorem: If $X\sim MN_{p,q}(M,U,V)$, with $U,V$ positive semidefinite, then $$ Y=AXB+C\sim MN_{r,s}\left(AMB+C,\;AUA^T,\;B^TVB\right). $$ This statement includes degenerate Gaussian laws. If $AUA^T$ and $B^TVB$ are both positive definite, the transformed law is nonsingular and has the ordinary matrix-normal density.

The theorem follows from the vectorization identity $\operatorname{vec}(AXB)=(B^T\otimes A)\operatorname{vec}(X)$. We give the covariance calculation explicitly, since it shows how the two scales separate.

Transform the mean.
Linearity of expectation gives $E[Y]=A E[X]B+C=AMB+C$.

Transform the centered vector.
Subtracting the mean yields $\operatorname{vec}(Y-E[Y])=(B^T\otimes A)\operatorname{vec}(X-M)$.

Transform the covariance.
Use the covariance transformation rule and the Kronecker-product multiplication identity to obtain the new separable covariance.

For completeness, the covariance calculation is

\begin{aligned} \operatorname{Cov}(\operatorname{vec}Y) &=(B^T\otimes A)(V\otimes U)(B\otimes A^T)\\ &=(B^TVB)\otimes(AUA^T). \end{aligned}

The vector $\operatorname{vec}(Y)$ is Gaussian because it is an affine transformation of the Gaussian vector $\operatorname{vec}(X)$. Its mean and covariance are precisely those in the definition of the transformed matrix-normal law, proving the theorem. $\square$

Invertible transformations and the density

If $A$ is square $p\times p$ and invertible, and $B$ is square $q\times q$ and invertible, then the transformation is a one-to-one change of coordinates on the space of $p\times q$ matrices. In this case the transformed scales $AUA^T$ and $B^TVB$ are positive definite whenever $U,V$ are positive definite, so the transformed law remains nonsingular.

The density calculation confirms that the affine rule is compatible with the matrix-normal density. For an invertible linear map on matrices, $X\mapsto AXB$, the corresponding map on vectors is $B^T\otimes A$. Its absolute Jacobian determinant is $|\det A|^q|\det B|^p$. Thus the inverse transformation contributes the factor $|\det A|^{-q}|\det B|^{-p}$ to the density of $Y$.

The scale determinants transform as $|AUA^T|=|\det A|^2|U|$ and $|B^TVB|=|\det B|^2|V|$. Consequently the matrix-normal normalizing factor for the transformed scales is the original factor multiplied by $|\det A|^{-q}|\det B|^{-p}$, exactly the inverse Jacobian factor. The quadratic form is unchanged after substituting $X=A^{-1}(Y-C)B^{-1}$: the factors $A$ and $B$ cancel against the inverse transformed scales inside the trace. Hence the change-of-variables density is the matrix-normal density with mean $AMB+C$ and scales $AUA^T,B^TVB$.

Worked Example: A Square Invertible Transformation

Suppose $X\sim MN_{2,2}(M,U,V)$, where $$ M=\begin{pmatrix}1&0\\2&-1\end{pmatrix},\quad U=\begin{pmatrix}2&1\\1&2\end{pmatrix},\quad V=\begin{pmatrix}1&0.2\\0.2&1\end{pmatrix}. $$ Take $$ A=\begin{pmatrix}1&1\\0&1\end{pmatrix},\quad B=\begin{pmatrix}1&0\\1&1\end{pmatrix},\quad C=\begin{pmatrix}0&2\\-1&0\end{pmatrix}. $$ Both $A$ and $B$ have determinant $1$, so both are invertible.

First, $AM=\begin{pmatrix}3&-1\\2&-1\end{pmatrix}$, and therefore $$ AMB+C= \begin{pmatrix}3&-1\\2&-1\end{pmatrix} \begin{pmatrix}1&0\\1&1\end{pmatrix} +\begin{pmatrix}0&2\\-1&0\end{pmatrix} = \begin{pmatrix}2&1\\0&-1\end{pmatrix}. $$

For the row scale, $AU=\begin{pmatrix}3&3\\1&2\end{pmatrix}$, so $$ AUA^T= \begin{pmatrix}3&3\\1&2\end{pmatrix} \begin{pmatrix}1&0\\1&1\end{pmatrix} = \begin{pmatrix}6&3\\3&2\end{pmatrix}. $$ For the column scale, $VB=\begin{pmatrix}1.2&0.2\\1.2&1\end{pmatrix}$, so $$ B^TVB= \begin{pmatrix}1&1\\0&1\end{pmatrix} \begin{pmatrix}1.2&0.2\\1.2&1\end{pmatrix} = \begin{pmatrix}2.4&1.2\\1.2&1\end{pmatrix}. $$ Thus $AXB+C$ has the matrix-normal law with the calculated mean and scales. The scales are positive definite: their determinants are $6\cdot2-3^2=3$ and $2.4\cdot1-1.2^2=0.96$, respectively, and their leading diagonal entries are positive.

Rank, degeneracy, and support

For fixed $A$ and $B$, the covariance formula also identifies when the transformed law is degenerate. With $U,V$ positive definite, the rank of the covariance matrix is $$ \operatorname{rank}\!\left((B^TVB)\otimes(AUA^T)\right) =\operatorname{rank}(B)\operatorname{rank}(A). $$ Indeed, positive definiteness of $U$ implies $\operatorname{rank}(AUA^T)=\operatorname{rank}(A)$, and positive definiteness of $V$ implies $\operatorname{rank}(B^TVB)=\operatorname{rank}(B)$. The rank of a Kronecker product is the product of the ranks of its factors.

The output space has dimension $rs$. Therefore the transformed vector has a full-dimensional covariance exactly when $A$ has full row rank $r$ and $B$ has full column rank $s$. When either condition fails, its covariance is singular, and the Gaussian law is supported on a proper affine subspace. This criterion is more precise than simply asking whether $A$ or $B$ is square. A rectangular transformation can still yield a full-dimensional output if it has the required output-side ranks.

Result: Suppose $U,V$ are positive definite. The covariance rank of $\operatorname{vec}(AXB+C)$ is $\operatorname{rank}(A)\operatorname{rank}(B)$. Its Gaussian law has a density on the full $r\times s$ matrix space if and only if $\operatorname{rank}(A)=r$ and $\operatorname{rank}(B)=s$. Otherwise, it is concentrated on a lower-dimensional affine subspace.

To see the support claim, write $X=M+Z$, where $\operatorname{vec}(Z)$ is centered Gaussian. A centered Gaussian vector with covariance $\Sigma$ is concentrated on the range of $\Sigma$; directions in the null space have variance zero and are zero almost surely. Adding the transformed mean shifts that range to an affine subspace. Here $\Sigma=(B^TVB)\otimes(AUA^T)$, whose range has dimension equal to its rank. This proves the stated criterion.

Worked Example: A Rank-Deficient Transformation

Let $X\sim MN_{3,2}(0,I_3,I_2)$, and take $$ A=\begin{pmatrix}1&1&0\\2&2&0\end{pmatrix},\qquad B=\begin{pmatrix}1\\0\end{pmatrix},\qquad C=0. $$ The output $Y=AXB$ is $2\times1$. Since the entries of $X$ are independent standard normal variables, its entries can also be read directly: $$ Y_1=X_{11}+X_{21},\qquad Y_2=2X_{11}+2X_{21}=2Y_1. $$

The transformed mean is zero. The row scale is $$ A I_3 A^T=AA^T=\begin{pmatrix}2&4\\4&8\end{pmatrix}, $$ and the column scale is $B^TI_2B=1$. The row-scale determinant is $2\cdot8-4\cdot4=0$, as expected because the second row of $A$ is twice its first row. In particular, $\operatorname{Var}(Y_1)=2$, $\operatorname{Cov}(Y_1,Y_2)=4$, and $\operatorname{Var}(Y_2)=8$. Since $Y_2=2Y_1$ with probability one, the law lies on a line and has no two-dimensional density.

Worked Example: A Rectangular Transformation With a Nonsingular Output

Let $X\sim MN_{2,3}(M,U,V)$, with $$ M=\begin{pmatrix}1&2&0\\-1&0&3\end{pmatrix},\quad U=\begin{pmatrix}1&0.3\\0.3&2\end{pmatrix},\quad V=\begin{pmatrix}2&1&0\\1&2&0\\0&0&1\end{pmatrix}. $$ Set $A=I_2$, $B=\begin{pmatrix}1\\-1\\0\end{pmatrix}$, and $C=0$. The output $Y=AXB$ has two rows and one column.

The mean is $$ MB=\begin{pmatrix}1&2&0\\-1&0&3\end{pmatrix} \begin{pmatrix}1\\-1\\0\end{pmatrix} =\begin{pmatrix}-1\\-1\end{pmatrix}. $$ The row scale remains $U$, while the column scale is the scalar $$ B^TVB= \begin{pmatrix}1&-1&0\end{pmatrix} \begin{pmatrix}2&1&0\\1&2&0\\0&0&1\end{pmatrix} \begin{pmatrix}1\\-1\\0\end{pmatrix} =2. $$ Thus $\operatorname{Cov}(\operatorname{vec}Y)=2U=\begin{pmatrix}2&0.6\\0.6&4\end{pmatrix}$. Its determinant is $8-0.36=7.64>0$, so this two-dimensional output is nonsingular. Although $B$ reduces three columns to one, it has full column rank, and $A$ has full row rank. This is why rectangular does not automatically mean degenerate.

Common mistake and practical use

A common error is to swap the roles of the row and column scales, or to omit the transpose on the right-hand scale. The formula is not $A V A^T$ and $B U B^T$: $A$ multiplies on the left and therefore acts on row indices, while $B$ multiplies on the right and acts on column indices. The covariance calculation fixes the order unambiguously:

\operatorname{Cov}(\operatorname{vec}(AXB)) =(B^T\otimes A)(V\otimes U)(B\otimes A^T) =(B^TVB)\otimes(AUA^T).

A second mistake is to conclude that a lower-dimensional output must be singular. The correct question is whether the transformed covariance has full rank in the output space. The rank test above answers that directly. This distinction is useful when choosing contrasts, aggregating measurements, or changing coordinates: the mean and scales update by simple matrix products, while the ranks tell whether the transformed model still admits an ordinary density.

Key takeaway: Under $Y=AXB+C$, the mean becomes $AMB+C$, the row scale becomes $AUA^T$, and the column scale becomes $B^TVB$. Invertible square transformations preserve nonsingularity; more generally, full row rank of $A$ and full column rank of $B$ are the conditions for a full-dimensional output law.

Check Your Understanding

Work through these questions using the transformation rule and its rank condition.

If $X\sim MN_{p,q}(M,U,V)$, what are the mean, row scale, and column scale of $AXB+C$ for compatible fixed matrices?
Why does the column scale involve $B^TVB$, rather than $BVB^T$?
For positive definite $U,V$, state the covariance rank of $\operatorname{vec}(AXB+C)$ in terms of the ranks of $A$ and $B$.
Can a rectangular $A$ and $B$ produce a nonsingular output law? State the precise rank conditions.
If a transformed covariance is singular, what geometric set supports the resulting Gaussian law?