Marginals And Linear Summaries Of Matrix-Normal Variables

Why Marginals and Linear Summaries Are Useful

A matrix-normal model describes an entire random matrix, but many questions concern only part of that matrix: a few rows or columns, a particular submatrix, or a contrast combining several entries. The vectorized Gaussian representation turns each of these questions into a calculation about a linear transformation of a multivariate normal vector.

The main bookkeeping issue is the order of the Kronecker factors. Throughout, $X$ is a $p\times q$ random matrix, $\operatorname{vec}(X)$ stacks its columns, and the row and column covariance matrices are denoted by $U$ and $V$, respectively. With this convention, the covariance of $\operatorname{vec}(X)$ is $V\otimes U$, not $U\otimes V$.

Definition: For a $p\times q$ matrix $M$ and positive-definite matrices $U$ ($p\times p$) and $V$ ($q\times q$), $X\sim MN_{p,q}(M,U,V)$ means $$ \operatorname{vec}(X)\sim N_{pq}\!\left(\operatorname{vec}(M),\,V\otimes U\right). $$ Here $U$ is the row covariance parameter and $V$ is the column covariance parameter. If a linear transformation produces a singular covariance matrix, its resulting Gaussian law is understood as a possibly degenerate multivariate normal distribution.

The General Linear-Transformation Rule

Let $A$ be an $r\times p$ matrix, $B$ a $q\times s$ matrix, and $D$ a fixed $r\times s$ matrix. The matrix-valued summary $AXB+D$ is linear in the entries of $X$, apart from its fixed shift. Its mean and covariance follow from a standard vectorization identity.

Theorem: If $X\sim MN_{p,q}(M,U,V)$, then $$ AXB+D\sim MN_{r,s}\!\left(AMB+D,\;AUA^{T},\;B^{T}VB\right), $$ where the matrix-normal notation on the right permits positive-semidefinite covariance parameters when the resulting law is degenerate. More generally, for any fixed $k\times pq$ matrix $C$ and fixed vector $c\in\mathbb{R}^k$, $$ C\operatorname{vec}(X)+c\sim N_k\!\left(C\operatorname{vec}(M)+c,\;C(V\otimes U)C^{T}\right). $$

Proof. For compatible matrices, column-stacking gives $$ \operatorname{vec}(AXB)=(B^{T}\otimes A)\operatorname{vec}(X). $$ Therefore $$ \operatorname{vec}(AXB+D) =(B^{T}\otimes A)\operatorname{vec}(X)+\operatorname{vec}(D). $$ A fixed affine transformation of a multivariate normal vector is multivariate normal. Its mean is $$ (B^{T}\otimes A)\operatorname{vec}(M)+\operatorname{vec}(D) =\operatorname{vec}(AMB+D). $$ Its covariance is $$ \begin{aligned} &(B^{T}\otimes A)(V\otimes U)(B\otimes A^{T})\\ &\qquad=(B^{T}VB)\otimes(AUA^{T}), \end{aligned} $$ using $(P\otimes Q)(R\otimes S)=(PR)\otimes(QS)$ when the products are defined. This is exactly the covariance required for the stated matrix-normal law. For the general summary, apply the same affine-normal rule directly to $\operatorname{vec}(X)$, giving mean $C\operatorname{vec}(M)+c$ and covariance $C(V\otimes U)C^{T}$. $\square$

The general formula covers more than transformations that retain a matrix-normal form. Every linear summary is multivariate normal, but its covariance need not factor as a single Kronecker product. The factorized form in the theorem applies to summaries specifically arranged as $AXB+D$.

Selected Rows, Columns, and Submatrices

Selection is a special case of the transformation rule. Let $I$ be an ordered list of distinct row indices and $J$ an ordered list of distinct column indices. Write $X[I,J]$ for the submatrix formed by those rows and columns, in the listed order; use the same convention for $M[I,J]$, $U[I,I]$, and $V[J,J]$.

Result: If $X\sim MN_{p,q}(M,U,V)$, then $$ X[I,J]\sim MN_{|I|,|J|}\!\left(M[I,J],\;U[I,I],\;V[J,J]\right). $$ In particular, selecting rows alone retains the corresponding principal submatrix of $U$ and all of $V$; selecting columns alone retains all of $U$ and the corresponding principal submatrix of $V$.

To see why, let $S_I$ be the row-selection matrix that extracts the rows in $I$, and $S_J$ the column-selection matrix that extracts the columns in $J$. Then $X[I,J]=S_I X S_J^{T}$. The general rule gives row covariance $S_I U S_I^{T}=U[I,I]$, column covariance $S_J V S_J^{T}=V[J,J]$, and mean $M[I,J]$. This is a marginal distribution: it describes the selected entries without conditioning on the entries that were left out.

Worked Example: A Selected Submatrix

Suppose $X\sim MN_{3,3}(M,U,V)$, where $$ M=\begin{pmatrix}1&0&2\\2&1&0\\-1&3&1\end{pmatrix},\quad U=\begin{pmatrix}2&0.5&0\\0.5&1&0.2\\0&0.2&1.5\end{pmatrix},\quad V=\begin{pmatrix}1&0.3&0\\0.3&2&0.4\\0&0.4&1\end{pmatrix}. $$ Find the distribution of the submatrix formed from rows $1,3$ and columns $1,2$.

The selected mean is $$ M[I,J]=\begin{pmatrix}1&0\\-1&3\end{pmatrix}. $$ The selected row covariance is $U[I,I]=\begin{pmatrix}2&0\\0&1.5\end{pmatrix}$, since $U_{13}=U_{31}=0$. The selected column covariance is $V[J,J]=\begin{pmatrix}1&0.3\\0.3&2\end{pmatrix}$. Thus $$ X[I,J]\sim MN_{2,2}\!\left( \begin{pmatrix}1&0\\-1&3\end{pmatrix}, \begin{pmatrix}2&0\\0&1.5\end{pmatrix}, \begin{pmatrix}1&0.3\\0.3&2\end{pmatrix} \right). $$ As a check on the vectorization order, its covariance is $V[J,J]\otimes U[I,I]$, namely $$ \begin{pmatrix} 2&0&0.6&0\\ 0&1.5&0&0.45\\ 0.6&0&4&0\\ 0&0.45&0&3 \end{pmatrix}. $$ The first two coordinates of this vectorization are the first selected column, and the last two are the second selected column.

Scalar Contrasts and Bilinear Summaries

A scalar summary of the form $a^{T}Xb$, where $a\in\mathbb{R}^{p}$ and $b\in\mathbb{R}^{q}$ are fixed, is a particularly useful case. It combines entries across both rows and columns. Taking $A=a^{T}$ and $B=b$ in the transformation theorem gives $$ a^{T}Xb\sim N\!\left(a^{T}Mb,\;(a^{T}Ua)(b^{T}Vb)\right). $$ The two quadratic forms each measure variation along one of the chosen directions; their product is the variance of the scalar summary.

Worked Example: A Scalar Matrix Contrast

Let $X\sim MN_{3,2}(M,U,V)$, with $$ M=\begin{pmatrix}1&0\\2&1\\-1&3\end{pmatrix},\quad U=\begin{pmatrix}2&0.5&0\\0.5&1&0.2\\0&0.2&1.5\end{pmatrix},\quad V=\begin{pmatrix}2&0.5\\0.5&2\end{pmatrix}. $$ Find the distribution of $a^{T}Xb$ for $a=(1,-1,0)^{T}$ and $b=(2,1)^{T}$.

First, $a^{T}M=(1,0)-(2,1)=(-1,-1)$, so the mean is $$ a^{T}Mb=(-1,-1)\begin{pmatrix}2\\1\end{pmatrix}=-3. $$ Next, $$ a^{T}Ua=U_{11}+U_{22}-2U_{12}=2+1-2(0.5)=2. $$ For the column direction, $$ b^{T}Vb =2(2^{2})+2(0.5)(2)(1)+2(1^{2}) =8+2+2=12. $$ Therefore the variance is $2(12)=24$, and $$ a^{T}Xb\sim N(-3,24). $$ This calculation uses the variance as the second argument of the univariate normal distribution.

Matrix-Valued Transformations

When both $A$ and $B$ combine entries, the result remains matrix-normal because the transformed covariance still has a row-by-column Kronecker factorization. The transformed row covariance is $AUA^{T}$; the transformed column covariance is $B^{T}VB$. Neither factor should be reversed.

Worked Example: Transforming Both Matrix Dimensions

Use the $M,U,V$ from the preceding example and set $$ A=\begin{pmatrix}1&1&0\\0&1&1\end{pmatrix}, \qquad B=\begin{pmatrix}1&1\\1&-1\end{pmatrix}. $$ Find the distribution of $AXB$.

The transformed mean is $$ AM=\begin{pmatrix}3&1\\1&4\end{pmatrix}, \qquad AMB=\begin{pmatrix}4&2\\5&-3\end{pmatrix}. $$ For the row covariance, the two rows of $A$ are $(1,1,0)$ and $(0,1,1)$. Their variances and covariance are $$ (1,1,0)U(1,1,0)^{T}=2+1+2(0.5)=4, $$ $$ (0,1,1)U(0,1,1)^{T}=1+1.5+2(0.2)=2.9, $$ and $$ (1,1,0)U(0,1,1)^{T}=0.5+0+1+0.2=1.7. $$ Thus $AUA^{T}=\begin{pmatrix}4&1.7\\1.7&2.9\end{pmatrix}$. The columns of $B$ are $(1,1)^{T}$ and $(1,-1)^{T}$; direct calculation gives their variances $5$ and $3$, and covariance $2-0.5+0.5-2=0$. Hence $$ B^{T}VB=\begin{pmatrix}5&0\\0&3\end{pmatrix}, $$ so the answer is $$ AXB\sim MN_{2,2}\!\left( \begin{pmatrix}4&2\\5&-3\end{pmatrix}, \begin{pmatrix}4&1.7\\1.7&2.9\end{pmatrix}, \begin{pmatrix}5&0\\0&3\end{pmatrix} \right). $$

General Linear Summaries and a Common Pitfall

Some summaries do not have the structured form $AXB$. For example, one coordinate might add entries in one pattern while another coordinate uses a different pattern. Such a summary can always be written $C\operatorname{vec}(X)+c$. Its distribution is multivariate normal, and $C(V\otimes U)C^{T}$ supplies every variance and covariance among the summary coordinates. One should not assume this covariance factors as a Kronecker product without checking its structure.

Common mistake: swapping the Kronecker factors. With column-wise vectorization, the covariance is $V\otimes U$. The first block of coordinates corresponds to the first column of $X$, so within that block the row covariance is $U$; the column covariance determines the blocks. Replacing $V\otimes U$ by $U\otimes V$ generally gives incorrect covariances and dimensions. Another related mistake is using a conditional covariance formula when only a marginal submatrix is requested. A marginal uses the selected covariance blocks; conditioning on omitted entries is a different problem.

Key takeaway: For matrix-normal marginals and linear summaries, begin with $\operatorname{vec}(X)\sim N(\operatorname{vec}(M),V\otimes U)$. A structured transformation $AXB+D$ has mean $AMB+D$ and covariance factors $AUA^{T}$ and $B^{T}VB$; a general linear summary uses $C(V\otimes U)C^{T}$.

Check Your Understanding

Use column-wise vectorization and the convention that $U$ is the row covariance and $V$ is the column covariance.

If $X\sim MN_{p,q}(M,U,V)$, what is the distribution of $X[I,J]$ for ordered row and column subsets $I,J$?
For compatible fixed matrices $A$ and $B$, state the mean and the two covariance factors of $AXB$.
Why is $a^{T}Xb$ normally distributed, and what is its variance?
For a general summary $C\operatorname{vec}(X)+c$, give its mean and covariance.
Explain why a marginal distribution of selected entries should not be calculated using a conditional covariance formula.