Why Sums Need Not Stay Matrix-Normal
Independent Gaussian random vectors have a Gaussian sum. For matrix-normal random matrices, the sum is also Gaussian after vectorization, but a matrix-normal law requires more: its covariance must separate into a row factor and a column factor. Adding two covariance matrices that each separate in this way does not necessarily preserve that structure.
This distinction matters when combining independent sources of matrix-valued variation. The sum always has a well-defined mean and covariance, but those alone do not justify calling it matrix-normal. Independence among entries or submatrices is also controlled by the covariance, not simply by the fact that the original matrices were independent.
Here \(U\) is the row covariance factor and \(V\) is the column covariance factor. The covariance of the vectorized matrix is \(V\otimes U\) under column-wise vectorization. The factors are identifiable only up to reciprocal scaling: \(V\otimes U=(c^{-1}V)\otimes(cU)\) for every \(c>0\).
The Sum: Gaussian Always, Matrix-Normal Sometimes
Let \(X_1,\ldots,X_q\) be independent \(m\times p\) matrix-normal random matrices with \(X_r\sim MN_{m,p}(M_r,U_r,V_r)\). Independence here is between the random matrices, and every matrix has the same dimensions. Write \(S=\sum_{r=1}^q X_r\). Vectorization turns this into a sum of independent multivariate normal vectors, so its distribution and covariance are determined directly.
Proof. Put \(Z_r=\operatorname{vec}(X_r)\). Each \(Z_r\) is multivariate normal with mean \(\operatorname{vec}(M_r)\) and covariance \(V_r\otimes U_r\), and the \(Z_r\)'s are independent. For any vector \(t\) of length \(mp\), the characteristic function of their sum is the product of their characteristic functions: $$ E\!\left[\exp\!\left(i t^\mathsf{T}\sum_{r=1}^q Z_r\right)\right] =\prod_{r=1}^q \exp\!\left(i t^\mathsf{T}\operatorname{vec}(M_r) -\tfrac12 t^\mathsf{T}(V_r\otimes U_r)t\right). $$ Combining exponents gives the characteristic function of a multivariate normal vector with mean \(\sum_r\operatorname{vec}(M_r)\) and covariance \(\sum_r(V_r\otimes U_r)\). This proves the stated mean and covariance, and proves that \(S\) is Gaussian after vectorization. By the definition of the matrix-normal law, it is matrix-normal exactly when this covariance has the form \(V\otimes U\) with positive definite factors. If all \(U_r=U\), then the covariance is \((\sum_r V_r)\otimes U\); the sum of positive definite matrices is positive definite. If all \(V_r=V\), it is \(V\otimes(\sum_r U_r)\), again with positive definite factors. \(\square\)
A useful exact test does not require guessing two factors across all \(mp\) coordinates. Partition the covariance of \(\operatorname{vec}(S)\) into \(p\times p\) blocks, each of size \(m\times m\), according to pairs of columns. The block for columns \(j,l\) is $$ C_{jl}=\sum_{r=1}^q (V_r)_{jl}U_r. $$ The sum is matrix-normal precisely when all these blocks are scalar multiples of one common positive definite matrix \(U\), with the scalars forming a positive definite matrix \(V\). Indeed, that condition says exactly \(C_{jl}=V_{jl}U\) for every \(j,l\), which is the block definition of \(V\otimes U\). This test also shows why separate matrix-normal distributions for the summands do not, by themselves, establish a matrix-normal law for their sum.
Worked Example: A Sum That Is Not Matrix-Normal
Let \(A\sim MN_{2,2}(0,I_2,\operatorname{diag}(1,2))\) and \(B\sim MN_{2,2}(0,\operatorname{diag}(1,2),I_2)\) be independent. Find the entry variances of \(A+B\), then determine whether the sum is matrix-normal.
The variance of entry \((i,j)\) in a matrix-normal matrix with factors \(U,V\) is \(U_{ii}V_{jj}\). For \(A\), the two rows have the same entry variances, \((1,2)\). For \(B\), the first row has variances \((1,1)\) and the second row has variances \((2,2)\). Independence adds the variances entry by entry, giving $$ \begin{pmatrix} \operatorname{Var}(A_{11}+B_{11})&\operatorname{Var}(A_{12}+B_{12})\\ \operatorname{Var}(A_{21}+B_{21})&\operatorname{Var}(A_{22}+B_{22}) \end{pmatrix} = \begin{pmatrix} 1+1&2+1\\ 1+2&2+2 \end{pmatrix} = \begin{pmatrix}2&3\\3&4\end{pmatrix}. $$
All four factor matrices are diagonal, so different entries of each summand have zero covariance, and different entries of the sum also have zero covariance. The sum is nevertheless not matrix-normal. If it were, its entry-variance table would be \((U_{ii}V_{jj})_{i,j}\), an outer product of two vectors, and would have determinant zero. But here $$ 2\cdot4-3\cdot3=-1\ne0. $$ Thus the sum is Gaussian after vectorization but its covariance is not separable; it is not matrix-normal.
Worked Example: A Common Row Factor Gives a Matrix-Normal Sum
Let \(A\) and \(B\) be independent with $$ A\sim MN_{2,2}\!\left( \begin{pmatrix}0&1\\2&0\end{pmatrix}, \begin{pmatrix}1&0\\0&2\end{pmatrix}, \begin{pmatrix}1&0\\0&2\end{pmatrix} \right),\qquad B\sim MN_{2,2}\!\left( \begin{pmatrix}1&0\\0&-1\end{pmatrix}, \begin{pmatrix}1&0\\0&2\end{pmatrix}, \begin{pmatrix}2&0.5\\0.5&1\end{pmatrix} \right). $$ Find the law of \(A+B\).
Both matrices have the same row factor \(U=\operatorname{diag}(1,2)\). Their mean matrices add to $$ \begin{pmatrix}0&1\\2&0\end{pmatrix} + \begin{pmatrix}1&0\\0&-1\end{pmatrix} = \begin{pmatrix}1&1\\2&-1\end{pmatrix}. $$ The column factors add to $$ V_A+V_B= \begin{pmatrix}1&0\\0&2\end{pmatrix} + \begin{pmatrix}2&0.5\\0.5&1\end{pmatrix} = \begin{pmatrix}3&0.5\\0.5&3\end{pmatrix}. $$ This is positive definite: its diagonal entry is positive and its determinant is \(9-0.25=8.75>0\). The common-factor result therefore gives $$ A+B\sim MN_{2,2}\!\left( \begin{pmatrix}1&1\\2&-1\end{pmatrix}, \begin{pmatrix}1&0\\0&2\end{pmatrix}, \begin{pmatrix}3&0.5\\0.5&3\end{pmatrix} \right). $$ For a check, the variances in the first row are \(1\cdot3=3\) in each column; in the second row they are \(2\cdot3=6\) in each column. Within row 1 the covariance between the two entries is \(1\cdot0.5=0.5\); within row 2 it is \(2\cdot0.5=1\), as specified by the resulting factors.
When Are Entries or Blocks Independent?
For a jointly Gaussian vector, zero cross-covariance between two subvectors is equivalent to their independence. This follows from the multivariate normal characteristic function: when the cross-covariance blocks vanish, the quadratic form in its exponent splits into a sum for the two subvectors, so the joint characteristic function factors into their marginal characteristic functions. Conversely, independence makes the cross-covariance zero whenever the covariances exist, as they do here. Since every finite collection of entries of a matrix-normal matrix is jointly Gaussian, this criterion applies directly to entries and submatrices.
Justification. The cross-covariance of entries \(X_{ij}\) and \(X_{kl}\) is \(U_{ik}V_{jl}\), so the entry claim follows from the Gaussian independence criterion. For the two submatrices, every cross-covariance between one entry from each must vanish. Those covariances are all products \(U_{ik}V_{jl}\), with \(i\in I,k\in K,j\in J,l\in L\). If either displayed submatrix is zero, every product is zero. If neither is zero, each has at least one nonzero entry, and the product of those two entries is a nonzero cross-covariance. Thus all cross-covariances vanish exactly when at least one of the two factor submatrices is zero. The Gaussian criterion then gives the result. \(\square\)
The same covariance principle works for a sum even when it is not matrix-normal. For independent matrix-normal summands, its entry covariance is $$ \operatorname{Cov}(S_{ij},S_{kl}) =\sum_{r=1}^q (U_r)_{ik}(V_r)_{jl}. $$ Because the vectorized sum is jointly Gaussian, two entries are independent exactly when this sum is zero. For blocks, require every cross-covariance between their entries to be zero. In a nonseparable sum, the individual matrix-normal block criterion need not apply, but the Gaussian zero-covariance criterion still does.
Worked Example: Independent Row Blocks but Dependent Column Blocks
Suppose \(X\sim MN_{3,2}(M,U,V)\), where $$ U=\begin{pmatrix}1&0&0\\0&2&0.3\\0&0.3&1\end{pmatrix}, \qquad V=\begin{pmatrix}1&0.4\\0.4&1\end{pmatrix}. $$ Check positive definiteness and decide whether row 1 is independent of rows 2 and 3, and whether the two columns are independent.
The leading principal minors of \(U\) are \(1\), \(2\), and \(2(1)-0.3^2=1.91\), all positive; \(U\) is positive definite. The diagonal entries of \(V\) are positive and its determinant is \(1-0.4^2=0.84>0\), so \(V\) is positive definite as well.
For the row blocks, \(U_{\{1\},\{2,3\}}=(0\ \ 0)\). Every covariance between an entry in row 1 and an entry in rows 2 or 3 is therefore zero, regardless of the column indices. Joint Gaussianity implies that row 1 is independent of the two-row block. In contrast, the cross-covariance between the two columns is \(0.4U\), which is not zero. For instance, \(\operatorname{Cov}(X_{11},X_{12})=U_{11}V_{12}=0.4\), so the column blocks are not independent.
Common Pitfall: Independence of Summands Is Not Separability
A frequent error is to argue that because the summands are independent and each is matrix-normal, their sum must be matrix-normal. Independence guarantees that covariances add; it does not guarantee that the sum of Kronecker products is a single Kronecker product. The first worked example gives a direct check: its variance table has determinant \(-1\), whereas every matrix-normal variance table must have determinant zero.
A related error is to infer independence of entries from a diagonal variance table alone. Variances only describe the diagonal of the covariance matrix; covariances between distinct entries must also be checked. For a matrix-normal matrix, the formula \(U_{ik}V_{jl}\) supplies those covariances, and for a sum the corresponding terms must be added. Conversely, even when all entries are independent, their variances may fail the required row-by-column factorization, as in the nonseparable example.
Check Your Understanding
Use the covariance formulas and Gaussian independence criterion to answer these questions.
- For two independent matrix-normal matrices with a common column factor \(V\), write the row and column factors of their sum.
- Why does a nonzero determinant of the entry-variance table rule out a matrix-normal distribution?
- For \(X\sim MN_{m,p}(M,U,V)\), state the condition for entries \(X_{ij}\) and \(X_{kl}\) to be independent.
- Two matrix-normal summands have different row and column factors. What covariance must be checked to determine whether two entries of their sum are independent?
- For row sets \(I,K\) and column sets \(J,L\), what condition makes \(X_{I,J}\) independent of \(X_{K,L}\)?