Common Mistakes In Matrix-Variate Dependence And Scales

Why Matrix-Variate Dependence Is Easy to Misread

Matrix-variate models describe an entire rectangular array as one random object. That structure is useful, but it creates several opportunities for errors: a row scale can be mistaken for a column scale, vectorization can silently change the order of coordinates, and an inverse scale in a density can be mistaken for a covariance. Independence claims require particular care: the Gaussian case and the matrix-t case do not behave alike.

The safest practice is to write down dimensions and an entrywise covariance formula before interpreting a model. We use an $r\times c$ random matrix $X$, row scale $U$ of dimension $r\times r$, and column scale $V$ of dimension $c\times c$. Both scale matrices are symmetric positive definite. Vectorization, denoted $\operatorname{vec}(X)$, stacks the columns of $X$ from left to right.

Definition: Under the convention used here, $X\sim MN_{r,c}(M,U,V)$ means that $X$ has matrix-normal mean $M$, row scale $U$, and column scale $V$. Its density is proportional to $$ |U|^{-c/2}|V|^{-r/2} \exp\!\left\{-\frac12\operatorname{tr}\!\left[U^{-1}(X-M)V^{-1}(X-M)'\right]\right\}. $$ Equivalently, $\operatorname{vec}(X)$ is multivariate normal with mean $\operatorname{vec}(M)$ and covariance $V\otimes U$. Thus $$ \operatorname{Cov}(X_{ij},X_{kl})=U_{ik}V_{jl}. $$ The first scale indexes rows; the second indexes columns.

The density shows where inverse matrices appear, but the covariance identity settles their interpretation: $U$ and $V$, not their inverses, are the covariance scales in this parameterization. In a matrix-t model, the matrices called scale parameters likewise need not equal the marginal covariance matrices. The degrees of freedom matter for that distinction.

Worked Checks for Scales and Vectorization

Worked Example: Find the Covariance of Two Entries

Let $X\sim MN_{2,2}(0,U,V)$, with $$ U=\begin{pmatrix}2&1\\1&2\end{pmatrix}, \qquad V=\begin{pmatrix}3&1\\1&2\end{pmatrix}. $$ Find the variances of $X_{11}$ and $X_{21}$, and their covariance.

The entrywise formula gives $\operatorname{Var}(X_{ij})=U_{ii}V_{jj}$. Therefore $\operatorname{Var}(X_{11})=2(3)=6$, and $\operatorname{Var}(X_{21})=2(3)=6$. For the covariance, both entries are in column 1, so $$ \operatorname{Cov}(X_{11},X_{21})=U_{12}V_{11}=1(3)=3. $$ The shared column index selects $V_{11}$; the differing row indices select $U_{12}$. Reversing those roles would give the wrong result.

Worked Example: Check the Kronecker Order

For the same $2\times2$ model, column-stacking gives $\operatorname{vec}(X)=(X_{11},X_{21},X_{12},X_{22})'$. Hence its covariance is $V\otimes U$. In particular, the covariance between the first and second coordinates is $$ (V\otimes U)_{1,2}=V_{11}U_{12}=3. $$ This agrees with $\operatorname{Cov}(X_{11},X_{21})=3$ from the entrywise calculation.

If one instead writes $U\otimes V$, the corresponding entry is $(U\otimes V)_{1,2}=U_{11}V_{12}=2$, which contradicts the direct calculation. The matrices happen to have the same dimensions here, so the erroneous Kronecker product is syntactically possible; checking an entry exposes the mistake. With row-stacking, the product order changes, so the vectorization convention must be stated.

Worked Example: Covariance Scale or Precision?

Consider a scalar normal variable with mean zero and covariance scale $U=4$. Its density contains the exponent $$ -\frac12 x'U^{-1}x=-\frac{x^2}{8}. $$ This is the exponent of $N(0,4)$, whose variance is 4. The inverse $U^{-1}=1/4$ appears in the quadratic form because the density uses precision; it is not the variance.

A reliable check is the scalar normal template: a variance $\sigma^2$ appears in the density as $-x^2/(2\sigma^2)$, while its inverse multiplies the quadratic form. Thus an inverse inside the exponent does not mean the parameter named as the scale is itself a precision.

Why the Matrix-Normal Covariance Has This Form

The Kronecker covariance and the entrywise formula are two descriptions of the same dependence structure. Establishing their agreement is a useful diagnostic, especially when switching between matrix and vector notation.

Result: If $X\sim MN_{r,c}(M,U,V)$ under column-stacking, then $\operatorname{Cov}(\operatorname{vec}(X))=V\otimes U$, and $\operatorname{Cov}(X_{ij},X_{kl})=U_{ik}V_{jl}$.

Use the density's quadratic form.
Write $Y=X-M$. The exponent's quadratic form is $\operatorname{tr}(U^{-1}YV^{-1}Y')$.

Apply the vectorization identity.
$\operatorname{tr}(A Y B Y')=\operatorname{vec}(Y)'(B'\otimes A)\operatorname{vec}(Y)$. Taking $A=U^{-1}$, $B=V^{-1}$, and using symmetry gives $(V^{-1}\otimes U^{-1})$.

Read off covariance.
The precision matrix of the vectorized normal is $V^{-1}\otimes U^{-1}$. Its inverse is $V\otimes U$, since $(V\otimes U)^{-1}=V^{-1}\otimes U^{-1}$.

For the entrywise identity, the coordinate for $X_{ij}$ in column-stacking is $(j-1)r+i$. The corresponding Kronecker entry is $$ (V\otimes U)_{(j-1)r+i,\,(l-1)r+k}=V_{jl}U_{ik}. $$ This proves the covariance formula, including its row and column index roles. The proof uses the stated positive-definite scales, which ensure the covariance and precision matrices are invertible. $\square$

Gaussian Independence: A Precise Claim

For a jointly Gaussian vector, zero covariance between two coordinates implies their independence. In a matrix-normal model, therefore, the entrywise covariance formula can be used to assess entry dependence. In particular, if $U$ and $V$ are diagonal, distinct entries have zero covariance and the jointly Gaussian entries are mutually independent. The converse also holds for positive-definite $U,V$: if every distinct pair of entries is independent, then both scale matrices must be diagonal. To see why, any off-diagonal $U_{ik}$ would give nonzero covariance between $X_{ij}$ and $X_{kj}$, since $V_{jj}>0$; any off-diagonal $V_{jl}$ would give nonzero covariance between $X_{ij}$ and $X_{il}$, since $U_{ii}>0$.

Result: For a matrix-normal random matrix with positive-definite $U,V$, all entries are mutually independent if and only if both $U$ and $V$ are diagonal.

The statement depends on joint Gaussianity. It is not a general rule that zero covariance means independence, and it must not be carried over automatically to matrix-t entries.

Matrix-T Entries: Zero Covariance Does Not Settle Independence

A useful diagnostic example comes from the $1\times2$ matrix-t case, represented as a shared-scale mixture. Let $Q\sim\chi^2_\nu$, let $Z_1,Z_2$ be independent $N(0,1)$ variables independent of $Q$, and set $$ X_{11}=\sqrt{\frac{\nu}{Q}}Z_1,\qquad X_{12}=\sqrt{\frac{\nu}{Q}}Z_2. $$ The two entries have zero covariance when their second moments exist, but the common random scale can make their magnitudes dependent.

Worked Example: Detect Dependence Through Squared Entries

Take $\nu=5$. Conditional on $Q$, the two entries are independent and have conditional mean zero. Thus $E[X_{11}X_{12}\mid Q]=0$, and consequently $E[X_{11}X_{12}]=0$. Their marginal variances are $$ E[X_{11}^2]=5E[Q^{-1}]=\frac{5}{3}, \qquad E[X_{12}^2]=\frac{5}{3}, $$ using $E[Q^{-1}]=1/(\nu-2)=1/3$ for $\nu=5>2$.

For a chi-square variable, $E[Q^{-2}]=1/((\nu-2)(\nu-4))$ when $\nu>4$, so here $E[Q^{-2}]=1/3$. Independence of $Z_1,Z_2$ gives $E[Z_1^2Z_2^2]=1$. Therefore $$ E[X_{11}^2X_{12}^2]=25E[Q^{-2}]E[Z_1^2Z_2^2]=\frac{25}{3}. $$ If the entries were independent, this expectation would equal $E[X_{11}^2]E[X_{12}^2]=25/9$, not $25/3$. Equivalently, $\operatorname{Cov}(X_{11}^2,X_{12}^2)=25/3-25/9=50/9>0$. The entries are therefore dependent despite having zero covariance.

In this example, the shared scale $Q^{-1}$ makes large magnitudes tend to occur together. The calculation requires $\nu>4$ for the fourth moments used; the weaker condition $\nu>2$ suffices for the displayed variances. Matrix-t scale parameters should not be read as marginal covariances without checking the degrees of freedom. For this scalar-row example, each marginal variance is $\nu/(\nu-2)$ times its $t$ scale, when $\nu>2$.

Common Mistakes and Corrections

Swapping the row and column scales. Wrong reasoning: “Both matrices are scales, so their order in the Kronecker product is interchangeable.” Correction: with column-stacking, the covariance is $V\otimes U$, and $\operatorname{Cov}(X_{ij},X_{kl})=U_{ik}V_{jl}$. Check a pair of entries directly.
Confusing covariance with precision. Wrong reasoning: “The density contains $U^{-1}$, so $U$ must be a precision matrix.” Correction: the inverse appears in the quadratic form as precision; in the stated matrix-normal convention, $U$ is the row covariance scale.
Changing vectorization order without changing the formula. Wrong reasoning: “The Kronecker product is always $V\otimes U$.” Correction: that order is for column-stacking. State the ordering and verify one coordinate pair.
Inferring matrix-t independence from zero covariance. Wrong reasoning: “The covariance is zero, so the entries are independent.” Correction: that implication holds for jointly Gaussian variables, not generally for matrix-t variables. Shared mixing can create dependence detectable through squared entries.
Calling a matrix-t scale a covariance without checking moments. Wrong reasoning: “The scale parameter is the variance.” Correction: for a univariate $t_\nu$ marginal with scale $s^2$, the variance is $\nu s^2/(\nu-2)$ only when $\nu>2$; for $\nu\le2$, it does not exist.

Key takeaway: Before interpreting a matrix-variate model, record its dimensions, name the row and column scales, state the vectorization order, and compute one entrywise covariance. For matrix-t models, separately check moment existence and do not treat zero covariance as proof of independence.

Check Your Understanding

Use the stated column-stacking convention and distinguish Gaussian from matrix-t conclusions.

If $X$ is $3\times2$, what are the dimensions of its row scale, column scale, and vectorized covariance matrix?
For a matrix-normal model, write the covariance of $X_{12}$ and $X_{32}$ in terms of $U$ and $V$.
Why does the covariance matrix of $\operatorname{vec}(X)$ use $V\otimes U$, rather than $U\otimes V$, under column-stacking?
What additional fact about the joint distribution permits zero covariance to imply independence in the matrix-normal case?
In the shared-scale example, what feature of the construction makes the squared entries dependent even though the entries have zero covariance?