Vectorization And Dependence In Matrix-T Laws

Why Vectorization Does Not Remove the Dependence

Vectorization turns a matrix into an ordinary vector, so it is tempting to treat a matrix-t variable as an ordinary multivariate-t variable with a Kronecker-structured scale matrix. The Kronecker matrix does describe the Gaussian covariance structure in the mixture representation, but the matrix-t density retains a determinant of a matrix, not merely a scalar quadratic form. This distinction is also central to dependence: entries that are uncorrelated, or even independent, conditional on a latent Gaussian scale can become dependent after that scale is averaged out.

Let $X$ be a $p\times q$ random matrix with location $M$, positive-definite row scale $\Sigma$, positive-definite column scale $\Omega$, and degrees of freedom $\nu>p-1$. Write $\operatorname{vec}(X)$ for the vector formed by stacking the columns of $X$. The convention here is that the matrix-t density has exponent $-(\nu+q)/2$; its row scale and column scale are $\Sigma$ and $\Omega$, respectively.

Definition: Under this matrix-t parameterization, $X$ has density $$ f_X(X)= \frac{\Gamma_p((\nu+q)/2)} {\pi^{pq/2}\Gamma_p(\nu/2)\,|\Sigma|^{q/2}|\Omega|^{p/2}} \left|I_p+\Sigma^{-1}(X-M)\Omega^{-1}(X-M)'\right|^{-(\nu+q)/2}. $$ Here $\Gamma_p$ is the multivariate gamma function, $|A|$ denotes a determinant, and $I_p$ is the $p\times p$ identity matrix. Both scale matrices are symmetric positive definite.

The Vectorized Density and Its Kronecker Scales

Set $z=\operatorname{vec}(X)$, $m=\operatorname{vec}(M)$, and $Y=X-M$. The Kronecker scale matrix for the corresponding matrix-normal model is $C=\Omega\otimes\Sigma$, so its inverse is $\Omega^{-1}\otimes\Sigma^{-1}$. The vec identity gives $ \operatorname{tr}(\Sigma^{-1}Y\Omega^{-1}Y')=(z-m)'(\Omega^{-1}\otimes\Sigma^{-1})(z-m) $. The determinant in the matrix-t density, however, is not obtained by replacing it with one plus this quadratic form.

f_z(z)= \frac{\Gamma_p((\nu+q)/2)} {\pi^{pq/2}\Gamma_p(\nu/2)\,|\Omega\otimes\Sigma|^{1/2}} \left|I_p+\Sigma^{-1}Y\Omega^{-1}Y'\right|^{-(\nu+q)/2}, \qquad Y=\operatorname{unvec}_{p,q}(z-m).

The equality of the normalizing constants follows from $|\Omega\otimes\Sigma|=|\Omega|^p|\Sigma|^q$. Column-stacking is a reordering of the $pq$ real coordinates, so the change-of-variables Jacobian has absolute value one. An equivalent determinant uses a $q\times q$ matrix: $|I_p+AB|=|I_q+BA|$ for compatible rectangular $A,B$, giving $\left|I_p+\Sigma^{-1}Y\Omega^{-1}Y'\right|=\left|I_q+\Omega^{-1}Y'\Sigma^{-1}Y\right|$.

Key distinction: The vectorized matrix-t density involves the determinant of $I_p+\Sigma^{-1}Y\Omega^{-1}Y'$ (equivalently, a $q\times q$ determinant). It is generally not an ordinary $pq$-dimensional multivariate-t density whose kernel is a power of $1+(z-m)'C^{-1}(z-m)$. The Kronecker matrix identifies the quadratic form, but does not turn the matrix determinant into that scalar kernel.

Worked Example: Reading the Vectorized Kernel

Take $p=q=2$, $\nu=5$, $M=0$, $\Sigma=\operatorname{diag}(2,1)$, $\Omega=\operatorname{diag}(3,1)$, and $X=\begin{pmatrix}1&0\\0&1\end{pmatrix}$. Calculate the determinant factor and compare it with the scalar quadratic form from the Kronecker scale matrix.

With column-stacking, $z=(1,0,0,1)'$, while $\Omega\otimes\Sigma=\operatorname{diag}(6,3,2,1)$. Thus the Kronecker quadratic form is $z'(\Omega^{-1}\otimes\Sigma^{-1})z=1/6+1=7/6$.

For the matrix determinant, $\Omega^{-1}=\operatorname{diag}(1/3,1)$, so $X\Omega^{-1}X'=\operatorname{diag}(1/3,1)$, and $\Sigma^{-1}X\Omega^{-1}X'=\operatorname{diag}(1/6,1)$. Consequently, $\left|I_2+\Sigma^{-1}X\Omega^{-1}X'\right|=(7/6)(2)=7/3$. This is not $1+7/6=13/6$, the scalar expression that would appear in an ordinary multivariate-t kernel. The full density at this $X$ is $$ f_X(X)= \frac{\Gamma_2(7/2)} {\pi^2\Gamma_2(5/2)\,2^{1}\,3^{1}} \left(\frac{7}{3}\right)^{-7/2}. $$ The scale determinant factors are $ |\Sigma|^{q/2}=2$ and $ |\Omega|^{p/2}=3$, and the exponent is $-(\nu+q)/2=-7/2$, as required.

Why This Density Arises: A Gaussian-Wishart Derivation

The density formula can be derived by mixing a matrix-normal distribution over a Wishart precision matrix. This derivation also identifies which covariance statements are conditional and which are unconditional. Let $W\sim W_p(\nu,\Sigma^{-1})$, using the convention that the second Wishart argument is its scale matrix, and, conditional on $W$, let $X$ be matrix normal with mean $M$, row covariance $W^{-1}$, and column covariance $\Omega$. Thus the conditional covariance of $\operatorname{vec}(X)$ is $\Omega\otimes W^{-1}$.

Result: If $\nu>p-1$, $\Sigma$ and $\Omega$ are positive definite, and the conditional model is as stated, integrating out $W$ gives the matrix-t density above.

To verify the result, write $Y=X-M$ and $Q=Y\Omega^{-1}Y'$. The conditional matrix-normal density contributes a factor proportional to $|W|^{q/2}|\Omega|^{-p/2}\exp\{-\operatorname{tr}(WQ)/2\}$. The Wishart density contributes $|W|^{(\nu-p-1)/2}|\Sigma|^{\nu/2}\exp\{-\operatorname{tr}(\Sigma W)/2\}$ and its normalizing constant. Multiplying the kernels, the power of $|W|$ is $(\nu+q-p-1)/2$, and the exponential involves $\Sigma+Q$.

\begin{aligned} f_X(X) &=\frac{|\Sigma|^{\nu/2}|\Omega|^{-p/2}} {(2\pi)^{pq/2}2^{\nu p/2}\Gamma_p(\nu/2)} \int_{W>0}|W|^{(\nu+q-p-1)/2} e^{-\operatorname{tr}\{(\Sigma+Q)W\}/2}\,dW\\ &=\frac{|\Sigma|^{\nu/2}|\Omega|^{-p/2}} {(2\pi)^{pq/2}2^{\nu p/2}\Gamma_p(\nu/2)} \,2^{p(\nu+q)/2}\Gamma_p((\nu+q)/2) |\Sigma+Q|^{-(\nu+q)/2}\\ &=\frac{\Gamma_p((\nu+q)/2)} {\pi^{pq/2}\Gamma_p(\nu/2)|\Sigma|^{q/2}|\Omega|^{p/2}} \left|I_p+\Sigma^{-1}Q\right|^{-(\nu+q)/2}. \end{aligned}

The integral in the second line is the Wishart gamma integral, valid because $\nu+q>p-1$ and $\Sigma+Q$ is positive definite. In the last line, use $|\Sigma+Q|=|\Sigma|\,|I_p+\Sigma^{-1}Q|$; the powers of $2$ reduce to $2^{pq/2}$, which cancels the corresponding factor in $(2\pi)^{pq/2}$. This proves the density expression. $\square$

Conditional Covariance and Unconditional Dependence

Conditional on $W$, the vectorized model is Gaussian. For entries $X_{ij}$ and $X_{k\ell}$, its covariance is $\operatorname{Cov}(X_{ij},X_{k\ell}\mid W)=(W^{-1})_{ik}\Omega_{j\ell}$. Thus some pairs have zero conditional covariance: for example, entries in the same row but different columns do when $\Omega$ is diagonal. Since the conditional distribution is Gaussian, zero conditional covariance for such a pair also gives conditional independence. It does not, by itself, imply unconditional independence after the shared random $W$ is integrated out.

Dependence principle: A shared random scale can couple variables that are independent conditional on that scale. Conditional covariance describes the Gaussian distribution at a fixed scale; unconditional dependence also reflects how the conditional distributions change as the scale varies.

Worked Example: Zero Conditional Covariance, Positive Dependence

Let $p=1$, $q=2$, $M=0$, $\Sigma=1$, $\Omega=I_2$, and $\nu=6$. The mixture has $W\sim\chi^2_6$, and, conditional on $W$, $X_1$ and $X_2$ are independent $N(0,1/W)$ variables. Their conditional covariance is zero. Show that their squares are dependent without relying on a nonzero covariance between the entries themselves.

For a $\chi^2_\nu$ variable, direct integration of its gamma density gives $E[W^{-1}]=1/(\nu-2)$ for $\nu>2$ and $E[W^{-2}]=1/\{(\nu-2)(\nu-4)\}$ for $\nu>4$. Here these are $1/4$ and $1/8$. Conditional independence and the second moment of a centered normal give $E[X_1^2X_2^2\mid W]=(1/W)^2$, so $E[X_1^2X_2^2]=1/8$. Also, $E[X_j^2]=E[1/W]=1/4$ for each $j$. Therefore, $$ \operatorname{Cov}(X_1^2,X_2^2) =E[X_1^2X_2^2]-E[X_1^2]E[X_2^2] =\frac18-\frac14\cdot\frac14 =\frac1{16}>0. $$ The squares are dependent, even though the entries have zero conditional covariance and are conditionally independent. In this example the random precision $W$ makes large conditional variances for both entries occur together.

Worked Example: Checking Entrywise Covariance at a Fixed Scale

Take $p=q=2$, $\Omega=I_2$, and a fixed conditional precision $W=\operatorname{diag}(2,4)$. Then $W^{-1}=\operatorname{diag}(1/2,1/4)$, so the conditional Kronecker covariance of $\operatorname{vec}(X)$ is $I_2\otimes W^{-1}$. With column-stacking, this is $$ I_2\otimes W^{-1} =\operatorname{diag}(1/2,1/4,1/2,1/4). $$ Thus, conditional on this $W$, the four entries have zero covariance with one another, and the Gaussian conditional law makes them independent. For example, $X_{11}$ and $X_{12}$ have conditional variances $1/2$ each and conditional covariance zero. If $W$ is instead random as in the matrix-t mixture, that conditional calculation applies at each realized $W$, but does not establish unconditional independence. A common random scale can still induce dependence in functions such as the squared entries.

Common Mistake and Why the Distinction Matters

Common mistake: “The conditional Gaussian covariance between two entries is zero, so the entries are independent without conditioning.” This conclusion drops the shared random scale. Conditional independence holds at each fixed value of $W$; after mixing, the entries can share information about which conditional variance was realized. The squared-entry calculation above verifies that this can produce unconditional dependence even when the entries’ linear covariance is zero.

A second related error is to substitute the Kronecker quadratic form into the scalar multivariate-t kernel. The Kronecker scales describe the conditional Gaussian covariance, while the marginal matrix-t kernel retains its matrix determinant. Keeping these two features separate matters when evaluating densities and interpreting dependence: a convenient Gaussian covariance description is not, by itself, a complete description of the marginal joint law.

Check Your Understanding

Use the density and mixture representation to check both the algebra and the dependence interpretation.

For column-stacked $z=\operatorname{vec}(X)$, what is the Kronecker matrix corresponding to row scale $\Sigma$ and column scale $\Omega$?
Why does the Jacobian for replacing $X$ by $\operatorname{vec}(X)$ have absolute value one?
What matrix determinant appears in the vectorized matrix-t density, and why is it generally not $1$ plus the Kronecker quadratic form?
If two entries are conditionally independent given $W$, what additional reasoning is needed before concluding they are unconditionally independent?
In the $\nu=6$ example, what positive value did the covariance of the two squared entries have, and what does it establish?