Matrix-Normal Density And Scale Parameters

Introduction

A matrix of measurements can have dependence along both its rows and its columns. For instance, rows might represent participants and columns repeated measurements, or the two dimensions might represent sites and outcomes. The matrix-normal distribution provides a Gaussian model for such data while keeping its density in matrix form.

Its parameters have distinct roles and dimensions: a mean matrix locates the distribution, while a row scale matrix and a column scale matrix enter the density on opposite sides of the centered random matrix. Getting those dimensions and the positive-definiteness conditions right is essential for the density to be defined.

Definition and Parameters

Definition: Let \(n\) and \(p\) be positive integers. An \(n\times p\) random matrix \(X\) has a matrix-normal distribution with mean matrix \(M\), row scale matrix \(U\), and column scale matrix \(V\), written \(X\sim MN_{n,p}(M,U,V)\), if its density with respect to Lebesgue measure on the \(np\) real entries of \(X\) is

f(X)=(2\pi)^{-np/2}|U|^{-p/2}|V|^{-n/2} \exp\!\left\{-\frac12\operatorname{tr}\!\left[V^{-1}(X-M)^{T}U^{-1}(X-M)\right]\right\}.

Here \(M\) is a real \(n\times p\) matrix, \(U\) is a symmetric positive-definite \(n\times n\) matrix, and \(V\) is a symmetric positive-definite \(p\times p\) matrix. The notation \(|A|\) denotes the determinant of \(A\).

The dimensions ensure that the expression is well-defined. The centered matrix \(X-M\) is \(n\times p\); multiplying it on the left by \(U^{-1}\) uses an \(n\times n\) matrix, and multiplying on the right by \(V^{-1}\) uses a \(p\times p\) matrix. The trace in the exponent is therefore a scalar. Positive definiteness means \(a^TUa>0\) for every nonzero \(a\in\mathbb R^n\), and \(b^TVb>0\) for every nonzero \(b\in\mathbb R^p\). In particular, both matrices are invertible and have positive determinants.

The word scale is important: \(U\) and \(V\) are the matrices appearing in this density’s parameterization. Their names indicate which matrix dimension they act on. They are not, individually, covariance matrices for the whole random matrix.

Worked Example: Evaluating a Matrix-Normal Density

Let \(X\sim MN_{2,3}(M,U,V)\), where \(M\) is the zero matrix, \(U=\begin{pmatrix}1&0\\0&4\end{pmatrix}\), and \(V=\begin{pmatrix}1&0&0\\0&4&0\\0&0&9\end{pmatrix}\). Evaluate the density at \(X=\begin{pmatrix}1&0&0\\0&2&0\end{pmatrix}\).

The determinants are \(|U|=4\) and \(|V|=36\). Since both scales are diagonal, the trace in the exponent is the sum of the squared entries of \(X\), each divided by the corresponding row and column scale:

\operatorname{tr}(V^{-1}X^TU^{-1}X) =\frac{1^2}{1\cdot1}+\frac{2^2}{4\cdot4} =1+\frac14=\frac54.

There are \(np=6\) entries. Thus the density value is

f(X)=(2\pi)^{-3}4^{-3/2}36^{-1} \exp\!\left(-\frac12\cdot\frac54\right) =\frac{1}{288(2\pi)^3}e^{-5/8}.

The determinant powers differ because the row scale is repeated across \(p=3\) columns, while the column scale is repeated across \(n=2\) rows.

Why These Conditions Give a Density

The positive-definiteness assumptions do more than permit matrix inverses. They allow square roots of the scale matrices and a change of variables that reduces the density to the product of \(np\) standard normal densities. This establishes that the displayed expression integrates to one.

Theorem: For \(M\in\mathbb R^{n\times p}\), symmetric positive-definite \(U\in\mathbb R^{n\times n}\), and symmetric positive-definite \(V\in\mathbb R^{p\times p}\), the matrix-normal density displayed above is normalized: its integral over \(\mathbb R^{n\times p}\) equals one.

Whiten the centered matrix.
Let \(A=U^{1/2}\), \(B=V^{1/2}\), and set \(X=M+AYB\), where \(Y\) ranges over all real \(n\times p\) matrices. The positive-definite square roots are invertible.

Transform the exponent and volume element.
The exponent’s quadratic trace becomes \(\operatorname{tr}(Y^TY)\). The Jacobian for left multiplication by \(A\) is \(|A|^p\), one factor for each column; the Jacobian for right multiplication by \(B\) is \(|B|^n\), one factor for each row. Hence \(dX=|U|^{p/2}|V|^{n/2}dY\).

Integrate the standard Gaussian kernel.
The integral over the entries of \(Y\) factors into \(np\) one-dimensional Gaussian integrals, each equal to \(\sqrt{2\pi}\). The determinant factors cancel the Jacobian factors.

For completeness, the trace transformation follows from \(X-M=AYB\), \(U^{-1}=A^{-2}\), and \(V^{-1}=B^{-2}\):

\operatorname{tr}\!\left[V^{-1}(X-M)^TU^{-1}(X-M)\right] =\operatorname{tr}(B^{-2}BY^TA A^{-2}AYB) =\operatorname{tr}(Y^TY).

In the last equality, \(A A^{-2}A=I\), \(B^{-2}B=B^{-1}\), and cyclic invariance of the trace cancel the remaining \(B\) and \(B^{-1}\). Consequently the integral of the density is

(2\pi)^{-np/2} \int_{\mathbb R^{n\times p}}\exp\!\left(-\frac12\operatorname{tr}(Y^TY)\right)dY =(2\pi)^{-np/2}(2\pi)^{np/2}=1.

The Jacobian calculation uses only ordinary linear changes of coordinates on the matrix entries: left multiplication acts separately on each of the \(p\) columns, and right multiplication acts separately on each of the \(n\) rows. This completes the normalization proof. \(\square\)

The Mean Matrix

Theorem: If \(X\sim MN_{n,p}(M,U,V)\) with the stated positive-definite scales, then every entry of \(X\) is integrable and \(E[X]=M\), where expectation of a matrix is taken entry by entry.

Use the same change of variables \(X=M+AYB\) as above. Under this transformation, the density of \(Y\) is proportional to \(\exp\{-\operatorname{tr}(Y^TY)/2\}\); after normalization, it is the product of \(np\) standard normal densities. Each entry \(Y_{ij}\) has a finite absolute first moment and mean zero, since its one-dimensional density is symmetric about zero. Each entry of \(AYB\) is a finite linear combination of entries of \(Y\), so it too has a finite absolute first moment. Linearity of expectation now gives

E[X]=M+A\,E[Y]\,B=M.

This proves the claim without treating the scale matrices as the mean or as substitutes for the mean. The location parameter is the matrix \(M\); the two scale parameters govern the shape of the density around that location. \(\square\)

Worked Example: Checking Dimensions and Positive Definiteness

Consider a proposed model with \(n=p=2\), mean matrix \(M=\begin{pmatrix}1&0\\2&-1\end{pmatrix}\), and both scales equal to \(S=\begin{pmatrix}2&1\\1&2\end{pmatrix}\). Check whether these parameters are admissible and find the density at \(X=M\).

The mean has the required \(2\times2\) dimension. The scale \(S\) is symmetric and \(2\times2\). For any \((a,b)^T\ne(0,0)^T\),

\begin{pmatrix}a&b\end{pmatrix}S\begin{pmatrix}a\\b\end{pmatrix} =2a^2+2ab+2b^2=(a+b)^2+a^2+b^2>0.

Thus \(S\) is positive definite, so both scale parameters meet the requirements. Its determinant is \(4-1=3\). At \(X=M\), the centered matrix is zero and the exponential factor is one. Because \(n=p=2\), the density is therefore

f(M)=(2\pi)^{-2}|S|^{-1}|S|^{-1} =\frac{1}{4\pi^2}\cdot\frac{1}{3}\cdot\frac{1}{3} =\frac{1}{36\pi^2}.

A singular choice such as \(\begin{pmatrix}1&0\\0&0\end{pmatrix}\) would not be an admissible scale here: it is not positive definite, its inverse does not exist, and the stated full-dimensional density formula cannot be used.

A Redundancy in the Scale Pair

The row and column scales are not uniquely determined by the density unless an additional convention fixes their relative scalar size. For any \(c>0\), replacing \(U\) by \(cU\) and \(V\) by \(V/c\) leaves the density unchanged. Both transformed matrices remain symmetric positive definite and retain their original dimensions.

To verify this, the determinant part of the density is unchanged because

|cU|^{-p/2}|V/c|^{-n/2} =c^{-np/2}|U|^{-p/2}c^{np/2}|V|^{-n/2} =|U|^{-p/2}|V|^{-n/2}.

In the exponent, \((cU)^{-1}=c^{-1}U^{-1}\) and \((V/c)^{-1}=cV^{-1}\). The two scalar factors cancel in the trace. Thus the full density is identical, not merely proportional. \(\square\)

Worked Example: Rescaling Without Changing the Density

In the first example, take \(c=4\). The original scales were \(U=\operatorname{diag}(1,4)\) and \(V=\operatorname{diag}(1,4,9)\). The rescaled pair is

U'=4U=\operatorname{diag}(4,16),\qquad V'=V/4=\operatorname{diag}\!\left(\frac14,1,\frac94\right).

Their determinants are \(|U'|=64\) and \(|V'|=9/16\). The determinant contribution is

|U'|^{-3/2}|V'|^{-1} =64^{-3/2}(9/16)^{-1} =\frac{1}{512}\cdot\frac{16}{9} =\frac{1}{288},

which equals \(4^{-3/2}36^{-1}=1/288\) for the original pair. The trace contribution is unchanged as well: \((U')^{-1}=U^{-1}/4\), while \((V')^{-1}=4V^{-1}\), so their factors cancel in the quadratic trace. Therefore the density at every \(X\), including the matrix evaluated in the first example, is the same under these two scale pairs.

Common Pitfalls and Why the Parameterization Matters

Swapping scale dimensions. The row scale must be \(n\times n\), matching the number of rows of \(X\); the column scale must be \(p\times p\), matching its number of columns.
Using positive semidefinite scales in this density. A singular scale has no inverse and does not define the displayed density with respect to Lebesgue measure on all \(np\) entries. Singular Gaussian models require a different formulation.
Calling \(U\) and \(V\) the mean or the full matrix covariance. The density’s location is \(M\). The matrices \(U\) and \(V\) are scale parameters acting on different dimensions, not standalone covariance matrices for \(X\).
Assuming the scales are uniquely identified. The scalar rescaling \(U\mapsto cU,\ V\mapsto V/c\) leaves the distribution unchanged, so a particular numerical scale pair can depend on the convention used.

These checks matter in applications because an otherwise plausible formula can fail through a dimension mismatch, a singular scale, or an unnoticed parameterization convention. Confirming the dimensions and definiteness first ensures the density is a valid full-dimensional matrix-normal model.

Check Your Understanding

Use the density and parameter conditions above to answer the following.

For an \(n\times p\) random matrix, what are the dimensions of \(M\), \(U\), and \(V\), and which matrices must be positive definite?
Why do the determinant factors in the density have powers \(-p/2\) for \(U\) and \(-n/2\) for \(V\)?
State the change of variables that reduces the matrix-normal exponent to a sum of squared entries.
Explain why the mean matrix is \(M\), rather than either scale matrix.
Show, in words or equations, why replacing \(U,V\) by \(cU,V/c\) for \(c>0\) leaves the density unchanged.