Deriving the Tensor-Normal Likelihood

Likelihood from Mode-Wise Whitening

For tensor-valued observations, the Gaussian likelihood has the same basic ingredients as an ordinary multivariate Gaussian likelihood: a determinant term and a quadratic term. The care required is in counting how often each covariance factor acts across the tensor’s modes. Mode-wise whitening makes those counts explicit and also gives a convenient route to the differential of the likelihood.

Suppose \(\mathcal X_1,\ldots,\mathcal X_n\) are independent observations, each an order-\(K\) tensor of dimensions \(d_1\times\cdots\times d_K\). Write \(m=\prod_{k=1}^K d_k\), let \(\mathcal M\) be their common mean tensor, and let \(\Sigma_k\) be a symmetric positive-definite \(d_k\times d_k\) covariance factor for mode \(k\). Under the vectorization convention in which the covariance is \(\Sigma_K\otimes\cdots\otimes\Sigma_1\), the tensor-normal model is a multivariate normal model on the \(m\) entries of each tensor with that separable covariance.

The inverse square root exists because each factor is positive definite. Define the squared whitened Frobenius norm \(Q(\mathcal X)=\|(\mathcal X-\mathcal M)\times_1\Sigma_1^{-1/2}\cdots\times_K\Sigma_K^{-1/2}\|_F^2\). The density with respect to Lebesgue measure on the tensor entries is

Each mode-\(k\) transformation acts on \(m/d_k\) fibers of length \(d_k\). Consequently its determinant contribution is repeated \(m/d_k\) times. The factor \(1/2\) in the exponent of the density turns that count into the power \(m/(2d_k)\). For independent observations, write \(\mathcal E_i=\mathcal X_i-\mathcal M\) and \(Q_i=Q(\mathcal X_i)\). The log-likelihood is therefore

Why the Determinant Exponents Have This Form

The determinant term can be checked directly from the change of variables that whitens the tensor. If a matrix \(A\) acts in mode \(k\), it acts on \(m/d_k\) fibers. In a mode-\(k\) unfolding, this is left multiplication by \(A\), repeated across \(m/d_k\) columns. Thus the absolute Jacobian determinant of that mode transformation is \(|A|^{m/d_k}\). Applying this fact successively to all modes gives the Jacobian for the full whitening map.

Proof. Fix a mode \(k\), and hold the other mode transformations fixed. In the mode-\(k\) unfolding, the transformation by \(A_k\) sends each of the \(m/d_k\) columns through \(A_k\). Its matrix representation is block diagonal with \(m/d_k\) copies of \(A_k\), so its absolute determinant is \(|A_k|^{m/d_k}\). The determinant of the composition of the mode transformations is the product of their determinants. Multiplying these contributions over \(k=1,\ldots,K\) proves the stated Jacobian. \(\square\)

To obtain the density, transform the residual tensor by \(A_k=\Sigma_k^{-1/2}\) in each mode. The whitened entries have the standard \(m\)-dimensional Gaussian density, whose exponential term is \(\exp\{-Q(\mathcal X)/2\}\). The inverse transformation contributes the reciprocal of the whitening Jacobian. Since \(|\Sigma_k^{-1/2}|=|\Sigma_k|^{-1/2}\), that reciprocal is \(\prod_k|\Sigma_k|^{-m/(2d_k)}\), giving the displayed density.

Matrix Differential for One Covariance Factor

To differentiate with respect to \(\Sigma_k\), it is useful to isolate the terms involving that factor. Hold the mean and all other covariance factors fixed. For each observation, whiten its residual in every mode except \(k\), and unfold the resulting tensor along mode \(k\). Denote this \(d_k\times(m/d_k)\) matrix by \(Y_{ik}\), and define the mode-\(k\) scatter matrix \(S_k=\sum_{i=1}^nY_{ik}Y_{ik}^{\mathsf T}\). This scatter matrix depends on the other covariance factors, but not on \(\Sigma_k\). The quadratic contribution for observation \(i\) is \(\operatorname{tr}(\Sigma_k^{-1}Y_{ik}Y_{ik}^{\mathsf T})\), so the log-likelihood terms involving \(\Sigma_k\) are

The differential below is taken along symmetric perturbations \(d\Sigma_k\), as required when \(\Sigma_k\) ranges over symmetric positive-definite matrices.

Proof. For an invertible matrix \(\Sigma\), differentiating \(\Sigma\Sigma^{-1}=I\) gives \(d(\Sigma^{-1})=-\Sigma^{-1}(d\Sigma)\Sigma^{-1}\). Also, the determinant differential is \(d\log|\Sigma|=\operatorname{tr}(\Sigma^{-1}d\Sigma)\). Apply these identities to \(\ell_k\). The determinant term contributes \(-\frac{nm}{2d_k}\operatorname{tr}(\Sigma_k^{-1}d\Sigma_k)\). For the trace term, cyclic invariance of trace gives

Adding the two contributions proves the formula. The matrix multiplying \(d\Sigma_k\) is symmetric because \(\Sigma_k^{-1}\) and \(S_k\) are symmetric. \(\square\)

With the Frobenius inner product, the symmetric matrix in braces is the gradient on the space of symmetric matrices. The differential formula is often the more fundamental statement: it specifies the first-order change for any symmetric perturbation, without requiring a particular vectorization of the free entries.

Common Pitfall and Interpretation

A common mistake is to assign the determinant term for mode \(k\) the power \(n/2\), as though each observation contributed only one copy of that factor. The factor acts on every mode-\(k\) fiber, and there are \(m/d_k\) such fibers per observation. Its log-likelihood coefficient is therefore \(nm/(2d_k)\), not \(n/2\) unless \(m/d_k=1\).

A related error is to build \(S_k\) by whitening mode \(k\) as well as the other modes. That would make the scatter matrix depend on \(\Sigma_k\), even though the differential above treats it as fixed. The correct construction whitens only the other modes; the remaining mode-\(k\) covariance enters through \(\operatorname{tr}(\Sigma_k^{-1}S_k)\).

These formulas matter whenever a separable tensor covariance is fitted: they turn a high-dimensional Gaussian likelihood into mode-specific determinant and scatter contributions. Keeping the dimension count and the whitening convention explicit prevents errors in both likelihood evaluation and matrix differentiation.