Latent Tensor Factor Models

Why Model Tensor Factors Latently?

A tensor-valued observation may have many entries but still vary through a much smaller collection of latent scores. A latent tensor factor model describes this structure by mapping a lower-dimensional score tensor into the observation through a loading matrix for each mode. This formulation makes two consequences explicit: the signal has bounded multilinear rank, and the observation covariance is the transformed covariance of the scores plus the covariance of measurement error.

The model below is a probabilistic version of a Tucker representation. The loadings describe how latent coordinates map into each observed mode; the score tensor describes the joint variation in those coordinates. The distinction matters: the signal is low-rank because of the loading maps, while the covariance also depends on the distribution of the scores and on the noise.

Model and Notation

Let the observed random tensor $Y$ have dimensions $p_1\times\cdots\times p_K$, where $K\geq 2$. For mode $k$, choose a latent dimension $r_k\leq p_k$ and a fixed loading matrix $A_k\in\mathbb{R}^{p_k\times r_k}$. Let the latent score tensor $Z$ have dimensions $r_1\times\cdots\times r_K$, and let $M$ be a fixed mean tensor of dimensions $p_1\times\cdots\times p_K$. The multilinear product $\times_k$ applies a matrix along mode $k$.

Definition: A latent tensor factor model has the form $$ Y=M+Z\times_1 A_1\times_2 A_2\cdots\times_K A_K+E, $$ where $E$ is a mean-zero noise tensor. Assume $E[Z]=0$, $E[E]=0$, and that $Z$ and $E$ are independent. The latent signal is $S=Z\times_1 A_1\cdots\times_K A_K$, so $Y=M+S+E$.

The loading matrices are fixed in this formulation. No full-column-rank assumption is needed for the rank upper bounds or covariance identities below. For covariance formulas, assume the entries of $Z$ and $E$ have finite second moments. Write $\operatorname{Cov}(\operatorname{vec}Z)=\Omega$ and $\operatorname{Cov}(\operatorname{vec}E)=\Gamma$, where vectorization stacks entries in column-major order. Use the same convention for $Y$ and $S$.

Worked Example: Mapping a Score Matrix to an Observation

Take $K=2$, $A_1$ of size $3\times2$, $A_2$ of size $2\times2$, and the score matrix $Z=I_2$. Set $$ A_1=\begin{pmatrix}1&0\\0&1\\1&1\end{pmatrix}, \qquad A_2=\begin{pmatrix}1&1\\0&1\end{pmatrix}. $$ With no mean or noise, the signal is $S=A_1ZA_2^{\mathsf T}=A_1A_2^{\mathsf T}$.

The two columns of $A_2^{\mathsf T}$ are $(1,1)^{\mathsf T}$ and $(0,1)^{\mathsf T}$. Multiplying each row of $A_1$ by these columns gives $$ S= \begin{pmatrix} 1\cdot1+0\cdot1 & 1\cdot0+0\cdot1\\ 0\cdot1+1\cdot1 & 0\cdot0+1\cdot1\\ 1\cdot1+1\cdot1 & 1\cdot0+1\cdot1 \end{pmatrix} = \begin{pmatrix}1&0\\1&1\\2&1\end{pmatrix}. $$ Its first two rows form a $2\times2$ identity matrix, so its matrix rank is 2. The construction gives a $3\times2$ signal from a $2\times2$ score matrix; in general the matrix rank cannot exceed either latent dimension.

Why the Signal Has Low Multilinear Rank

For a tensor $T$, its mode-$k$ unfolding $T_{(k)}$ is the matrix whose rows index mode $k$, while its columns index all remaining modes in a fixed consistent order. The mode-$k$ rank is $\operatorname{rank}(T_{(k)})$. The list of these ranks is the tensor’s multilinear rank. This is the rank notion directly controlled by the loading matrices in the model.

Theorem: For $S=Z\times_1 A_1\cdots\times_K A_K$, the mode-$k$ unfolding satisfies $$ S_{(k)}=A_k Z_{(k)} \left(A_K\otimes\cdots\otimes A_{k+1}\otimes A_{k-1}\otimes\cdots\otimes A_1\right)^{\mathsf T}. $$ Consequently, $\operatorname{rank}(S_{(k)})\leq\operatorname{rank}(A_k)\leq r_k$, for every $k$.

Proof. In the mode-$k$ unfolding, applying $A_k$ multiplies on the left. Applying a loading matrix in any other mode transforms the columns, and the combined column transformation is the Kronecker product of those other loading matrices in the displayed order. This gives the stated unfolding identity. For any compatible matrices $U,V,W$, $\operatorname{rank}(UVW)\leq\operatorname{rank}(U)$. Applying this fact with $U=A_k$ yields $\operatorname{rank}(S_{(k)})\leq\operatorname{rank}(A_k)$, which is at most the number $r_k$ of columns of $A_k$. $\square$

This is an upper bound, not a guarantee of equality. A score tensor may fail to use some latent directions, or a loading matrix may have dependent columns. Therefore, specifying latent dimensions $r_k$ does not by itself assert that the signal has exactly those mode ranks.

Worked Example: Checking the Multilinear Rank Bound

In the preceding matrix example, mode 1 has dimension 3 and latent dimension $r_1=2$, while mode 2 has dimension 2 and latent dimension $r_2=2$. For a matrix, the two mode unfoldings are the matrix and its transpose, so both have rank 2. Thus the signal has multilinear rank $(2,2)$, satisfying the bounds $2\leq r_1$ and $2\leq r_2$.

The bound can be strict. Keep the same loading matrices but take $Z$ to be the zero matrix. Then $S=A_1ZA_2^{\mathsf T}=0$, whose mode ranks are $(0,0)$, even though both latent dimensions equal 2. The latent dimensions limit the ranks; they do not force the scores to occupy every latent direction.

Observation Covariance

To derive the covariance, define the matrix that maps the vectorized score tensor to the vectorized signal: $$ B=A_K\otimes A_{K-1}\otimes\cdots\otimes A_1. $$ For column-major vectorization, the multilinear product obeys $\operatorname{vec}(S)=B\operatorname{vec}(Z)$. Since $M$ is fixed, it affects the mean but not the covariance.

Theorem: If $Z$ and $E$ are independent with finite second moments, then $$ E[\operatorname{vec}(Y)]=\operatorname{vec}(M), \qquad \operatorname{Cov}(\operatorname{vec}Y)=B\Omega B^{\mathsf T}+\Gamma. $$

Derivation. The vectorized observation is $\operatorname{vec}(M)+B\operatorname{vec}(Z)+\operatorname{vec}(E)$. The two random terms have mean zero, giving the stated mean. For the covariance, expand the centered outer product. The score contribution is $B\Omega B^{\mathsf T}$, since $$ \operatorname{Cov}(B\operatorname{vec}Z) =B\operatorname{Cov}(\operatorname{vec}Z)B^{\mathsf T} =B\Omega B^{\mathsf T}. $$ The noise contribution is $\Gamma$. Each cross-covariance is zero: independence and the zero means give $E[\operatorname{vec}(Z)\operatorname{vec}(E)^{\mathsf T}]=0$, and its transpose is zero as well. Adding the contributions proves the formula. $\square$

Worked Example: Computing a Covariance With One Latent Score

Let $K=2$, with $A_1=(1,2)^{\mathsf T}$, $A_2=(1,-1)^{\mathsf T}$, and scalar score $z$ satisfying $E[z]=0$ and $\operatorname{Var}(z)=3$. Let the four entries of $E$ have independent mean-zero noise with variance 1. Thus $\Gamma=I_4$. The signal matrix is $S= zA_1A_2^{\mathsf T}$, and column-major vectorization gives $$ B=A_2\otimes A_1= \begin{pmatrix}1\\2\\-1\\-2\end{pmatrix}. $$

Here $\Omega=(3)$, so the signal covariance is $3BB^{\mathsf T}$. Direct multiplication gives $$ 3BB^{\mathsf T}= \begin{pmatrix} 3&6&-3&-6\\ 6&12&-6&-12\\ -3&-6&3&6\\ -6&-12&6&12 \end{pmatrix}. $$ Adding $I_4$ increases each diagonal entry by 1 and leaves the off-diagonal entries unchanged. Therefore $$ \operatorname{Cov}(\operatorname{vec}Y)= \begin{pmatrix} 4&6&-3&-6\\ 6&13&-6&-12\\ -3&-6&4&6\\ -6&-12&6&13 \end{pmatrix}. $$ The signal covariance has rank one because it is a scalar multiple of $BB^{\mathsf T}$. The observation covariance is full rank here because independent noise adds $I_4$.

Separable Score Covariance

A useful special case is a latent score tensor with separable covariance. Assume that, for positive semidefinite matrices $C_k\in\mathbb{R}^{r_k\times r_k}$, $$ \Omega=C_K\otimes\cdots\otimes C_1. $$ This assumption concerns the covariance of the scores; it does not follow merely from their tensor shape. Substituting into the covariance formula and using the mixed-product rule for Kronecker products gives $$ B\Omega B^{\mathsf T} =(A_KC_KA_K^{\mathsf T})\otimes\cdots\otimes(A_1C_1A_1^{\mathsf T}). $$ Thus the signal covariance is separable across modes under this score-covariance assumption. Adding general noise covariance $\Gamma$ need not preserve separability.

Worked Example: Transforming Separable Score Covariance

Consider two modes with $A_1=\operatorname{diag}(1,2)$, $A_2=\operatorname{diag}(1,3)$, and score covariance $\Omega=I_2\otimes I_2$. In this case $C_1=C_2=I_2$, so the two transformed mode covariances are $$ A_1C_1A_1^{\mathsf T}=\begin{pmatrix}1&0\\0&4\end{pmatrix}, \qquad A_2C_2A_2^{\mathsf T}=\begin{pmatrix}1&0\\0&9\end{pmatrix}. $$ Therefore, under column-major vectorization, the signal covariance is $$ (A_2C_2A_2^{\mathsf T})\otimes(A_1C_1A_1^{\mathsf T}) =\operatorname{diag}(1,4,9,36). $$ For example, the fourth diagonal entry is $9\cdot4=36$, the product of the second-mode variance 9 and first-mode variance 4. If independent unit-variance noise is added to every observation entry, the observation covariance becomes $\operatorname{diag}(2,5,10,37)$.

Interpretation and a Common Pitfall

The loading matrices restrict the directions in which the latent signal can vary. The covariance formula describes how uncertainty in the scores is carried into the observed coordinates, then adds uncertainty from the noise. These are related but distinct statements: a low-multilinear-rank signal does not imply that the noisy observation has low multilinear rank, and a low-rank signal covariance does not imply a low-rank observation covariance.

Wrong reasoning: “The signal has mode ranks bounded by $r_k$, so the observed tensor must have those same rank bounds.” This ignores $E$. Adding a general noise tensor can increase the mode ranks of an observed realization.
Correction: Apply the rank bound to $S$, not automatically to $Y=M+S+E$. A covariance statement also requires attention to the noise: if $\Gamma$ is positive definite, then $B\Omega B^{\mathsf T}+\Gamma$ is positive definite, even when the signal covariance is low-rank.
Another common error: Treating the score covariance as separable just because the score is a tensor. Separability is an additional covariance assumption; for general $\Omega$, use $B\Omega B^{\mathsf T}$ without the Kronecker simplification.

Key takeaway: The mode-wise loading maps impose the signal’s multilinear rank bounds. The vectorized observation covariance is $B\Omega B^{\mathsf T}+\Gamma$; it reduces to a Kronecker product for the signal only when the latent score covariance is separable.

Check Your Understanding

Use the model and vectorization convention from this tutorial.

For three modes, with loading matrices $A_1,A_2,A_3$, write the matrix $B$ mapping $\operatorname{vec}(Z)$ to $\operatorname{vec}(S)$.
Why does the mode-$k$ rank of the signal not exceed $r_k$, and why need it not equal $r_k$?
If the noise is independent of the scores but has covariance $\Gamma$, state the observation covariance in terms of $B$, $\Omega$, and $\Gamma$.
What additional assumption permits the signal covariance to be written as a Kronecker product of transformed mode covariances?
Can a low-rank signal have a full-rank observation covariance? Explain using the role of the noise covariance.

What You'll Learn