Identifiability in Tensor Factor Models

What Does It Mean for a Tensor Factorization to Be Identifiable?

A fitted tensor can be uniquely determined even when the factors used to represent it are not. This distinction matters whenever factors are interpreted as latent profiles, spatial patterns, or mode-specific effects: two different parameter values may encode exactly the same observable tensor. Identifiability analysis asks which features are determined by the model and which are choices of representation.

Consider first a deterministic tensor representation. A parameterization is identifiable if no distinct admissible parameter value produces the same represented tensor, apart from any explicitly allowed equivalences. In statistical models, the corresponding definition uses the observable probability distribution: two parameter values are observationally equivalent if they induce the same distribution. Here we focus on the factorization of a fitted tensor; when a model's observable law depends on that tensor, the same ambiguities can carry over to its parameters.

Definition: A tensor parameterization is identifiable if equality of the represented tensor (or, for a probabilistic model, equality of the observable distributions) implies equality of the parameters, subject to stated equivalences such as reordering components. A quantity is identifiable if it has the same value for every parameterization that gives the same observable object.

The tensor itself may be identifiable while its factors are identifiable only up to transformations. It is therefore useful to state the target precisely: the whole parameter tuple, a collection of rank-one components, or a mode-wise column space may have different identifiability properties.

CP Factors: Scaling and Permutation

For a three-way tensor, a rank-\(R\) CP representation is

\mathcal X=\sum_{r=1}^R \lambda_r\,a_r\otimes b_r\otimes c_r,

where \(a_r\in\mathbb R^{p_1}\), \(b_r\in\mathbb R^{p_2}\), \(c_r\in\mathbb R^{p_3}\), and the weights \(\lambda_r\) are nonzero. Equivalently, weights can be absorbed into one of the factor vectors. The outer product \(a_r\otimes b_r\otimes c_r\) has entries \((a_r)_i(b_r)_j(c_r)_k\).

Each component has a scaling ambiguity: for any nonzero scalars \(s_r,t_r\), replacing \(a_r,b_r,c_r\) by \(s_ra_r,t_rb_r,(s_rt_r)^{-1}c_r\) leaves its outer product unchanged. The components can also be reordered without changing their sum. These are intrinsic symmetries of the representation, not evidence that the fitted tensor is ambiguous.

Worked Example: A CP Component Has No Unique Scale Allocation

Let \(a=(1,2)^\mathsf T\), \(b=(3,1)^\mathsf T\), and \(c=(1,-1)^\mathsf T\). The single component \(a\otimes b\otimes c\) is unchanged if \(a\) is multiplied by \(2\) and \(c\) by \(1/2\).

For example, its entry at index \((1,1,1)\) is \(1\cdot3\cdot1=3\). In the rescaled representation that entry is \(2\cdot3\cdot(1/2)=3\). At index \((2,1,2)\), the original entry is \(2\cdot3\cdot(-1)=-6\); the rescaled entry is \(4\cdot3\cdot(-1/2)=-6\). Every entry agrees because the product of the three scaling factors is \(2\cdot1\cdot(1/2)=1\).

Thus the vectors' individual lengths are not determined by the tensor. One common convention makes each factor column have unit Euclidean norm and places the component's magnitude in \(\lambda_r\). A sign convention is still needed if a unique displayed parameterization is desired: signs can be moved between factors while their product remains unchanged.

Permutation creates a separate ambiguity. For any permutation \(\pi\) of \(1,\ldots,R\), summing the components in the order \(\pi(1),\ldots,\pi(R)\) gives the same tensor. Sorting components by a chosen rule can provide a display convention, but a tie in that rule does not create an intrinsic ordering.

Tucker Factors: Invertible Changes of Coordinates

A Tucker representation of an order-\(K\) tensor is

\mathcal X=\mathcal G\times_1 A^{(1)}\times_2 A^{(2)}\cdots\times_K A^{(K)},

where \(A^{(k)}\) is a \(p_k\times r_k\) factor matrix, \(\mathcal G\) is an \(r_1\times\cdots\times r_K\) core tensor, and \(\times_k\) denotes multiplication along mode \(k\). If \(Q_k\) is any invertible \(r_k\times r_k\) matrix, then

A^{(k)}_{\mathrm{new}}=A^{(k)}Q_k,\qquad \mathcal G_{\mathrm{new}}=\mathcal G\times_k Q_k^{-1}

preserves \(\mathcal X\) when the other mode factors are unchanged. Applying such transformations in several modes also preserves the tensor. This follows because the new core's coordinate change is canceled by the corresponding change in the factor matrix. As a result, Tucker factor matrices are generally not identifiable as particular bases: they can change by invertible transformations while their column spaces remain the same.

An orthonormality constraint \(A^{(k)\mathsf T}A^{(k)}=I\) reduces the allowed transformations to orthogonal matrices, but it does not remove the rotational ambiguity. If \(Q_k^\mathsf TQ_k=I\), then \(A^{(k)}Q_k\) is still orthonormal, and a compensating change in the core leaves the tensor unchanged. Orthonormal factors can be convenient and can specify a basis convention, but orthonormality alone does not identify a unique basis.

When Tucker Mode Subspaces Are Identifiable

Although the Tucker bases can rotate or change coordinates, their column spaces can be determined by the fitted tensor under a rank condition. Write \(\mathcal X_{(k)}\) for the mode-\(k\) unfolding, which arranges the mode-\(k\) index as the row index. For full-column-rank factor matrices, the unfolding has the form

X_{(k)}=A^{(k)}G_{(k)} \left(A^{(K)}\otimes\cdots\otimes A^{(k+1)} \otimes A^{(k-1)}\otimes\cdots\otimes A^{(1)}\right)^\mathsf T,

with the Kronecker factors ordered consistently with the unfolding convention.

Theorem: Suppose each \(A^{(j)}\) has full column rank \(r_j\), and the mode-\(k\) core unfolding \(G_{(k)}\) has full row rank \(r_k\). Then \(\operatorname{rank}(X_{(k)})=r_k\) and \(\operatorname{col}(X_{(k)})=\operatorname{col}(A^{(k)})\). Consequently, for a fixed multilinear rank with these rank conditions, the mode-\(k\) factor subspace is identifiable from \(\mathcal X\), although its basis is not.

Proof. The Kronecker product of the other factor matrices has full column rank: each factor has a left inverse, and the Kronecker product of those left inverses is a left inverse for their Kronecker product. Its transpose is therefore full row rank and maps onto the full core-coordinate space. Multiplying the full-row-rank matrix \(G_{(k)}\) on the right by this surjective matrix preserves its row rank. Call the resulting matrix \(M\); it has row rank \(r_k\). Thus \(X_{(k)}=A^{(k)}M\). Since \(M\) has \(r_k\) rows and full row rank, its columns span \(\mathbb R^{r_k}\). Therefore the columns of \(A^{(k)}M\) span exactly the columns of \(A^{(k)}\), and \(\operatorname{rank}(X_{(k)})=r_k\). The unfolding is computed from \(\mathcal X\), so its column space is determined by \(\mathcal X\). \(\square\)

Worked Example: Recovering a Tucker Mode Subspace

Let \(A^{(1)}\) be the \(3\times2\) matrix with columns \(e_1=(1,0,0)^\mathsf T\) and \(e_2=(0,1,0)^\mathsf T\). Let the other two factor matrices be \(2\times2\) identity matrices. Choose a core whose mode-1 unfolding is

G_{(1)}= \begin{pmatrix} 1&0&0&0\\ 0&1&0&0 \end{pmatrix}.

The core unfolding has row rank \(2\), so \(X_{(1)}=A^{(1)}G_{(1)}\) and its columns span \(\operatorname{span}(e_1,e_2)\). In fact, the first two columns of \(X_{(1)}\) are \(e_1,e_2\), and the remaining columns are zero. Any other full-column-rank Tucker representation with mode-1 dimension \(2\) must have a mode-1 factor whose column space contains \(\operatorname{col}(X_{(1)})\). Both spaces have dimension \(2\), so they must be equal. The basis vectors themselves may differ, but the fitted mode-1 subspace cannot.

The rank hypothesis is essential. If the core unfolding has deficient row rank, some proposed factor directions do not contribute to the tensor. The tensor cannot determine those unused directions, even if the factor matrix itself has full column rank.

Worked Example: A Nominal Factor Direction That Cannot Be Identified

Take \(A^{(1)}=[e_1\ e_2]\) in \(\mathbb R^{3\times2}\), and let the other two factor matrices be \(I_2\). Let the only nonzero core entry be \(G_{111}=1\). The resulting tensor is \(e_1\otimes e_1\otimes e_1\), and the mode-1 core unfolding has rank \(1\), not \(2\).

Now replace \(A^{(1)}\) by \(A_{\mathrm{new}}^{(1)}=[e_1\ e_3]\), keeping the same core and other factors. The columns of \(A_{\mathrm{new}}^{(1)}\) are still linearly independent, but the second column is multiplied only by zero core entries. The represented tensor remains \(e_1\otimes e_1\otimes e_1\), while the two proposed mode-1 factor subspaces, \(\operatorname{span}(e_1,e_2)\) and \(\operatorname{span}(e_1,e_3)\), differ. A normalization such as orthonormal columns cannot identify the missing direction; the deficient rank is the obstacle.

CP Uniqueness and Rank Conditions

CP has scaling and permutation ambiguities too, but it does not have the general Tucker freedom to apply arbitrary invertible rotations among factor columns. Under suitable rank conditions, its rank-one components are unique up to the unavoidable scaling and permutation operations. One widely used sufficient condition is Kruskal's condition for a three-way CP decomposition.

Theorem: Let \(A=[a_1,\ldots,a_R]\), \(B=[b_1,\ldots,b_R]\), and \(C=[c_1,\ldots,c_R]\) be the factor matrices of a three-way rank-\(R\) CP representation. The Kruskal rank of a matrix is the largest integer \(q\) such that every set of \(q\) columns is linearly independent. If the Kruskal ranks satisfy \(k_A+k_B+k_C\geq 2R+2\), the CP representation is unique up to a common permutation of its components and reciprocal rescalings within each component.

This is a sufficient condition, not a necessary one: failure to meet the inequality does not prove nonuniqueness. It also makes clear why normalizing columns is not, by itself, a uniqueness theorem. The rank condition addresses whether the component structure can be separated; normalization addresses how scale is assigned after that structure is unique.

Worked Example: Checking Kruskal’s Condition

Consider a \(3\times3\times3\) tensor represented with \(R=3\), with all three factor matrices equal to \(I_3\). Every set of three columns in each matrix is linearly independent, so \(k_A=k_B=k_C=3\). The condition gives

k_A+k_B+k_C=3+3+3=9,\qquad 2R+2=2(3)+2=8.

Since \(9\geq8\), Kruskal’s condition applies: the rank-three CP components are unique up to permutation and componentwise reciprocal rescalings. Unit-norm columns can then fix their magnitudes, with sign conventions used to settle any remaining sign choices. The inequality guarantees uniqueness only up to those stated equivalences, not a unique ordering or sign display without conventions.

Common Pitfall and Practical Consequences

Common mistake: “The fitted tensor has a unique value, and I normalized the columns, so every factor is uniquely identified.” This conclusion confuses representation with identification. In Tucker models, rotations preserve orthonormality and can still be absorbed into the core. In either Tucker or CP models, a rank-deficient core or inadequate component rank conditions can leave directions or components unresolved. Normalization removes selected parameter symmetries; it cannot create information absent from the represented tensor.

A careful interpretation separates three questions: whether the tensor is determined, whether its mode subspaces are determined, and whether a particular factor basis or component ordering is determined. For Tucker models, full mode-unfolding ranks identify the mode subspaces, while a basis requires an additional convention or structural restriction. For CP models, a condition such as Kruskal’s can establish uniqueness of the component decomposition modulo scaling and permutation; normalization then makes the chosen representation easier to compare across fits.

This distinction is important when reporting latent profiles or comparing estimates across samples. If the model identifies only a subspace, a claim about a particular basis vector needs additional justification. If a CP fit is unique only up to permutation, components should not be matched by their column position alone. State the equivalences and rank conditions alongside any substantive interpretation.

Check Your Understanding

Answer each question by distinguishing tensor, subspace, and factor identifiability.

In a CP component, what constraint must the three rescaling factors satisfy for the outer product to remain unchanged?
Why does orthonormality of Tucker factor columns leave a rotational ambiguity?
State the mode-\(k\) rank condition that identifies the Tucker mode-\(k\) column space from the unfolding.
What does Kruskal’s condition guarantee, and which ambiguities does it leave?
Why can column normalization not identify a Tucker direction that makes no contribution to the tensor?