Identifiability of Separable Covariance Factors

Why Factor Identifiability Needs a Scale Convention

A separable covariance model represents the covariance of a vectorized array as a Kronecker product of mode-wise covariance matrices. The representation is compact and gives each factor a useful mode-specific interpretation. But without a scale convention, those factors are not individually determined by the product: multiplying one factor by a positive constant and another by its reciprocal leaves the covariance unchanged.

The issue is structural, not numerical. Even if the full covariance matrix were known exactly, its unnormalized factors would not be unique. We will impose a determinant-one convention on every mode factor and place the overall magnitude in a separate positive scalar. Under this convention, equality of covariance products does force equality of the corresponding factors.

The scalar \(\lambda\) carries the overall scale. The determinant-one conditions fix the scale of each mode factor separately. Other normalizations can also be used, but the conditions must remove the factor-rescaling ambiguity; the theorem below concerns this explicit convention.

A Partial-Trace Identity

The proof uses a matrix operation that removes all but one Kronecker factor. For a matrix whose rows and columns are indexed by tuples \((i_K,\ldots,i_1)\), the partial trace over all modes except \(k\) sums entries whose indices agree in each removed mode, leaving a \(p_k\times p_k\) matrix. Denote this operation by \(T_k\).

To verify the identity, write the entry of the Kronecker product indexed by row tuple \((i_K,\ldots,i_1)\) and column tuple \((i'_K,\ldots,i'_1)\) as \(\prod_{j=1}^K (M_j)_{i_j i'_j}\). In the partial trace for mode \(k\), the indices in every mode \(j\ne k\) are set equal and summed over \(i_j\). Each such sum contributes \(\sum_{i_j}(M_j)_{i_j i_j}=\operatorname{tr}(M_j)\). The remaining entry is \((M_k)_{i_k i'_k}\). Multiplication by \(\lambda\) gives the stated covariance formula.

Because the factors in the definition are positive definite, their traces are positive. Therefore each \(T_k(C)\) is a positive scalar multiple of \(\Sigma_k\). This gives a direct route from equality of full covariances to proportionality of their corresponding mode factors.

Uniqueness Under Determinant-One Normalization

Proof. Fix a mode \(k\). Applying \(T_k\) to both expressions for \(C\) gives \(\lambda\left(\prod_{j\ne k}\operatorname{tr}(\Sigma_j)\right)\Sigma_k =\mu\left(\prod_{j\ne k}\operatorname{tr}(\Gamma_j)\right)\Gamma_k\). Every scalar multiplying a factor is positive, since the overall scales and the traces of positive-definite matrices are positive. Consequently, \(\Gamma_k=c_k\Sigma_k\) for some \(c_k>0\).

Taking determinants yields \(1=\det(\Gamma_k)=\det(c_k\Sigma_k)=c_k^{p_k}\det(\Sigma_k)=c_k^{p_k}\). Since \(c_k>0\) and \(p_k\) is a positive integer, this forces \(c_k=1\). Thus \(\Gamma_k=\Sigma_k\). The argument applies to each mode, so all corresponding factors agree. Substituting these equal factors into the original equality gives \(\lambda M=\mu M\), where \(M=\Sigma_K\otimes\cdots\otimes\Sigma_1\) is invertible because each factor is positive definite. Hence \(\lambda=\mu\). \(\square\)

The theorem is an identifiability statement: there cannot be two different normalized parameter values that generate the same covariance. It does not assert that a covariance can be estimated accurately from a particular sample size, or that a chosen estimation algorithm will find the parameters reliably.

Recovering Factors and the Overall Scale

The same identities give an explicit recovery formula when the full covariance \(C\) is known. For each mode, \(T_k(C)\) is a positive scalar multiple of \(\Sigma_k\). If \(D=a\Sigma_k\) with \(a>0\), then \(\det(D)=a^{p_k}\det(\Sigma_k)=a^{p_k}\). Therefore \(\Sigma_k=D/\det(D)^{1/p_k}\). Applying this to the partial trace gives \(\Sigma_k=T_k(C)/\det(T_k(C))^{1/p_k}\).

The overall scale is also determined. Let \(N=\prod_{k=1}^K p_k\), the dimension of \(C\). The eigenvalues of a Kronecker product are all products of one eigenvalue from each factor. Equivalently, \(\det(A\otimes B)=\det(A)^{\dim(B)}\det(B)^{\dim(A)}\). Applying this determinant identity repeatedly gives \(\det(C)=\lambda^N\prod_{k=1}^K\det(\Sigma_k)^{N/p_k}=\lambda^N\). Hence \(\lambda=\det(C)^{1/N}\), using \(\lambda>0\).

Why the Result Matters—and a Common Pitfall

A covariance factor is often interpreted as describing dependence along one mode, while the overall scalar describes covariance magnitude. Identifiability makes that interpretation mathematically coherent: once the normalization is fixed, the same population covariance cannot correspond to two different collections of mode factors. It also clarifies what a fitted factor means when comparing models or reporting results. The theorem concerns the parameterization itself; it does not remove sampling variability or guarantee that estimates from finite data are stable.

• Pitfall: assuming a Kronecker product uniquely determines its raw factors. That reasoning overlooks reciprocal rescalings. In general, \((cA)\otimes(B/c)=A\otimes B\) for \(c>0\). Positive definiteness does not prevent this ambiguity.
• Correction: state the normalization before claiming uniqueness. Under determinant-one normalization, the partial-trace argument first shows that corresponding factors are proportional; their determinants then force the proportionality constants to equal one.
• Pitfall: treating the scalar as another free factor scale. If every mode factor is separately allowed to absorb scale, the representation again has redundant descriptions. Keeping one overall \(\lambda\) and fixing every mode determinant separates these roles.