Scale Nonidentifiability in Kronecker Covariance

Why Factor Scales Are Ambiguous

A Kronecker covariance model represents a covariance matrix as a product of smaller matrices. This structure can describe multidimensional measurements compactly, but the factors themselves have a built-in scale ambiguity. Increasing one factor by a positive constant and decreasing another by the reciprocal constant leaves the covariance unchanged. The covariance may therefore be identifiable even when its individual factors are not.

For two dimensions, write the covariance as \(\Sigma=A_2\otimes A_1\), where \(A_1\) and \(A_2\) are symmetric positive-definite matrices of sizes \(p_1\times p_1\) and \(p_2\times p_2\). Positive definiteness ensures that each factor is nonzero and has a positive trace, facts that will be useful for proving uniqueness after normalization.

For every \(c>0\), the Kronecker product satisfies

This identity gives one direction of the ambiguity: reciprocal rescaling never changes the covariance. The more important question is whether there are other ways for two positive-definite factor pairs to give the same covariance. The result below shows that, for two factors, reciprocal scaling is the entire ambiguity.

Exactly Which Factorizations Agree?

The following theorem characterizes all equivalent two-factor representations. It assumes that the factors are positive definite; in particular, neither factor is the zero matrix.

Proof. The “if” direction follows directly from the scalar rule for Kronecker products: \((c^{-1}A_2)\otimes(cA_1)=A_2\otimes A_1\). For the converse, write the entries of the two products using row indices \(i_1,i_2\) and column indices \(j_1,j_2\). Equality says

for every choice of indices. Regard each matrix’s entries as a vector, with its coordinates indexed by its row-column pair. The displayed equality says that the outer products of the two pairs of entry-vectors agree. For nonzero vectors \(x,y,u,v\), equality \(x\otimes y=u\otimes v\) forces \(y\) to be proportional to \(v\): if they were not, a linear functional could be chosen that vanishes on \(y\) but not on \(v\); applying it to the second coordinate would make one side zero and the other nonzero. Substitution then forces \(x\) to be proportional to \(u\), with the two proportionality constants having product one. Applying this fact to the entry-vectors gives \(B_1=cA_1\) and \(B_2=c^{-1}A_2\) for some nonzero real \(c\). Since \(A_1\) and \(B_1\) are positive definite, for any nonzero vector \(z\), \(z^\mathsf{T}B_1z=c\,z^\mathsf{T}A_1z>0\); hence \(c>0\). This proves the claim. \(\square\)

More Than Two Factors

For \(K\) positive-definite factors, write \(\Sigma=A_K\otimes\cdots\otimes A_1\), with \(A_j\) of size \(p_j\times p_j\). The same argument applies to the entry arrays indexed by all \(K\) row-column pairs: equality of two products forces each corresponding factor to be a scalar multiple of the other. Consequently, if \(\Sigma=B_K\otimes\cdots\otimes B_1\) as well, then

The product constraint is necessary because the scalar multiplier on the full Kronecker product is \(\prod_j c_j\), which must equal one. It is also sufficient: any positive constants satisfying that constraint leave \(\Sigma\) unchanged. There are \(K-1\) freely chosen positive scale multipliers; the final one is determined by their product. Therefore, for three or more factors, normalizing just one factor does not generally remove all scale ambiguity.

Proof. First, such a normalized representation can be constructed from any representation. For \(j<K\), let \(c_j=p_j/\operatorname{tr}(A_j)\), which is positive because \(A_j\) is positive definite. Set \(c_K=(\prod_{j=1}^{K-1}c_j)^{-1}\) and replace each \(A_j\) by \(c_jA_j\). The product of the \(K\) multipliers is one, so the Kronecker product is unchanged. For \(j<K\), the new trace is \(c_j\operatorname{tr}(A_j)=p_j\).

For uniqueness, take two normalized representations of the same \(\Sigma\). The factorization result gives \(B_j=c_jA_j\), with positive \(c_j\) and \(\prod_jc_j=1\). For every \(j<K\), the trace constraints imply \(p_j=\operatorname{tr}(B_j)=c_j\operatorname{tr}(A_j)=c_jp_j\). Since \(p_j>0\), \(c_j=1\). The product constraint then gives \(c_K=1\). Thus \(B_j=A_j\) for every \(j\), proving uniqueness. \(\square\)

Statistical Meaning and a Common Pitfall

In a centered multivariate normal model, the covariance determines the distribution. Thus, if the model uses \(\Sigma=A_K\otimes\cdots\otimes A_1\), the scale ambiguity concerns the factor parameters, not different covariance matrices or different normal distributions. For other models, equality of covariance alone need not imply equality of the entire distribution; the identification statement here is specifically about the Kronecker covariance parameterization.

A common mistake is to conclude that a covariance is not identifiable because its factors are not uniquely determined. These are different claims: many factor tuples can represent one covariance, while the covariance itself can still be determined uniquely. A related mistake in a model with three or more factors is to constrain only one factor’s trace. That fixes one scale but leaves reciprocal rescaling possible between the other factors. Constraining the traces of \(K-1\) factors, with the final factor absorbing the balancing scale, resolves the full ambiguity.