Choosing Tensor Rank by Error and Stability

Why Rank Selection Needs More Than a Small Residual

A tensor rank determines how much structure a CP or Tucker model can represent. A rank that is too small may leave meaningful structure in the residuals; a rank that is too large may fit noise, produce little predictive improvement, or yield factors that change substantially under small perturbations of the data. These concerns are related, but they are not interchangeable: reconstruction error measures fit, residual diagnostics look for what the model misses, and stability asks whether the fitted structure is reproducible.

A useful comparison therefore evaluates candidate ranks on data not used to fit them and examines the fitted factors as well as the prediction errors. This is a model-selection strategy, not a guarantee that one rank is uniquely correct. In particular, the rank with the smallest validation error need not be the best scientific summary if its improvement is negligible and its factors are unstable.

For a CP model, compare candidate component counts \(R\). For a Tucker model of order \(K\), compare candidate multilinear rank vectors \((r_1,\ldots,r_K)\). The validation entries must not be used to estimate the core, factors, initialization choices selected by fit, or other model parameters. If tuning choices are made using validation performance, a separate test set or an outer resampling procedure is needed for an unbiased assessment of the entire selection procedure.

What Held-Out Error Estimates

The following calculation clarifies what validation error means. Suppose the true mean at validation entry \(i\) is \(M_i\), and the observed value is \(Y_i=M_i+\varepsilon_i\). Let \(\widehat{M}_i\) be the prediction produced from the training data. Assume, conditional on the training data, that \(E[\varepsilon_i\mid\mathcal{T}]=0\) and \(E[\varepsilon_i^2\mid\mathcal{T}]=\sigma^2\), where \(\mathcal{T}\) denotes all training information. The conditional assumptions matter: independence of an error from the fitted prediction alone does not imply either condition.

To prove the result, write the prediction error as \(Y_i-\widehat{M}_i=(M_i-\widehat{M}_i)+\varepsilon_i\). The fitted prediction and \(M_i\) are fixed after conditioning on \(\mathcal{T}\). Expanding the square gives $$ E[(Y_i-\widehat{M}_i)^2\mid\mathcal{T}] =(M_i-\widehat{M}_i)^2 +2(M_i-\widehat{M}_i)E[\varepsilon_i\mid\mathcal{T}] +E[\varepsilon_i^2\mid\mathcal{T}]. $$ The middle term is zero by the conditional mean assumption, and the last term is \(\sigma^2\). Averaging over validation entries proves the corresponding MSE identity. If errors have different conditional variances, the same derivation holds entry by entry with the appropriate variance in place of \(\sigma^2\). \(\square\)

Training Error, Validation Error, and Residual Structure

For exact least-squares fitting, larger nested model classes cannot have a larger minimum training error. The statement is about the global minimum, not necessarily the output of an algorithm that may stop at a local solution.

Indeed, every candidate \(A\in\mathcal{M}_1\) is also in \(\mathcal{M}_2\). Therefore the infimum of the training loss over \(\mathcal{M}_2\) cannot exceed its infimum over the smaller set: $$ Q_2=\inf_{A\in\mathcal{M}_2}\sum_{i\in\Omega_{\mathrm{tr}}}(Y_i-A_i)^2 \leq \inf_{A\in\mathcal{M}_1}\sum_{i\in\Omega_{\mathrm{tr}}}(Y_i-A_i)^2=Q_1. $$ CP models with at most \(R\) terms are nested as \(R\) increases, since an additional term can have zero weight. Tucker classes with componentwise increasing upper bounds on multilinear rank are nested as well, by embedding a smaller core and factors in the larger class. The conclusion concerns best achievable training error; it does not imply that validation error decreases, or that a numerical fitting algorithm finds that best value. \(\square\)

Residuals on held-out entries are \(e_i=Y_i-\widehat{Y}_i\). Their magnitude contributes to validation error, but their arrangement can also be informative. For example, residuals consistently positive in one mode level and negative in another may indicate that a mode-specific pattern remains unexplained. Residuals clustered by a time, subject, or batch index may indicate dependence or a split that failed to reflect the sampling design. Such structure is a diagnostic clue, not by itself proof that a higher tensor rank is the correct remedy: the pattern could instead reflect an omitted covariate, noise misspecification, or a data-processing problem.

Stability of Factors and Subspaces

Factor comparisons require care because tensor decompositions have intrinsic ambiguities. In a CP decomposition, component order can be permuted, and a component can be rescaled across modes without changing the represented tensor. For example, in a three-way term \(a\otimes b\otimes c\), replacing the factors by \((sa,tb,(st)^{-1}c)\), for nonzero \(s,t\), leaves the term unchanged. Consequently, comparing raw factor entries before matching components and fixing a normalization can make identical decompositions look different.

A practical CP stability analysis fits the same candidate rank to perturbed data or to multiple training resamples, normalizes factor columns, matches components across fits (allowing permutations and sign conventions), and then compares the matched factors. In a Tucker model, individual factor columns are generally not unique: a change of basis in a factor subspace can be offset by a corresponding transformation of the core. Compare the estimated subspaces, for instance through their principal angles, rather than treating particular columns as uniquely meaningful.

Stability is evidence about the reproducibility of a fitted representation, not a substitute for predictive assessment. A stable factor may summarize a persistent pattern while predictions remain mediocre; conversely, predictions can be stable even when individual factors are poorly determined. CP fits can be particularly sensitive near ill-conditioned or nearly nonidentifiable configurations. For Tucker fits, a stable subspace can coexist with rotating columns within that subspace.

Putting the Evidence Together

A defensible rank choice balances predictive performance, residual diagnostics, and stability. For each candidate, record validation error using the same split or repeated splits, inspect residuals for systematic patterns, and assess whether the fitted factors or subspaces persist under reasonable perturbations. When validation errors are close relative to their variation across splits, selecting the smaller or more stable model can be a reasonable parsimony choice, provided it does not leave important residual structure.

Common Mistakes

• Choosing the rank with the smallest training error. For nested rank classes, the globally minimized training error cannot increase with rank. This is precisely why training error alone does not guard against unnecessary complexity; compare held-out prediction instead.
• Assuming validation errors are conditionally mean-zero from independence of the fitted values. Independence from a fitted prediction does not establish conditional mean zero given the training data. The validation-risk derivation requires the conditional error assumptions stated above, or other assumptions that genuinely imply them.
• Calling every large residual evidence for a missing component. Check whether residuals have systematic structure and consider the design, covariates, and error model before attributing them to rank.
• Comparing unaligned factor columns directly. CP permutations and rescalings, and Tucker changes of basis, can alter factor coordinates without changing the represented tensor. Align CP components and compare Tucker subspaces.
• Treating a tiny validation improvement as decisive. Validation error itself varies with the split. If candidate performance is nearly tied, report that uncertainty and weigh parsimony and stability rather than asserting a uniquely determined rank.