Estimating the Structural Dimension

Why Structural Dimension Is an Eigenvalue Question

In sufficient dimension reduction, the structural dimension is the number of linearly independent predictor directions needed to capture the relationship between the response and the predictors. An inverse-regression method summarizes information about those directions in a matrix. Its eigenvectors describe candidate directions, while its eigenvalues indicate how much of that matrix’s signal lies along each direction. Estimating dimension therefore means deciding how many eigenvalues represent signal rather than sampling noise.

That decision is not automatic. A population matrix may have exact zero eigenvalues, but a sample estimate will generally have small nonzero eigenvalues even when the corresponding population directions carry no signal. Moreover, changing the scale of the matrix changes its eigenvalues without changing the subspace it represents. A useful dimension estimate must therefore combine the ordered spectrum with a scale-aware rule and an assessment of sampling variability.

Coverage is essential. An inverse-regression method can capture only some of the central-subspace directions in a particular setting. If its population matrix has range strictly smaller than the central subspace, counting its nonzero eigenvalues underestimates \(d\). Thus, the spectrum estimates structural dimension only under the coverage conditions appropriate to the chosen method. The earlier material on SIR, SAVE, and PHD addresses why method choice matters.

For a positive semidefinite \(p\)-by-\(p\) matrix \(M\), write its ordered eigenvalues as \(\lambda_1\geq\lambda_2\geq\cdots\geq\lambda_p\geq0\). The sample operator \(\hat M\) has ordered eigenvalues \(\hat\lambda_1\geq\cdots\geq\hat\lambda_p\geq0\). In practice, dimension rules are applied to the latter. For a symmetric operator that can have negative eigenvalues, such as some signed moment constructions, rank concerns nonzero eigenvalues of either sign; a strength-based diagnostic can order their absolute values, while retaining the signs for interpretation.

Population Rank and Sample Eigenvalues

The population rank characterization follows directly from the spectral theorem. A positive semidefinite matrix has an orthonormal eigenbasis and nonnegative eigenvalues. In that basis, it has one nonzero eigenvalue for each independent direction in its range. If it is covering, that range is the central subspace, so the number of nonzero eigenvalues is exactly \(d\).

Proof. By the spectral theorem, there is an orthogonal matrix \(Q\) such that \(M=Q\operatorname{diag}(\lambda_1,\ldots,\lambda_p)Q^\mathsf{T}\), with each \(\lambda_j\geq0\). In these coordinates, the range of \(M\) is spanned by the columns of \(Q\) whose corresponding eigenvalues are nonzero. Its dimension is therefore the number of nonzero eigenvalues, which is the rank of \(M\). Since the range is the central subspace and that subspace has dimension \(d\), exactly \(d\) eigenvalues are positive. Ordering them gives the stated pattern. \(\square\)

The theorem gives an exact population answer, not a direct sample rule. In finite samples, estimation error perturbs the eigenvalues: some that are zero in the population can become positive in the sample. In addition, the design of an estimator can impose a rank limit. For example, a SIR matrix formed from \(H\) nonempty slices has rank at most \(H-1\), because its slice-mean deviations satisfy a weighted sum-to-zero constraint. A sample spectrum cannot reveal more directions than the operator can represent.

Scale-Aware Thresholds and Eigenvalue Gaps

A direct sample rule counts eigenvalues exceeding a threshold \(\tau\): \(\hat d(\tau)=\#\{j:\hat\lambda_j>\tau\}\). The threshold should be chosen relative to the scale and estimation noise of the matrix. An absolute cutoff such as \(0.1\) has no stable meaning if the same operator is multiplied by a constant. One simple normalization, when the trace is positive, is to compare eigenvalues with the trace \(T=\sum_{j=1}^p\hat\lambda_j\), or equivalently examine the proportions \(r_j=\hat\lambda_j/T\).

The normalization has a precise scale property. If the operator is replaced by \(c\hat M\) for \(c>0\), its eigenvalues become \(c\hat\lambda_j\) and its trace becomes \(cT\). Therefore \((c\hat\lambda_j)/(cT)=r_j\): the relative eigenvalue proportions are unchanged. This removes arbitrary positive scalar rescaling, but it does not remove differences caused by changing the method, the predictors, or the estimation procedure.

An eigenvalue gap offers another diagnostic. Define \(\hat\Delta_k=\hat\lambda_k-\hat\lambda_{k+1}\) for \(1\leq k<p\). A relatively large gap after the \(k\)th eigenvalue suggests a break between stronger and weaker directions. Since the gap also scales with the matrix, one can report \(\hat\Delta_k/T\) alongside it. Neither the largest raw gap nor the largest relative gap is automatically the correct dimension: a large gap can occur among eigenvalues that are all small, and weak but real directions may have no sharp separation.

Sampling Variability and Stability

A thresholded sample eigenvalue is an estimate, not a known population quantity. Its size can vary with the sample, slice boundaries, and other choices in constructing the operator. In particular, a small positive eigenvalue could reflect a weak population direction or noise around a zero population eigenvalue. A single spectrum cannot distinguish those explanations with certainty.

A practical diagnostic is to recompute the complete operator across bootstrap samples, resampling observational units in a way that respects the study design. For each replicate, record the ordered eigenvalues, the relative proportions, and the dimension selected by the stated rule. This does not by itself supply a formal test or guarantee that a bootstrap procedure is valid. It does reveal whether a proposed dimension is stable to the observed sampling variation. The calculations should also be repeated across reasonable threshold choices; a result that changes from two to three under a small adjustment deserves more cautious interpretation than one that remains two over a broad range.

Common Pitfall: Treating Every Positive Eigenvalue as a Direction

Wrong reasoning: “The sample matrix has three positive eigenvalues, so the structural dimension is three.” In finite samples, noise can make population-zero eigenvalues positive. Also, a method may fail to cover all central-subspace directions, or its construction may limit the rank it can attain. Positivity alone is therefore not enough.

Correction: Relate the rank interpretation to a covering population operator, then treat the sample spectrum as an estimate. Report how the dimension was selected, use a scale-aware threshold or a clearly interpreted gap, and examine sensitivity to sampling variation. If the spectrum is ambiguous, the appropriate conclusion is uncertainty about dimension, not an artificially precise count.

Dimension estimation matters because it determines how many directions are retained in the reduction. Retaining too few can discard response-relevant information; retaining too many can preserve noise and complicate interpretation. A defensible estimate makes those trade-offs visible by connecting the selected count to the spectrum, scale, method, and observed variability.