Sequential Tests for Central-Subspace Dimension

Why Test a Candidate Dimension?

Estimating a structural dimension from a scree plot or an eigenvalue threshold is useful for exploration, but the decision can be unclear when some directions are weak. A formal test makes the question explicit: is the inverse-regression operator compatible with having at most $r$ nonzero directions, or is there evidence of another one? Testing successive candidate dimensions also clarifies what a rejection and a non-rejection do—and do not—establish.

The tests in this tutorial are based on the eigenvalues of the sliced inverse regression (SIR) operator. They directly assess the dimension represented by that operator. Interpreting that dimension as the full central-subspace dimension requires an additional coverage condition; without it, SIR may fail to detect a genuine regression direction.

Population Operator and Candidate Hypotheses

Definition: Let $X$ be a $p$-dimensional predictor with mean $\mu_X$ and nonsingular covariance matrix $\Sigma_X$, and define the standardized predictor $Z=\Sigma_X^{-1/2}(X-\mu_X)$. Divide the response into $H$ slices $S_1,\ldots,S_H$, each with positive probability $p_h=P(Y\in S_h)$. Write $\mu_h=E[Z\mid Y\in S_h]$. The population SIR operator for these slices is $$ M=\sum_{h=1}^{H} p_h\mu_h\mu_h^{T}. $$ It is positive semidefinite. Its rank is the dimension of the span of the slice-specific inverse-regression means.

The central subspace is the smallest subspace $\mathcal S_{Y\mid X}$ such that the conditional distribution of $Y$ given $X$ depends on $X$ only through a projection onto that subspace. Its dimension is the central-subspace dimension. Under the linearity condition, the range of $M$ is contained in the central subspace. Equality of these spaces requires SIR to have adequate coverage of the central subspace. Consequently, an eigenvalue test of $M$ is a test of SIR-detectable dimension; it is a test of the full central-subspace dimension only when that coverage condition is justified.

Let the eigenvalues of $M$, in decreasing order, be $\lambda_1\ge\cdots\ge\lambda_p\ge0$, and let $\hat\lambda_1\ge\cdots\ge\hat\lambda_p\ge0$ be the eigenvalues of the sample SIR operator $\hat M$. For a candidate dimension $r$, the hypotheses are

Candidate-dimension test: $H_0(r):\operatorname{rank}(M)\le r$, against $H_1(r):\operatorname{rank}(M)>r$. Equivalently, the null requires $\lambda_{r+1}=\cdots=\lambda_p=0$; the alternative requires at least one of these eigenvalues to be positive.

A common statistic is the scaled sum of the trailing sample eigenvalues,

T_r=n\sum_{j=r+1}^{p}\hat\lambda_j.

Large values count against the null: they indicate that the estimated operator has appreciable mass outside its leading $r$-dimensional eigenspace. The factor $n$ is appropriate for common fixed-dimension settings in which the sample operator’s deviations are of order $n^{-1/2}$; the actual null calibration still depends on the sampling scheme and estimator.

Why the Trailing Eigenvalues Measure Residual Dimension

The statistic has a useful geometric interpretation that does not depend on the particular eigenvector basis. For a symmetric positive-semidefinite matrix $A$ with ordered eigenvalues $\lambda_1(A)\ge\cdots\ge\lambda_p(A)$, consider all orthogonal projections $P$ of rank $r$. Then

\min_{\operatorname{rank}(P)=r}\operatorname{tr}\{(I-P)A\} =\sum_{j=r+1}^{p}\lambda_j(A).

To verify this, take an orthonormal eigenbasis $u_1,\ldots,u_p$ of $A$, and set $q_j=u_j^TPu_j$. Since $P$ is an orthogonal projection, $0\le q_j\le1$ and $\sum_jq_j=\operatorname{tr}(P)=r$. Therefore

\operatorname{tr}(PA)=\sum_{j=1}^{p}\lambda_j(A)q_j \le \sum_{j=1}^{r}\lambda_j(A),

because the largest possible weighted sum, subject to those bounds and the sum constraint, assigns weight one to the $r$ largest eigenvalues. Equality is attained by projecting onto their eigenvectors. Since $\operatorname{tr}\{(I-P)A\}=\operatorname{tr}(A)-\operatorname{tr}(PA)$, minimizing the residual trace gives the sum of the remaining eigenvalues. In particular, that sum is zero exactly when $\operatorname{rank}(A)\le r$. Applying this result to $\hat M$ shows that $T_r/n$ is the smallest residual SIR trace achievable by an $r$-dimensional projection. $\square$

This characterization explains why the sum is often used rather than testing only $\hat\lambda_{r+1}$: it measures the total estimated signal beyond dimension $r$. Other statistics, such as the largest trailing eigenvalue, are possible, but they require their own calibration.

Calibration: The Reference Distribution Matters

A test statistic is not a test until its null reference distribution or critical value has been specified. There is no universal chi-square cutoff for $T_r$. Under a particular SIR estimator and suitable regularity conditions—such as fixed predictor dimension and slice structure, positive slice probabilities, and an appropriate null expansion—an asymptotic null law may be derived. It is often a weighted sum of chi-square variables, with weights depending on the design, slice probabilities, and estimator. The weights must be obtained for the actual setting; replacing them automatically by one, or using a generic chi-square degrees-of-freedom formula, is not justified.

A practical alternative is a null-constrained parametric bootstrap. Fit a model satisfying $H_0(r)$, simulate data from that fitted null model, and for every bootstrap data set repeat the full analysis: apply the same slicing rule, estimate and standardize the SIR operator, and calculate $T_r$. Compare the observed statistic with the resulting null distribution. This approach requires a credible null model and a bootstrap scheme that reproduces relevant features of the data. In particular, the null restriction must be imposed on the model being simulated, not merely on the displayed eigenvalues after fitting an unrestricted model.

A permutation calibration is not automatically interchangeable with this bootstrap. Permuting responses typically represents a stronger null of no association between $X$ and $Y$; it does not in general represent the composite null that the association has dimension at most $r$. Use permutation only when the exchangeability assumptions and the intended null justify it.

Worked Example: Building a Test From Slice Means

Suppose $p=3$, and a categorical response $S$ has three equally likely slices. Let the standardized predictor satisfy $Z\mid S=h\sim N(\mu_h,I-M)$, where

\mu_1=(1,0,0)^T,\qquad \mu_2=(-1,0,0)^T,\qquad \mu_3=(0,0,0)^T, \qquad M=\operatorname{diag}(2/3,0,0).

This is a valid specification: the average conditional mean is $(\mu_1+\mu_2+\mu_3)/3=0$, and $I-M=\operatorname{diag}(1/3,1,1)$ is positive definite. The marginal covariance of $Z$ is $I-M+M=I$. Directly from the definition,

\sum_{h=1}^{3}p_h\mu_h\mu_h^T =\frac13 \begin{pmatrix} 1&0&0\\0&0&0\\0&0&0 \end{pmatrix} +\frac13 \begin{pmatrix} 1&0&0\\0&0&0\\0&0&0 \end{pmatrix} =\operatorname{diag}(2/3,0,0).

Thus the population SIR operator has eigenvalues $2/3,0,0$, and its rank is one. For the candidate $r=1$, the null is compatible with this operator; for $r=0$, the null is not. In a real sample, the corresponding decisions would require the chosen calibration, not just the population eigenvalues.

Sequential Testing and Its Interpretation

The candidate nulls are nested: if $H_0(r)$ is true, then $H_0(r+1)$ is also true. A sequential procedure tests $r=0,1,\ldots$ in order, stopping at the first candidate whose null is not rejected. The step at $r$ asks whether the data require more than $r$ SIR directions. Rejection means there is evidence against a dimension of at most $r$; non-rejection means the test has not found sufficient evidence against that bound.

A non-rejection does not establish that the dimension is exactly $r$. It may be that the dimension is smaller, or that a direction beyond $r$ is too weak for the available sample to detect. Accordingly, a first non-rejection is best described as the smallest candidate dimension not rejected by this procedure, rather than as a proof of the true dimension.

There is a useful one-sided error guarantee for the first-non-rejection rule. Suppose the true SIR rank is $d_0$, each test has level at most $\alpha$ under its null, and the procedure starts at zero and stops at its first non-rejection. If the selected dimension exceeds $d_0$, then the procedure must have rejected $H_0(d_0)$: it could only reach a candidate above $d_0$ by rejecting at $d_0$. Since $H_0(d_0)$ is true, this event has probability at most $\alpha$. Thus the probability of overestimating the rank is at most $\alpha$. This argument does not bound the probability of selecting a dimension below $d_0$, which can occur when a test fails to detect a weak direction. $\square$

The guarantee concerns the rank of the tested operator under correctly calibrated tests. It does not repair a failure of SIR coverage: a direction absent from the population SIR operator cannot be recovered by more precise testing of that operator.

Worked Example: A Sequential Decision From Calibrated Tests

Suppose a sample of $n=120$ observations gives the ordered eigenvalues $(0.31,0.09,0.022,0.008)$. Assume a suitable null calibration has supplied the following 5% critical values for the candidate tests $r=1,2,3$: $7.8,4.1,2.0$, respectively. These are stipulated calibration results for this example, not universal cutoffs.

For $r=1$, the statistic is

$$ T_1=120(0.09+0.022+0.008)=120(0.120)=14.4. $$

Since $14.4>7.8$, reject $H_0(1)$. For $r=2$,

$$ T_2=120(0.022+0.008)=120(0.030)=3.6. $$

Since $3.6<4.1$, do not reject $H_0(2)$, so the sequential rule stops at two. The appropriate conclusion is that the first non-rejected candidate is two. This does not prove that the rank is exactly two; in particular, the non-rejection could reflect limited power against a third weak direction.

Worked Example: A Null-Constrained Bootstrap Decision

For a candidate $r=1$, suppose the observed eigenvalues are $(0.24,0.031,0.012)$ with $n=100$. The observed statistic is

$$ T_1=100(0.031+0.012)=4.3. $$

A researcher fits a model constrained to have at most one SIR direction, simulates 1,999 bootstrap data sets from that fitted model, and repeats the complete analysis for each. Suppose 73 bootstrap statistics are at least $4.3$. Using the standard plus-one estimate,

\hat p=\frac{73+1}{1999+1}=\frac{74}{2000}=0.037.

At level 0.05, this bootstrap result rejects $H_0(1)$. The conclusion depends on the adequacy of the constrained model and bootstrap; the arithmetic alone does not validate those assumptions. Recomputing the slicing and standardization in each replicate matters because those steps are part of the statistic’s sampling procedure.

Common Pitfalls and Practical Significance

Treating every small eigenvalue as zero. A numerical threshold is not a calibrated test. Sampling variation can produce positive trailing eigenvalues even under a rank-restricted null.
Using a generic chi-square reference. The null law can depend on the SIR construction and the data. Use a derived asymptotic calibration or a defensible null-constrained resampling method.
Reading non-rejection as proof. A non-rejected candidate may be adequate for the data and test, but it is not evidence that every higher direction is absent.
Calling SIR rank the central-subspace dimension without qualification. That interpretation requires adequate SIR coverage. A failure to reject a larger SIR dimension does not diagnose directions SIR cannot see.
Permuting under the wrong null. Independence is stronger than having a low-dimensional but nonzero association. The resampling null must match the candidate-dimension hypothesis.

Formal sequential tests are most useful when the candidate dimensions, statistic, calibration, and stopping rule are stated together. That reporting makes the scientific claim interpretable: it distinguishes evidence for additional inverse-regression directions from a failure to detect them, and it keeps the conclusion tied to the operator and assumptions actually tested.

Check Your Understanding

Answer each question by referring to the candidate null, the statistic, and its calibration where relevant.

For a candidate $r$, state the null and alternative in terms of the rank of the population SIR operator.
Why is the sum of the trailing eigenvalues a basis-invariant measure of residual operator signal?
What does a first non-rejection in a sequential procedure establish, and what does it not establish?
Why might a response-permutation test fail to calibrate the null that the SIR rank is at most $r$?
Under what additional condition can a test of SIR rank be interpreted as a test of central-subspace dimension?