Existence and Uniqueness of the Central Subspace

Why Existence and Uniqueness Need an Assumption

Sufficient dimension reduction seeks a subspace of predictor directions that retains all information in $X$ relevant to the conditional distribution of $Y$. More than one subspace can preserve that distribution: a sufficient subspace remains sufficient if extra, irrelevant directions are added. The important question is whether there is a smallest such subspace, containing only directions that are needed.

A natural candidate is the intersection of every sufficient subspace. But intersections do not automatically preserve conditional independence for arbitrary distributions. We will prove that they do under a useful support condition: the predictor vector has a density that is positive almost everywhere throughout its ambient Euclidean space. The argument first establishes a fact about functions of linear projections, then applies it to conditional response probabilities.

Definitions and the Support Condition

Definition: Let $X\in\mathbb R^p$ be a predictor vector and $Y\in\mathbb R^d$ a response. For a linear subspace $S\subseteq\mathbb R^p$, let $P_S$ denote orthogonal projection onto $S$. The subspace $S$ is a sufficient dimension reduction subspace if $$ Y\perp X\mid P_SX. $$ Equivalently, the conditional distribution of $Y$ given $X$ is determined by $P_SX$: for every Borel set $A\subseteq\mathbb R^d$, there is a measurable function $g_A$ such that $$ P(Y\in A\mid X)=g_A(P_SX)\quad\text{almost surely}. $$

The equivalence follows from the definition of conditional independence: once $P_SX$ is known, observing $X$ gives no further information about $Y$. Conditional distributions exist here because the response and predictor take values in Euclidean spaces.

Assumption (full-support density): The distribution of $X$ has a density $f_X$ with respect to $p$-dimensional Lebesgue measure, and $f_X(x)>0$ for Lebesgue-almost every $x\in\mathbb R^p$. Thus a set has probability zero under $X$ if and only if it has Lebesgue measure zero.

This is a sufficient condition for the result, not a claim that every predictor distribution must satisfy it. It ensures that almost-sure statements about $X$ do not hide discrepancies on large parts of the ambient space. The condition is used in translating conditional probabilities into functions that agree almost everywhere with respect to Lebesgue measure.

Worked Example: A Model With a Sufficient Direction

Let $X=(X_1,X_2)^\mathsf T\sim N(0,I_2)$, and, conditional on $X=x$, let $Y$ be Bernoulli with success probability $$ P(Y=1\mid X=x)=\frac{1}{1+\exp(-x_1-2x_2)}. $$ Identify a sufficient subspace.

The conditional distribution of a Bernoulli response is determined by its success probability. Here that probability depends on $x$ only through $x_1+2x_2$. Set $\beta=(1,2)^\mathsf T$ and $S=\operatorname{span}(\beta)$. The projection $P_SX$ determines $\beta^\mathsf TX=X_1+2X_2$, since $\beta^\mathsf TX$ is a function of that projection. Therefore the conditional distribution of $Y$ given $X$ is determined by $P_SX$, and $S$ is sufficient.

This verifies sufficiency, not minimality: to establish that a subspace is the unique smallest sufficient one, we need a result about intersections.

A Projection-Intersection Lemma

The central step is that, under Lebesgue measure, a bounded measurable function that can be expressed using either of two projections can also be expressed using their common subspace. We state the result for real-valued functions, which is enough to handle conditional probabilities.

Lemma: Let $U,V\subseteq\mathbb R^p$ be linear subspaces. Suppose $f:\mathbb R^p\to\mathbb R$ is bounded and Lebesgue-measurable, and there are measurable functions $g,h$ such that $$ f(x)=g(P_Ux)=h(P_Vx) $$ for Lebesgue-almost every $x$. Then there is a measurable function $r$ such that $f(x)=r(P_{U\cap V}x)$ for Lebesgue-almost every $x$.

Proof. Write $W=U\cap V$. If $u\in U^\perp$, then $P_U(x+u)=P_Ux$. The representation through $U$, and translation invariance of Lebesgue-null sets, imply $f(x+u)=f(x)$ for almost every $x$. The representation through $V$ gives the same invariance for every $v\in V^\perp$. Consequently, for each $d\in U^\perp+V^\perp$, $f(x+d)=f(x)$ for almost every $x$: write $d=u+v$ and apply the two translation invariances in succession. Finite-dimensional linear algebra gives $$ U^\perp+V^\perp=(U\cap V)^\perp=W^\perp. $$ Thus $f$ is invariant, up to null sets for each fixed translation, in every direction perpendicular to $W$.

To conclude that $f$ depends only on $P_Wx$, smooth it with a standard approximate identity. Let $\rho_\epsilon$ be a smooth, compactly supported probability density concentrated near zero, and define $$ f_\epsilon(x)=\int_{\mathbb R^p} f(x-z)\rho_\epsilon(z)\,dz. $$ For each fixed $d\in W^\perp$, the almost-everywhere translation invariance just proved implies $f_\epsilon(x+d)=f_\epsilon(x)$ for every $x$; the integral is unchanged by altering its integrand on a null set. Since $f_\epsilon$ is continuous and invariant under every such translation, it is a function of $P_Wx$.

The approximate-identity property gives $f_\epsilon\to f$ in $L^1$ on each bounded set as $\epsilon\to0$. Choose a sequence $\epsilon_k\to0$ along which convergence also holds almost everywhere, using a diagonal subsequence over an increasing sequence of bounded sets. In orthogonal coordinates $x=(w,z)\in W\oplus W^\perp$, every $f_{\epsilon_k}(w,z)$ is constant in $z$. By Fubini's theorem, for almost every $w$, the sequence converges for almost every $z$; its values are the same for all $z$, so its limit, and hence $f(w,z)$ for almost every $z$, is constant in $z$. This yields a measurable version $r(w)$ with $f(x)=r(P_Wx)$ almost everywhere. $\square$

The proof uses boundedness to ensure local integrability and make the smoothing argument valid. Conditional probabilities satisfy this condition because their values lie between zero and one.

The Intersection Theorem

Theorem: Suppose $X\in\mathbb R^p$ satisfies the full-support density assumption, $Y\in\mathbb R^d$, and $S_1,S_2\subseteq\mathbb R^p$ are sufficient dimension reduction subspaces. Then $S_1\cap S_2$ is also a sufficient dimension reduction subspace.

Proof. Let $\mathcal C$ be the countable collection of lower rectangles $$ (-\infty,q_1]\times\cdots\times(-\infty,q_d],\qquad (q_1,\ldots,q_d)\in\mathbb Q^d, $$ together with $\mathbb R^d$. This collection is closed under finite intersections and generates the Borel sets of $\mathbb R^d$; it determines probability measures on $\mathbb R^d$.

For any $A\in\mathcal C$, sufficiency of $S_1$ and $S_2$ gives measurable versions of the same conditional probability: $$ P(Y\in A\mid X)=g_A(P_{S_1}X)=h_A(P_{S_2}X)\quad\text{almost surely}. $$ Because the density of $X$ is positive almost everywhere, this equality holds for Lebesgue-almost every $x$ when the conditional probability is represented as a function of $x$. The projection-intersection lemma therefore gives a measurable $r_A$ such that $$ P(Y\in A\mid X)=r_A(P_{S_1\cap S_2}X)\quad\text{almost surely}. $$

It remains to verify that these probabilities establish sufficiency for the whole conditional distribution, rather than just for individual sets. Since $\mathcal C$ is countable, there is one probability-one event on which the conditional distribution given $X$ and the conditional distribution given $P_{S_1\cap S_2}X$ agree on every set in $\mathcal C$. Probability measures agreeing on this determining collection agree on all Borel sets. Thus the conditional distribution of $Y$ given $X$ is determined by $P_{S_1\cap S_2}X$, proving that the intersection is sufficient. $\square$

Using countably many rational rectangles is important for a vector-valued response: it supplies a single determining collection of Borel sets on which the conditional probabilities can be compared. A one-dimensional family of half-lines would only suffice for a real-valued response.

Worked Example: Intersecting Two Sufficient Subspaces

Let $X=(X_1,X_2,X_3)^\mathsf T\sim N(0,I_3)$, let $\varepsilon\sim N(0,1)$ be independent of $X$, and set $Y=X_2+\varepsilon$. Consider $$ S_1=\operatorname{span}(e_1,e_2),\qquad S_2=\operatorname{span}(e_2,e_3), $$ where $e_1,e_2,e_3$ are the coordinate vectors.

The intersection is $S_1\cap S_2=\operatorname{span}(e_2)$: a vector in both subspaces must have zero first and third coordinates, so it has the form $(0,c,0)^\mathsf T$.

Given $X=x$, the conditional distribution of $Y$ is $N(x_2,1)$. This depends only on $x_2$, so it is determined by the projection onto $\operatorname{span}(e_2)$. In particular, each of $S_1$ and $S_2$ is sufficient, and their intersection is sufficient as the theorem guarantees. The standard normal density of $X$ is positive everywhere, so the theorem's support condition holds.

The Central Subspace

Definition: When the intersection of all sufficient dimension reduction subspaces is itself sufficient, that intersection is called the central subspace, denoted $\mathcal S_{Y\mid X}$. It is the smallest sufficient subspace under inclusion.

The theorem proves existence under the full-support density assumption. To see why the intersection over all sufficient subspaces is sufficient, first note that a finite intersection remains sufficient by repeatedly applying the theorem. Although there may be infinitely many sufficient subspaces, their total intersection equals a finite intersection from that collection. Indeed, among all finite intersections choose one, $T$, of smallest possible dimension; such a choice exists because dimensions are integers between zero and $p$. For any other sufficient subspace $S$, $T\cap S$ is a finite intersection and is sufficient. Minimality of $\dim(T)$ forces $\dim(T\cap S)=\dim(T)$, so $T\subseteq S$. Thus $T$ is contained in every sufficient subspace, and the intersection of the entire collection is exactly $T$. It is therefore sufficient.

Uniqueness follows directly from the definition: the intersection of a specified collection of subspaces is unique. If another subspace were also the smallest sufficient subspace, each smallest subspace would have to be contained in the other, so they would be equal. This is uniqueness as a subspace, not as a choice of basis or coordinate representation.

Worked Example: Why the Support Condition Matters

Let $Z\sim N(0,1)$, set $X=(Z,Z)^\mathsf T$, and let $Y=Z$. Take $S_1=\operatorname{span}(e_1)$ and $S_2=\operatorname{span}(e_2)$.

Projection onto $S_1$ reveals $X_1=Z$, and projection onto $S_2$ reveals $X_2=Z$. In either case $Y$ is determined by the projection, so both subspaces are sufficient. Their intersection is $\{0\}$, whose projection is constant.

But conditioning on a constant does not make $Y$ independent of $X$. For example, $Y=X_1$ almost surely and $\operatorname{Var}(Y)=1$, so $Y$ is not independent of $X$. The predictor distribution is concentrated on the diagonal $\{(z,z):z\in\mathbb R\}$, not spread through $\mathbb R^2$ with a positive density. The theorem does not apply, and the intersection is not sufficient.

Common Pitfall and Practical Meaning

A tempting but incorrect argument is: “Both projections preserve all information about $Y$, so their intersection must preserve it too.” That conclusion does not follow from conditional independence alone. The diagonal example shows why: on the observed support, either coordinate recovers the same latent value, even though the two coordinate subspaces intersect only at zero. The full-support assumption rules out this kind of support-induced equivalence and allows the projection-intersection lemma to operate in the ambient predictor space.

The central subspace gives a population target for dimension reduction: it identifies the smallest collection of linear predictor directions sufficient for the conditional law of the response. Its existence does not by itself provide an estimator, guarantee that a particular method will recover it, or establish that its directions are identifiable from a chosen matrix representation. Those are separate questions. Here the result is about the existence and uniqueness of the underlying sufficient subspace, subject to the stated regularity condition.

Check Your Understanding

Use the definitions and results above to answer the following questions.

State the conditional-independence definition of a sufficient dimension reduction subspace.
Why does the proof use a countable determining class of Borel sets for a response in $\mathbb R^d$?
In the projection-intersection lemma, which directions leave the function unchanged, and what is their relation to $U\cap V$?
Explain how the intersection theorem leads to a sufficient intersection over an initially infinite collection of subspaces.
In the diagonal example, why are both coordinate subspaces sufficient while their intersection is not?