Selective Inference Conditional On Cluster Selection

Why Condition on the Observed Partition?

A marker test usually compares expression between groups of cells. If those groups were formed using the same expression data, however, the group labels are not fixed independently of the values being tested. A clustering procedure may produce a partition partly because the observed data separate cells along some expression direction. A conventional test that treats that partition as fixed can therefore use the wrong null distribution.

Selective inference addresses this issue by asking a conditional question: under the null hypothesis, how unusual is the observed gene statistic among datasets for which the clustering procedure would have produced this same partition? The conditioning changes the reference distribution; it does not change the observed gene values or make the partition independent of the data.

Definition: Let \(Y\) denote the expression data, and let a specified clustering procedure \(A\) return a partition \(S=A(Y)\). For an observed partition \(s\), a selective test evaluates a gene statistic \(T_g(Y;s)\) using its null distribution conditional on \(A(Y)=s\). The event \(A(Y)=s\) is the selection event. The test is valid only if that event and the conditional null distribution are defined to match the analysis actually performed.

For example, let \(Y_{ig}\) be the expression measurement for cell \(i\) and gene \(g\). Once a partition \(s\) is observed, a contrast \(T_g(Y;s)\) might compare the mean expression of gene \(g\) in one cluster with its mean in another. A selective null can state that the underlying mean contrast for this gene and this specified partition is zero. This is a post-selection hypothesis about the contrast being tested, not a claim that the clusters were generated independently of gene \(g\).

The conditional question is different from the usual fixed-group question. The ordinary null distribution describes \(T_g\) under a model for the data without restricting which partition the algorithm returns. The selective null distribution restricts that model to outcomes for which \(A(Y)=s\). If the selection event tends to occur when the contrast is large, this restriction can make large observed statistics less surprising than they would be under the unconditioned reference distribution.

Assumptions and the Conditional Test

A conditional test requires a specified probability model for the expression data under the null, as well as a sufficiently precise account of the selection procedure. A useful general formulation assumes that, under a null parameter value, the data have a known distribution, such as a multivariate Gaussian model. The method is not inherently limited to Gaussian data, but the conditional distribution must be calculable or simulatable under the model used.

The tested contrast is defined. After observing \(s\), the gene statistic and the null contrast must be stated in terms of that partition.
The selection event is represented correctly. The event must describe the clustering procedure and any other data-dependent decisions that affect which test is reported.
The null model is appropriate. Its distributional assumptions, including dependence or variance structure relevant to the statistic, must be credible for the data being analyzed.
The conditioning event has positive probability under the null. Otherwise the conditional distribution is not defined at that null parameter value.
Nuisance parameters are handled validly. Exact calibration requires a pivotal conditional distribution or known nuisance parameters. Plugging in estimates does not automatically preserve exact type I error control.

For a two-sided test, write \(T_g^*\) for a statistic drawn from the null distribution conditional on \(A(Y)=s\). A conditional p-value based on absolute magnitude is

p_{\mathrm{sel}} = P\!\left( |T_g^*|\geq |T_g^{\mathrm{obs}}| \mid A(Y)=s,\ H_0 \right).

For a directional alternative, use the corresponding conditional upper- or lower-tail probability. When the null distribution is discrete, counting outcomes at least as extreme as the observed one gives a conservative p-value, because ties are included. If the null has unknown nuisance parameters, one possible conservative construction takes the supremum of this tail probability over the nuisance values allowed by \(H_0\); a less conservative method needs separate justification.

Why Conditioning Can Restore Calibration

The justification is conditional calibration. Suppose the conditional null distribution of a continuous statistic is correctly specified for the observed selection event. Its conditional cumulative distribution function is

F_s(t)=P(T_g^*\leq t\mid A(Y)=s,H_0).

Under the null and conditional on \(A(Y)=s\), the probability integral transform gives \(F_s(T_g^{\mathrm{obs}})\sim \mathrm{Uniform}(0,1)\). Thus a correctly calculated tail probability has the advertised conditional error rate: for a level \(\alpha\), the probability of rejecting \(H_0\), given that the procedure produced \(s\), is at most \(\alpha\). For discrete distributions this statement is an inequality rather than exact equality because conservative tail probabilities can have jumps.

This argument depends on the conditional distribution, not simply on using a familiar test statistic. A normal statistic before selection need not remain normally distributed after conditioning on a clustering result. A derivation of a particular conditional law can be technically demanding; in practice, one needs either a justified analytic calculation or valid conditional simulation.

Worked Example: Conditioning on a Sign-Based Selection

Consider a deliberately simple hypothetical model. Under \(H_0\), \(Z\sim N(0,1)\). A selection rule reports a result only when \(Z>0\), and the observed value is \(Z=2.10\). Find the two-sided p-value conditional on selection. This toy rule illustrates the conditioning principle; it is not a model of a full clustering algorithm.

The selection event is \(Z>0\). Among selected outcomes, the event \(|Z|\geq 2.10\) is equivalent to \(Z\geq2.10\), since negative values were not selected. Therefore,

p_{\mathrm{sel}} = P(Z\geq2.10\mid Z>0) = \frac{P(Z\geq2.10)}{P(Z>0)} = \frac{1-\Phi(2.10)}{0.5} = 2\{1-\Phi(2.10)\}.

Using \(\Phi(2.10)\approx0.9821356\), the conditional p-value is \(2(1-0.9821356)=0.0357288\), or approximately \(0.0357\). This is the exact conditional tail calculation rounded to four significant figures. If one first rounds the intermediate value to \(0.9821\), then \(2(1-0.9821)=0.0358\); that is only the approximation obtained from the rounded intermediate value. The unconditioned two-sided p-value would be \(2\{1-\Phi(2.10)\}\), which happens to be numerically the same here because the selected sign determines which half of the two-sided tail remains and the denominator is \(0.5\). This equality is specific to this rule and does not imply that selection can generally be ignored.

The assumptions are explicit: the null distribution is standard normal, the selection event is exactly \(Z>0\), and the observed value meets that rule. In an actual analysis, the corresponding event would need to represent the clustering procedure, rather than an oversimplified sign rule.

Worked Example: An Exact Conditional Distribution for a Partition

Suppose a hypothetical null model has eight equally likely outcomes, numbered 1 through 8. A clustering algorithm returns partition \(s\) for outcomes 1 through 4 and another partition otherwise. For partition \(s\), a prespecified contrast statistic takes values \(-2,-1,1,2\) on outcomes 1 through 4, respectively, and the observed outcome is 4, so \(T_g^{\mathrm{obs}}=2\). Find the two-sided conditional p-value using absolute magnitude.

Conditional on \(A(Y)=s\), the four outcomes remain equally likely, because they had equal probability before conditioning. The statistic's conditional null values are therefore \(-2,-1,1,2\), each with probability \(1/4\). Two of the four values have absolute magnitude at least 2:

p_{\mathrm{sel}} = P(|T_g^*|\geq2\mid A(Y)=s,H_0) = \frac{2}{4} = 0.50.

The unconditioned fraction of all eight outcomes satisfying \(|T_g|\geq2\) cannot be calculated from the selected-partition values alone: the statistic and its null distribution outside \(s\) have not been specified. That is precisely why one must define and use the distribution conditional on the actual selection event, rather than infer it from a test statistic's label or from a generic reference distribution. The example assumes a correct null model, a known selection rule, and equal probabilities as stated.

Conditional Simulation and Practical Choices

For complex clustering procedures, enumerating all data sets that produce \(s\) may be infeasible. One strategy is to generate data under the null, rerun the specified procedure, retain replicates that produce \(s\), and compare their test statistics with the observed value. This requires that the simulated data follow the null model and that the complete selection rule is rerun in each replicate.

Worked Example: Estimating a Conditional Tail Probability

In a hypothetical analysis, the observed absolute marker statistic is 2.4. A null simulation produces 1,000 accepted replicates that satisfy the observed partition-selection event. Of these, 40 have an absolute statistic at least 2.4. Using the usual add-one Monte Carlo estimate, calculate the conditional p-value.

\widehat p_{\mathrm{sel}} = \frac{b+1}{B+1} = \frac{40+1}{1000+1} = \frac{41}{1001} \approx0.04096.

The estimate is about \(0.0410\); at a nominal \(0.05\) level it is below \(0.05\). It is an estimate based on finite simulation, not an exact p-value computed from the full conditional distribution. The add-one calculation presumes that the observed statistic and the simulated statistics are comparable draws under the conditional null procedure, with ties counted as extreme. If accepted replicates are generated from an inaccurate null model, or the simulation does not condition correctly on the observed selection event, the arithmetic cannot repair that problem.

The principal alternatives address data reuse in different ways. Sample splitting or validation in an independent dataset tests markers using data not used to form the original clusters, avoiding the need to calculate a difficult conditional law but often reducing precision and requiring a suitable held-out sample. A conventional test on the same cells is simpler, but it does not account for selection unless its assumptions justify treating the groups as fixed. Conditional selective inference retains the data and explicitly accounts for a stated selection event, at the cost of stronger modeling and computational demands.

Diagnostics, Failure Modes, and Limits

Before interpreting a selective p-value, verify that the conditioning event corresponds to every choice that influenced the reported test. If the number of clusters, preprocessing, tuning parameters, partition, or gene-screening rule was selected adaptively, conditioning only on the final partition may leave relevant selection unaccounted for. Likewise, if the analyst reports the most significant gene after examining many genes, the selection event includes that choice; conditioning only on the partition does not correct that additional selection.

For a simulation-based calculation, inspect how many null replicates satisfy the selection event. Very few accepted replicates make the estimated tail probability unstable and may signal that direct rejection sampling is inefficient. Check that the simulated pipeline reproduces the observed clustering rule, including handling of label permutations or algorithmic randomness. If the clustering algorithm is random, either condition on its random seed or include its randomness in the joint selection mechanism, and be explicit about which target is being used.

A conditional test can also be valid under a specified model that is a poor description of real expression data. Model misspecification, unaccounted dependence, inaccurate variance assumptions, and nuisance-parameter estimation can all undermine calibration. Conditioning does not guarantee high power: a restrictive selection event may leave little variation in the data, and the resulting conditional reference distribution may be wide or difficult to estimate.

Finally, the inferential claim is limited to the tested contrast under the stated model and selection event. A small conditional p-value is evidence against that conditional null; it is not proof that a gene causes the cluster structure or that the partition represents a biological cell type. Cluster stability and marker-test validity are distinct questions, and conditioning on a partition does not itself establish biological interpretation.

Common Mistake

Wrong reasoning: “The clustering result is fixed now, so I can use the ordinary two-sample test as if the groups had been specified in advance.” The partition may be fixed in the analyst’s report, but it was produced from the same data, and that production can change the null distribution of the reported statistic.

Correction: State the null model, the contrast, and the selection event. Then calculate or simulate the statistic’s distribution under the null conditional on that event. If the event cannot be represented or the conditional law cannot be justified, use a design that separates discovery from testing or describe the ordinary result as descriptive rather than as a calibrated post-selection test.

Key takeaway: Selective inference conditions a gene test on the event that the clustering procedure returned the observed partition. Its error-rate guarantee is conditional on that event and depends on a correct null model, a correctly specified selection event, and valid treatment of nuisance parameters and any additional data-driven choices.

Check Your Understanding

Answer each question, then compare with the brief solutions.

What is the selection event when the analysis conditions on an observed partition \(s\)?
Why can an ordinary null distribution fail to calibrate a marker test after clustering on the same data?
In the sign-selection example, what is the conditional two-sided p-value for \(Z=2.10\), rounded to four significant figures?
A researcher conditions on the final partition but then reports the gene with the largest observed statistic. What additional issue remains?
What does a conditional p-value of \(0.03\) mean at significance level \(0.05\), assuming the conditional procedure is valid?

Solutions. (1) It is the event \(A(Y)=s\), meaning the specified clustering procedure returns the observed partition. (2) Selection can make extreme statistics more likely among datasets producing that partition, changing the reference distribution. (3) It is \(2\{1-\Phi(2.10)\}\approx0.0357\). (4) Choosing the largest statistic is another data-dependent selection and must be accounted for if inferential calibration is claimed. (5) Under the stated null model and conditional on the selection event, the test rejects at level \(0.05\); the p-value is not the probability that the null is true.