Conditional Likelihood for a Complete Selection Path

Why the Whole Decision Path Matters

In a multi-arm multi-stage trial, later outcomes are observed only for arms that survive earlier decisions. A likelihood based only on the final retained arms can therefore miss how the data led to that particular set of arms. For example, an arm retained after an interim comparison has passed a data-dependent rule; an arm dropped at that same interim has passed the opposite rule. Those decisions contain information about the treatment effects.

A complete selection path records the retention and dropping decisions at every decision stage, including which arms were eligible to provide later outcomes. This tutorial constructs the likelihood of the observed stagewise estimates conditional on that specified path. It focuses on representing the path and its probability correctly, rather than on choosing a particular post-selection estimator.

Definition: Let $Y$ denote the vector of stagewise treatment-effect estimates that are observed, and let $\theta$ denote the vector of unknown treatment effects. A specified complete selection path corresponds to an event $A$ in the data space: the set of data values that would produce exactly that sequence of retention and dropping decisions under the trial’s rules. The likelihood conditional on that path is the conditional density $$ L_A(\theta;y)=f_\theta(y\mid A) =\frac{f_\theta(y)}{P_\theta(A)},\qquad y\in A, $$ when $y$ contains all observed coordinates and the path event is determined by those coordinates. If some coordinates are unobserved, the numerator is the appropriate joint density integrated over those missing coordinates, subject to the path event.

The numerator describes the observed estimates under the assumed sampling model. The denominator is the probability that this model, with parameter $\theta$, would produce the realized path. It generally depends on $\theta$. Thus, conditioning on the path is not accomplished by merely deleting the likelihood contributions associated with dropped arms.

Model and Construction

For a useful general formulation, collect the stagewise estimates in a vector $Y$. A common working model is $$ Y\sim N\{\mu(\theta),\Sigma\}, $$ where $\mu(\theta)$ gives the expected estimates under treatment effects $\theta$, and $\Sigma$ is their covariance matrix. The covariance may include dependence between arms caused by a shared control group and dependence between stagewise estimates if the data overlap. The normal model is an assumption, not a consequence of selection.

Suppose the trial’s prespecified rules map the observed data to a path. For example, an arm may be retained when its interim estimate exceeds a threshold; a later decision may use a cumulative statistic. Write $A$ for the event that all the observed decisions match the specified path. Then $$ P_\theta(A)=\int_{y\in A} f_\theta(y)\,dy. $$ If the path has several stages, $A$ is the intersection of the restrictions imposed by every decision along the path. The conditional likelihood divides the joint density by this one path probability. When the estimates are independent and each decision is a separate threshold, the probability may factor into simple normal tail probabilities. With correlation or cumulative decision statistics, it generally requires a joint probability calculation.

Specify the sampling model.
State the distribution of each stagewise estimate and the covariance assumptions before calculating any likelihood.

Translate the decisions into restrictions.
Write the inequalities that characterize the realized retention and dropping decisions, including decisions at every stage.

Calculate the path probability.
Evaluate or express $P_\theta(A)$ under the same model. Use a joint probability when the decision statistics are dependent.

Form the conditional likelihood.
Divide the density of the observed data by $P_\theta(A)$, integrating over unobserved outcomes when necessary.

Worked Path Likelihoods

Worked Example: Three Arms at a Screening Stage

Hypothetical data. Suppose three independent stage-one estimates $Y_1,Y_2,Y_3$ each have variance 1 and $Y_i\sim N(\theta_i,1)$. The rule retains an arm if its estimate exceeds 1. The observed estimates are $1.8,0.4,2.1$, so the path retains arms 1 and 3 and drops arm 2. Consider the candidate effects $(\theta_1,\theta_2,\theta_3)=(1.2,0.2,1.5)$.

The path event is $A=\{Y_1>1,\;Y_2\leq1,\;Y_3>1\}$. The observations satisfy all three inequalities. Independence gives $$ P_\theta(A) =[1-\Phi(1-1.2)]\Phi(1-0.2)[1-\Phi(1-1.5)] =\Phi(0.2)\Phi(0.8)\Phi(0.5) \approx (0.5793)(0.7881)(0.6915) \approx 0.3160. $$

The joint density at the observed estimates is $$ f_\theta(y) =\phi(1.8-1.2)\phi(0.4-0.2)\phi(2.1-1.5) =\phi(0.6)\phi(0.2)\phi(0.6) \approx (0.3332)(0.3910)(0.3332) \approx 0.04341. $$ Therefore the conditional density at these observations, given this path, is approximately $0.04341/0.3160=0.1374$. This is a density, not a probability assigned to the exact continuous data values.

The calculation relies on independent estimates with known unit variances. If the estimates share a control group, independence need not hold: the event probability must then be computed using their joint distribution rather than multiplying three marginal tail probabilities.

Worked Example: A Complete Two-Stage Path With a Dropped Arm

Hypothetical data. Two arms, A and B, have independent stagewise estimates with variance 1; all estimates are mutually independent under the model. At stage one, an arm is retained if its estimate exceeds 1. Only retained arms receive a stage-two estimate. For retained A, the stage-two decision retains it if its stage-two estimate exceeds 0.5. The observed data are $Y_{A1}=1.4$, $Y_{A2}=0.8$, and $Y_{B1}=0.4$. Thus the complete path is: retain A at stage one, drop B at stage one, and retain A at stage two. Use candidate effects $\theta_A=0.7$ and $\theta_B=0.2$.

The path event for the observed coordinates is $A=\{Y_{A1}>1,\;Y_{B1}\leq1,\;Y_{A2}>0.5\}$. Under the stated independence assumptions, $$ P_\theta(A) =[1-\Phi(1-0.7)]\Phi(1-0.2)[1-\Phi(0.5-0.7)] =\{1-\Phi(0.3)\}\Phi(0.8)\Phi(0.2) \approx (0.3821)(0.7881)(0.5793) \approx 0.1744. $$

The observed-data density is $$ f_\theta(y) =\phi(1.4-0.7)\phi(0.4-0.2)\phi(0.8-0.7) =\phi(0.7)\phi(0.2)\phi(0.1) \approx (0.3123)(0.3910)(0.3970) \approx 0.04847. $$ Consequently $L_A(\theta;y)\approx0.04847/0.1744=0.2779$. There is no $Y_{B2}$ in this likelihood: B was dropped before that estimate was collected. The path probability also does not include a stage-two decision for B. If the decision rules instead used cumulative estimates or correlated stagewise estimates, the path event and its probability would have to reflect those details.

Why the Normalizing Probability Changes Estimation

Because $P_\theta(A)$ depends on the unknown effects, it cannot generally be treated as a constant when maximizing the conditional likelihood. A one-arm calculation shows the effect directly. Suppose $Y_1,Y_2$ are independent $N(\theta,1)$ estimates, but $Y_2$ is observed only after the stage-one rule $Y_1>c$. Conditional on that selection event, the density at observed values $y_1>c,y_2$ is $$ f_\theta(y_1,y_2\mid Y_1>c) =\frac{\phi(y_1-\theta)\phi(y_2-\theta)} {1-\Phi(c-\theta)}. $$ The second-stage density remains in the numerator, while the selection adjustment is the probability of passing the first-stage rule.

Taking logarithms and differentiating with respect to $\theta$ gives the conditional score. Let $\phi$ and $\Phi$ be the standard normal density and distribution functions. Since the derivative of $\log\phi(y-\theta)$ is $y-\theta$, and the derivative of $\log\{1-\Phi(c-\theta)\}$ is $\phi(c-\theta)/\{1-\Phi(c-\theta)\}$, the score is $$ \frac{\partial}{\partial\theta}\log f_\theta(y_1,y_2\mid Y_1>c) =(y_1-\theta)+(y_2-\theta) -\frac{\phi(c-\theta)}{1-\Phi(c-\theta)}. $$ The final term is the derivative contribution from the parameter-dependent normalization. Omitting it gives the ordinary, unconditioned score, not the score for the likelihood conditional on selection.

Worked Example: The Conditional Score After Screening

Hypothetical data. In the preceding one-arm model, let $c=1$, $y_1=1.4$, $y_2=0.8$, and evaluate the conditional likelihood at $\theta=0.5$. The observations are consistent with the rule because $1.4>1$; the independent second-stage estimate is available because the arm passed the screen.

The conditional density is $$ \frac{\phi(1.4-0.5)\phi(0.8-0.5)} {1-\Phi(1-0.5)} =\frac{\phi(0.9)\phi(0.3)}{1-\Phi(0.5)} \approx\frac{(0.2661)(0.3814)}{0.3085} \approx 0.3289. $$

The score at this candidate value is $$ (1.4-0.5)+(0.8-0.5) -\frac{\phi(0.5)}{1-\Phi(0.5)} =0.9+0.3-\frac{0.3521}{0.3085} \approx 1.2-1.141 \approx 0.059. $$ A positive score means the conditional log-likelihood is increasing locally as $\theta$ increases from 0.5. It does not by itself identify the maximum; that requires solving the score equation or directly maximizing the conditional likelihood over the parameter space.

Assumptions, Diagnostics, and Alternatives

A path-conditional likelihood is only as credible as its model and its description of the decision process. Before using it, verify that the estimates’ sampling distributions and covariance structure are appropriate; that the thresholds, ranking rules, and tie-breaking rules are specified accurately; and that the observed data include exactly the coordinates available along the realized path. If a decision used a statistic not included in $Y$, include that statistic or account for its distribution in the path probability.

Common warning signs include a path probability of zero under a proposed parameter value, a calculated probability that does not match simulation under the assumed model, and large changes in the likelihood when a plausible covariance is substituted. A zero path probability makes the conditional density undefined at that parameter value. For more complex paths, numerical integration or simulation may be needed to evaluate the denominator; numerical error should be checked, especially when the path is rare.

Shared controls deserve particular care. Even if arm-specific treatment outcomes are independent, treatment-effect estimates using the same control data are generally correlated. Treating correlated estimates as independent can miscalculate $P_\theta(A)$, and therefore distort the conditional likelihood. Likewise, if stagewise estimates use overlapping observations, independence across stages must not be assumed without justification.

The main alternative is the ordinary likelihood for the sampling model, which uses the observed data density without dividing by the probability of the path. It can be appropriate when the analysis conditions on a design whose selection mechanism is ignorable for the parameter of interest, or when the inferential target and model justify a marginal analysis. But it is not the likelihood conditional on the realized path. Conversely, conditioning on a very detailed path can discard information and produce a difficult or weakly informative likelihood. The choice is an inferential choice, not a universal correction: report whether the analysis is conditional on the path or marginal over paths.

This construction also has limits. It does not repair misspecified outcome distributions, unrecorded adaptations, or incorrect decision thresholds. If the path event depends on unobserved outcomes, those outcomes must be integrated out rather than assigned their observed-data density. Nor does the conditional likelihood alone guarantee unbiased estimation or calibrated intervals. Its role here is precise: it gives the likelihood under the stated model conditional on the stated complete path.

Common Mistakes

Leaving out the denominator. Using only $f_\theta(y)$ treats the observed path as if it carried no information. For a path-conditional likelihood, divide by $P_\theta(A)$.
Using the wrong path event. Conditioning on “A was selected” is not equivalent to conditioning on the complete sequence if the sequence also records other arms dropped or later decisions made.
Multiplying marginal probabilities under dependence. Products of tail probabilities require the relevant independence. Shared controls or overlapping stages may require a joint probability instead.
Adding outcomes from dropped arms. An estimate not collected after dropping is not an observed-data likelihood contribution. If needed for a path probability, integrate over it under the model.
Calling a conditional density a probability. For continuous estimates, the likelihood density at the observed values is not the probability of those exact values.

Key takeaway: A complete path is a region defined by all the trial’s retention and dropping rules. Under a specified sampling model, the likelihood conditional on that path is the observed-data density divided by the model probability of the entire path.

Check Your Understanding

Answer each question, then compare with the brief solutions below.

For independent $Y_1,Y_2\sim N(\theta,1)$, an arm is retained when $Y_1>0$. What is the conditional density of $Y_1=y_1,Y_2=y_2$ given retention, for $y_1>0$?
Two independent arms have stage-one estimates $N(\theta_i,1)$, and each is retained if its estimate exceeds 1. Write the event for the path that retains arm 1 and drops arm 2.
Why is it generally incorrect to multiply marginal path probabilities for estimates that share a control group?
If an arm is dropped at stage one and its stage-two outcome is never collected, should that unobserved outcome appear as a density contribution in the observed-data likelihood?

Solutions. (1) The density is $\phi(y_1-\theta)\phi(y_2-\theta)/\{1-\Phi(-\theta)\}$, because $P_\theta(Y_1>0)=1-\Phi(-\theta)$. (2) The path event is $\{Y_1>1,\;Y_2\leq1\}$. (3) Sharing a control can induce correlation, so the joint probability of the two decisions need not equal the product of their marginal probabilities. (4) No. It is not observed; integrate over it only if it is needed to calculate a path probability or another marginal likelihood.