Conditioning on Adaptation Events

Why Condition on the Adaptation Event?

In a two-stage trial, an interim result may determine whether the trial continues, stops, or changes its design. The final estimator therefore has a distribution that depends on the rule and, when we condition on a particular outcome of that rule, on the event that occurred. For example, the estimator among trials that continue after an interim estimate falls below a futility boundary need not have the same mean or variance as the estimator across all trials.

This conditional distribution is useful for understanding what the estimator does in a specified branch of the design. It is not, by itself, the unconditional sampling distribution of the estimator over repeated trials. Nor does conditioning turn a data-dependent rule into a fixed design: the restriction defining the event must remain part of the calculation.

A Two-Stage Normal Model

Suppose the two stages produce estimates \(\hat\theta_1\) and \(\hat\theta_2\) of a common treatment effect \(\theta\). For the calculations in this tutorial, assume \(\hat\theta_1\sim N(\theta,\tau_1^2)\) and \(\hat\theta_2\sim N(\theta,\tau_2^2)\), with independent stage estimates and known standard deviations \(\tau_1\) and \(\tau_2\). Independence is appropriate, for example, when stage-specific data are disjoint and the estimates are based on independent observations. The adaptation event \(A\) is determined by \(\hat\theta_1\). In a branch where both stages contribute, consider a fixed weighted estimator \(T=w\hat\theta_1+(1-w)\hat\theta_2\), where the weight \(w\) does not vary with the observed interim value within that branch.

Because \(\hat\theta_2\) is independent of the interim data, conditioning on \(A\) does not change its distribution. The interim estimate, however, is restricted to the values that define \(A\). Thus

The variance formula follows because the conditioned interim estimate and the independent second-stage estimate remain independent. In general, the formula gives the conditional variance, not a claim that \(T\mid A\) is normal. A truncated normal variable plus an independent normal variable is generally not normal.

Truncated-Normal Calculations

Write \(\phi\) and \(\Phi\) for the standard normal density and cumulative distribution function. If \(Z\sim N(\mu,\sigma^2)\), standardizing \(Z\) turns a boundary on \(Z\) into a boundary on a standard normal variable. For an upper-bound event \(Z\leq c\), let \(b=(c-\mu)/\sigma\) and \(\lambda=\phi(b)/\Phi(b)\). Direct integration of the normal density gives

For a lower-bound event \(Z>c\), use \(b=(c-\mu)/\sigma\) and \(\lambda=\phi(b)/[1-\Phi(b)]\). The corresponding moments are

These are moments of a distribution restricted to the specified region, not moments obtained by treating the boundary as a new fixed mean. For an interval event \(l\leq Z\leq u\), the conditional density is the original density divided by \(P(l\leq Z\leq u)\) within the interval and zero outside it. The moments can be obtained by integrating that density. In all cases, conditioning is meaningful only if the event has positive probability.

Conditioning on Early Stopping

When a trial stops at an interim look, a common estimator for that branch is the interim estimate itself. Conditioning on the stopping event then means using a truncated distribution for that estimate. This calculation describes the estimator among trials stopping by that rule; it does not describe the estimator across the full design, which also includes trials that continue.

Conditioning on an Interval

Not every adaptation event is a one-sided threshold. A rule may, for example, continue with a specified design only when the interim estimate lies within a range. In that case, the interim distribution is truncated on both sides. The second-stage estimate remains unchanged under the independent-stage assumption, but the interim contribution is now centered on the mean within that interval.

General Construction and Its Limits

For a weighted estimator \(T=aX+bY\), where \(Y\) is independent of the interim estimate \(X\), a conditional distribution can be constructed directly from the conditioned density of \(X\). If \(b>0\), its cumulative distribution function is

where \(f_{X\mid A}\) is the density of \(X\) given \(A\), and \(F_Y\) is the cumulative distribution function of \(Y\). The integral ranges over the values allowed by \(A\). This expression is a mixture over possible interim values: for each \(x\), it evaluates the probability that the second-stage contribution puts \(T\) below \(t\), then averages those probabilities using the conditional distribution of \(X\).

If the adaptation changes the second-stage sample size, the second-stage variance can differ across branches. If weights themselves depend on the interim value, the fixed-weight mean and variance formulas above no longer apply as written. One must retain that dependence inside the conditional distribution or calculate the conditional moments from the full rule. Likewise, if stage estimates are correlated, then the conditional distribution of the second-stage estimate may change after conditioning on the interim data.

The assumptions are therefore part of the result, not bookkeeping details. Before applying a conditional formula, specify the estimator on the branch, the event defining that branch, the stagewise sampling distributions, whether the stages are independent, and whether the weights and variances are fixed within the branch. If an event has extremely small probability, its conditional moments may also be numerically sensitive; the event probability and the truncation calculation should be checked rather than inferred from rounded tail probabilities.

Diagnostics, Alternatives, and a Common Pitfall

A useful diagnostic is to compare conditional and unconditional summaries. If an event is defined by large interim estimates, a conditional mean that is systematically shifted upward may be a consequence of the selection rule, not a coding error. Conversely, a conditional mean that ignores the event restriction, or a calculation that changes the distribution of an independent second-stage estimate without a reason, signals a likely mistake. Check that the event has positive probability, that its boundaries are standardized correctly, and that conditional variances are nonnegative.

There are two related but different summaries. The conditional distribution describes the estimator within a specified branch, such as continuing or stopping. The unconditional distribution averages over branches, weighting each branch by its probability. For an event \(A\) and its complement \(A^c\), for example,

This identity does not mean the conditional and unconditional means are interchangeable. An estimate can be conditionally shifted in one branch while the full-design mean reflects a different balance across all branches. Conditioning on a different event also answers a different question: conditioning only on continuation is not the same as conditioning on a particular interim value or on a more detailed adaptation history.

A common mistake is to say, “The second-stage estimate is independent, so the whole final estimator has its original normal distribution after conditioning.” Independence preserves the second-stage distribution; it does not undo the truncation of the first-stage contribution. For instance, in the continuation example, \(Y\) remains \(N(0,1)\), but \(X\mid X\leq1\) does not remain \(N(0,1)\), so their weighted sum is not generally normal. The correction is to condition the interim component first and then combine its conditional distribution with the correctly specified later-stage component.

Conditional distributions are valuable for describing an adaptation branch and for understanding how a decision rule reshapes estimator behavior. They do not alone supply an interval with a desired coverage probability, nor do they replace analysis of the full design when the target is unconditional performance. Those questions require their own definitions and methods.