Conditional and Unconditional Bias Targets

Two Bias Questions After a Sequential Trial

A group sequential design can stop at different looks, and the estimate reported at a given look may be affected by the event that the trial stopped there. This creates two related but distinct questions. Among trials that stop at the observed stage, how far from the parameter is the estimator on average? Across repetitions of the whole design, including every possible stopping stage, what is the estimator’s average bias?

The first question is conditional on the stopping stage; the second is unconditional over the design’s stopping rule. They are not competing definitions of one quantity. Each averages over a different set of repetitions and answers a different practical question. The earlier material on stopping-induced bias and stage selection motivates why conditioning on a stopping event can shift an estimator’s distribution.

Definition: Let $\theta$ be the parameter of interest, $T$ the stopping stage, and $\hat{\theta}_T$ the estimator reported at the stage where stopping occurs. The bias conditional on stopping at stage $k$ is $$ b_k(\theta)=E_\theta(\hat{\theta}_T-\theta\mid T=k), $$ provided $P_\theta(T=k)>0$. The unconditional bias under the design is $$ B(\theta)=E_\theta(\hat{\theta}_T-\theta) =\sum_k P_\theta(T=k)b_k(\theta), $$ where the sum is over stages with positive stopping probability.

The conditional quantity describes the average estimation error in the subset of repetitions that stop at stage $k$. The unconditional quantity describes average error across repetitions of the entire design. Both are functions of $\theta$, and both depend on the stopping rule and on how $\hat{\theta}_T$ is defined. The conditional bias is undefined at a parameter value if stopping at that stage has probability zero; it is not made zero by that fact.

Why the Weighted-Average Identity Holds

The unconditional identity follows by partitioning repetitions according to their stopping stage. The events $T=k$ are mutually exclusive, and, if the design always stops at one of its specified stages, their probabilities sum to one. Applying the law of total expectation to the estimation error gives

\begin{aligned} E_\theta(\hat{\theta}_T-\theta) &=\sum_k E_\theta\{(\hat{\theta}_T-\theta)I(T=k)\}\\ &=\sum_k P_\theta(T=k) E_\theta(\hat{\theta}_T-\theta\mid T=k)\\ &=\sum_k P_\theta(T=k)b_k(\theta). \end{aligned}

Here $I(T=k)$ is one when the trial stops at stage $k$ and zero otherwise. The identity does not require the conditional biases to have the same sign. Positive and negative stage-specific biases can offset one another, so a small unconditional bias does not imply that the estimate is nearly unbiased among trials stopping at every stage.

For this decomposition to describe a particular estimator, define that estimator consistently across repetitions: at each possible stage, specify which data enter $\hat{\theta}_T$. Also specify the stopping rule and the parameter value at which probabilities and expectations are evaluated. If the design has an additional terminal outcome, such as no estimate being reported, it must be represented in the outcome partition and the target must specify how that outcome is handled.

A Two-Look Normal Model

The following model makes the selection effect and the averaging operation explicit. It is a deliberately simple hypothetical design, not a report of an actual trial. Assume two independent stage observations $X_1$ and $X_2$, each with distribution $N(\theta,1)$, where the unit variance is known. At the first look, stop for efficacy if $X_1>c$; otherwise continue. Use $\hat{\theta}_1=X_1$ if the trial stops at look 1, and $\hat{\theta}_2=(X_1+X_2)/2$ if it reaches look 2. Thus $T=1$ when $X_1>c$, and $T=2$ when $X_1\leq c$.

The assumptions are independent normal stage observations with a common mean, a fixed threshold $c$, and the stated stage-specific estimators. In particular, the stopping event depends on $X_1$, while the second stage observation is independent of that event. The examples below use these assumptions; a different information pattern, stopping rule, or estimator would generally give different conditional biases.

Result: Write $a=c-\theta$, and let $\phi$ and $\Phi$ denote the standard normal density and distribution functions. If $P_\theta(T=1)>0$ and $P_\theta(T=2)>0$, then $$ \begin{aligned} P_\theta(T=1)&=1-\Phi(a),& b_1(\theta)&=\frac{\phi(a)}{1-\Phi(a)},\\ P_\theta(T=2)&=\Phi(a),& b_2(\theta)&=-\frac{\phi(a)}{2\Phi(a)},\\ B(\theta)&=\frac{\phi(a)}{2}. \end{aligned} $$

The first-stage conditional bias is positive because stopping at look 1 selects values of $X_1$ above the threshold. The second-stage conditional bias is negative: reaching look 2 selects $X_1\leq c$, shifting the first observation downward relative to its unconditional mean. The independent $X_2$ remains unbiased even after this selection, but it only receives half the weight in the look-2 estimator.

Worked Calculations

Worked Example: Bias Among Trials Stopping at Look 1

Suppose the hypothetical design above has $\theta=0$ units and $c=1$ unit. For trials stopping at the first look, calculate the conditional bias and interpret it.

The assumptions are those of the two-look normal model: $X_1\sim N(0,1)$, the trial stops at look 1 when $X_1>1$, and the reported estimate at that look is $X_1$. The stopping probability is positive: $$ P_0(T=1)=P(X_1>1)=1-\Phi(1)=0.1587. $$ For a standard normal variable truncated above $a$, its conditional mean is $\phi(a)/[1-\Phi(a)]$. Here, $$ b_1(0) =E_0(X_1-0\mid X_1>1) =\frac{\phi(1)}{1-\Phi(1)} =\frac{0.2420}{0.1587} =1.525\text{ units}. $$

Thus, among repetitions of this design that stop at look 1, the mean reported estimate is 1.525 units above the true parameter. This is a stage-conditional statement; it does not say that the mean estimation error across all repetitions is 1.525 units.

Worked Example: Bias Among Trials Reaching Look 2

Keep $\theta=0$ units and $c=1$ unit. Calculate the bias conditional on reaching the second look, using $\hat{\theta}_2=(X_1+X_2)/2$.

Reaching look 2 means $X_1\leq1$, an event with probability $\Phi(1)=0.8413$, so the conditional target is defined. Independence gives $E_0(X_2\mid X_1\leq1)=E_0(X_2)=0$. The lower-truncated normal mean of $X_1$ is $-\phi(1)/\Phi(1)$. Therefore, $$ \begin{aligned} b_2(0) &=E_0\left(\frac{X_1+X_2}{2}-0\ \middle|\ X_1\leq1\right)\\ &=\frac{1}{2}\left\{-\frac{\phi(1)}{\Phi(1)}+0\right\}\\ &=-\frac{0.2420}{2(0.8413)} =-0.1438\text{ units}. \end{aligned} $$

Among repetitions reaching look 2, the estimator is on average 0.1438 units below the true parameter. The negative sign is consistent with the selection event: the first observation must not have crossed the efficacy threshold.

Worked Example: Averaging Across Both Stopping Stages

For the same design and parameter value, calculate the unconditional bias from the two conditional biases. Compare it with the conditional bias at look 1.

The two stage probabilities sum to one: $0.1587+0.8413=1.0000$. Weight each bias by its probability: $$ \begin{aligned} B(0) &=P_0(T=1)b_1(0)+P_0(T=2)b_2(0)\\ &=(0.1587)(1.525)+(0.8413)(-0.1438)\\ &\approx0.2420-0.1210\\ &=0.1210\text{ units}. \end{aligned} $$

The unconditional bias is positive but much smaller than the look-1 conditional bias. It averages the upward shift among early stops together with the downward shift among trials that continue. The value 0.1210 units applies to repetitions of the entire design at $\theta=0$; it is not the expected error for the subset stopping at either particular look.

Worked Example: Small Overall Bias With Material Stage-Specific Bias

Suppose a separate hypothetical three-stage design, evaluated at one fixed parameter value, has the following operating characteristics. Its listed $b_k$ values are conditional means of the same estimator’s error, in units of the parameter.

Stopping stage	Stopping probability	Conditional bias
1	0.20	0.60 units
2	0.50	-0.10 units
3	0.30	-0.20 units

The probabilities sum to one. The unconditional bias is $$ B=(0.20)(0.60)+(0.50)(-0.10)+(0.30)(-0.20) =0.12-0.05-0.06 =0.01\text{ units}. $$

The overall average error is only 0.01 units, but the stage-1 conditional bias is 0.60 units. Reporting only the overall figure would conceal the larger average error among early stops. Conversely, reporting only the stage-1 figure would not describe the design-wide average. Which summary is useful depends on whether the question concerns the observed stage or repeated use of the full design.

Choosing and Reporting the Target

If an analyst has observed a trial stop at stage $k$, $b_k(\theta)$ addresses a repeated-sampling question about trials that would stop at that same stage under the specified parameter value and design. It does not mean the particular observed estimate has an error equal to $b_k(\theta)$, and it does not condition on the exact observed data. The unconditional bias, $B(\theta)$, instead averages over all stopping stages according to their probabilities under the design.

These quantities also differ from the bias of a fixed-sample estimator evaluated at a prespecified sample size. Here the stopping rule changes which repetitions contribute to each conditional average, and the design-wide average accounts for the probabilities of those outcomes. When comparing estimators or designs, state the target explicitly and keep the estimator definition and parameter value fixed. A conditional bias at one stage cannot be compared directly to an unconditional bias as if both summarized the same population of repetitions.

A useful reporting statement identifies the stage, parameter value or evaluation point, estimator, and conditioning event. For example: “At $\theta=0$ units, the mean error of the look-2 estimator conditional on reaching look 2 is $-0.1438$ units.” A design-wide statement should instead name the weighting: “Averaged over repetitions stopping at either look, the bias is $0.1210$ units.”

Diagnostics, Alternatives, and Limitations

When estimating these quantities by simulation, record the stopping stage and estimator for every repetition. For each stage, estimate its stopping probability and compute the mean error only among repetitions stopping there. Estimate the unconditional bias by averaging all repetition-level errors, or equivalently by weighting the stage-specific estimates by their estimated stopping frequencies. The two calculations should agree up to Monte Carlo error. If a stage has few simulated stops, its conditional mean can be unstable even when the unconditional estimate is reasonably precise.

Check the conditioning event. Confirm that $P_\theta(T=k)>0$ and that the subset used to calculate $b_k(\theta)$ really consists of repetitions with $T=k$.
Keep the estimator fixed by definition. Do not average the look-1 estimator for one target and the look-2 estimator for another while describing both as bias of the same rule without specifying that the estimator depends on $T$.
Check the weights. Stage probabilities must be evaluated under the same parameter value and stopping rule as the conditional biases. Their sum should be one when the listed stages exhaust all outcomes.
Investigate cancellation. A small overall bias alongside large conditional biases may reflect opposing stage-specific errors, not negligible selection effects.

The main alternative summaries are not interchangeable: stage-conditional bias answers a selected-stage question, whereas design-unconditional bias answers a repeated-design question. A table reporting both is often more informative than a single average, particularly when stopping probabilities or conditional biases differ substantially by stage. Neither summary alone describes the full distribution of estimation error; bias is only its mean. The calculations also rely on the specified model and stopping mechanism, so a misspecified outcome distribution or an incorrectly implemented stopping rule can invalidate the numerical results.

Common Mistake

Common mistake: “The overall bias is small, so the estimator is nearly unbiased among trials stopping at each look.” This conclusion is not justified. Unconditional bias is a probability-weighted average, and positive and negative conditional biases can cancel. Report or calculate the stage-specific quantities when the question concerns a particular observed stopping stage, and use the weighted overall quantity when the target is performance across repetitions of the full design.

Check Your Understanding

Answer each question by identifying the averaging population as well as the calculation.

A design has stopping probabilities 0.25 and 0.75 at two stages, with conditional biases 0.40 and $-0.10$ units. What is the unconditional bias?
Can conditional bias at stage 2 be defined if $P_\theta(T=2)=0$? Explain.
In the two-look normal example, why does the second-stage observation $X_2$ contribute zero to the look-2 conditional bias?
What does an unconditional bias of 0.01 units imply about a conditional bias of 0.60 units at one stage?

Solutions. (1) $B=(0.25)(0.40)+(0.75)(-0.10)=0.100-0.075=0.025$ units. (2) No. The conditional expectation given $T=2$ is undefined when that event has probability zero under the parameter and design in question. (3) $X_2$ is independent of $X_1$, and reaching look 2 is determined by $X_1$; hence $E_\theta(X_2\mid T=2)=E_\theta(X_2)=\theta$. (4) It does not rule it out: the overall bias may be small because other stages have biases of the opposite sign that offset it after probability weighting.