Unequal Information Increments in Estimation

Why Unequal Information Increments Matter

In a group sequential design, an estimate available at an early analysis is generally less precise than one available later. When the information increments are unequal, the correlation between the analysis statistics is also not the correlation that would arise from equally spaced information. That correlation matters: stopping selects particular paths through the sequence of statistics, and those paths determine the bias of the reported estimate.

The goal here is to calculate the contribution to bias from each possible stopping stage. A contribution is not the bias conditional on reaching that stage; it is the contribution weighted by the probability of stopping there. Keeping those quantities distinct is essential when combining stages to obtain the bias of the overall stopped estimator.

Definition: Let $I_k$ be the cumulative information at analysis $k$, with $0<I_1<\cdots<I_K$, and let $\hat\theta_k$ estimate a scalar parameter $\theta$. Write the centered canonical statistic as $Z_k=\sqrt{I_k}(\hat\theta_k-\theta)$. If $T$ is the stopping stage, the stage-$k$ bias contribution is $$ B_k(\theta)=E_\theta\left[(\hat\theta_k-\theta)\,1\{T=k\}\right] =\frac{1}{\sqrt{I_k}}E_\theta\left[Z_k\,1\{T=k\}\right]. $$ The bias of the stopped estimator $\hat\theta_T$ is $B(\theta)=\sum_{k=1}^K B_k(\theta)$. When $P_\theta(T=k)>0$, its conditional bias at stage $k$ is $B_k(\theta)/P_\theta(T=k)$.

Canonical Joint Normal Model With Unequal Increments

Assume the canonical joint normal model: under the true parameter, each centered statistic $Z_k$ has a standard normal distribution, and for $j\leq k$,

\operatorname{Cov}_\theta(Z_j,Z_k)=\sqrt{\frac{I_j}{I_k}}.

This model follows from independent information increments. In particular, for two analyses, let $I_1<I_2$, and define $\rho=\sqrt{I_1/I_2}$. Then

Z_2=\rho Z_1+\sqrt{1-\rho^2}\,\varepsilon, \qquad \varepsilon\sim N(0,1), \qquad \varepsilon\ \text{independent of }Z_1.

Thus $Z_2\mid Z_1=z$ is normal with mean $\rho z$ and variance $1-\rho^2$. If the information increment from analysis 1 to analysis 2 is large relative to $I_1$, then $\rho$ is smaller; if it is small, then $\rho$ is closer to one. The increments need not be equal for the canonical model to apply.

The assumptions are that the information values are correctly specified, the estimate has the stated canonical normal joint distribution, and the stopping rule is represented correctly as events in the analysis statistics. In a large-sample application, the normal model may be an approximation, so its adequacy should be checked. For example, if information is estimated rather than fixed, or the endpoint is discrete with a poor normal approximation, the calculated contributions may not accurately describe the actual bias.

In practice, a trial’s boundaries are often defined for statistics centered at a null value rather than at the true $\theta$. The bias calculation must use the stopping events under the parameter value being evaluated: translate the null-centered boundaries to the corresponding events for the centered process above. The numerical examples below use $\theta=0$, so the stated boundaries apply directly to $Z_k$.

Two-Analysis Formulas

Suppose the design stops for efficacy at analysis 1 when $Z_1\geq c$, and otherwise proceeds to the final analysis, where it stops without any additional efficacy or futility selection. The stopping events are $A_1=\{Z_1\geq c\}$ and $A_2=\{Z_1<c\}$. Using $\hat\theta_k-\theta=Z_k/\sqrt{I_k}$, the early contribution is

B_1(\theta) =\frac{1}{\sqrt{I_1}}\int_c^\infty z\phi(z)\,dz =\frac{\phi(c)}{\sqrt{I_1}},

where $\phi$ is the standard normal density. At the final analysis, use the conditional representation of $Z_2$:

\begin{aligned} B_2(\theta) &=\frac{1}{\sqrt{I_2}}E_\theta[Z_2\,1\{Z_1<c\}]\\ &=\frac{\rho}{\sqrt{I_2}}E_\theta[Z_1\,1\{Z_1<c\}]\\ &=-\frac{\rho\phi(c)}{\sqrt{I_2}}. \end{aligned}

For the second equality, the independent residual has mean zero, so its expectation multiplied by $1\{Z_1<c\}$ is zero. For the last equality, $\int_{-\infty}^c z\phi(z)\,dz=-\phi(c)$. The early contribution is positive because early stopping selects large positive statistics; the final contribution is negative because the final group excludes those paths.

More generally, if the final-stage event also depends on $Z_2$, then its contribution is still $E_\theta[Z_2\,1\{T=2\}]/\sqrt{I_2}$, but the simple truncated-normal expression above no longer applies. One can condition on $Z_1$ and integrate the conditional normal distribution over the actual final-stage stopping region. For multiple looks, the same principle applies: define each stopping event from the full path and calculate its expectation under the canonical joint distribution.

Worked Example: Efficacy Stopping With Unequal Increments

Consider a hypothetical two-analysis design with $I_1=25$, $I_2=100$, and an early efficacy boundary $c=2$. The increments are 25 and 75, so they are unequal. At $\theta=0$, suppose an illustrative interim result is $Z_1=2.15$. The stage-1 estimate is $$ \hat\theta_1=0+\frac{2.15}{\sqrt{25}}=0.4300 $$ parameter units, and the design stops at the interim analysis.

To calculate the design’s bias contributions, rather than the estimate from that one realized path, first note that $\rho=\sqrt{25/100}=0.5$. With $\phi(2)=0.05399097$, $$ B_1=\frac{\phi(2)}{5} =\frac{0.05399097}{5} =0.01079819 $$ parameter units. The contribution from paths continuing to the final analysis is $$ B_2=-\frac{0.5\phi(2)}{10} =-\frac{0.5(0.05399097)}{10} =-0.002699548 $$ parameter units. Therefore the overall bias is $$ B_1+B_2=0.008098645\ldots=0.008099 $$ parameter units to four significant figures.

The stage probabilities are $P(T=1)=1-\Phi(2)=0.0227501$ and $P(T=2)=\Phi(2)=0.9772499$. Hence the conditional biases are $B_1/P(T=1)=0.4746$ and $B_2/P(T=2)=-0.002762$ parameter units. The early conditional bias is much larger than its contribution because early stopping is uncommon.

Worked Example: Futility Stopping With Unequal Increments

In a different hypothetical design, let $I_1=16$, $I_2=100$, and stop for futility when $Z_1\leq -1.5$; otherwise continue to the final analysis. The increments are 16 and 84, and $\rho=\sqrt{16/100}=0.4$. At $\theta=0$, an illustrative interim value $Z_1=-1.8$ gives $$ \hat\theta_1=0+\frac{-1.8}{\sqrt{16}}=-0.4500 $$ parameter units, so the path stops for futility.

Because $\int_{-\infty}^a z\phi(z)\,dz=-\phi(a)$, the stage-1 contribution is $$ B_1=-\frac{\phi(1.5)}{4} =-\frac{0.1295176}{4} =-0.03237940 $$ parameter units. On paths continuing to the final analysis, $$ B_2=\frac{\rho}{\sqrt{I_2}}\phi(1.5) =\frac{0.4(0.1295176)}{10} =0.005180704 $$ parameter units. The total bias is $-0.02719870$ parameter units. Here early stopping selects negative interim estimates, while continued paths have a positive mean final statistic because paths with very negative interim statistics have been removed.

The stopping probability is $\Phi(-1.5)=0.0668072$, and the continuation probability is $0.9331928$. Thus the conditional biases are approximately $-0.4847$ parameter units at the futility stop and $0.005551$ parameter units at the final analysis. The conditional early bias is negative, as expected for stopping on a very low statistic.

Symmetry, Contributions, and Diagnostics

Worked Example: Symmetric Early Stopping

Suppose a hypothetical design with $I_1=36$, $I_2=100$, and unequal increments stops early if $|Z_1|\geq1.96$; all other paths continue. At $\theta=0$, the early-stop probability is $2\{1-\Phi(1.96)\}$, approximately 0.0500. The separate upper- and lower-tail contributions are $$ \frac{\phi(1.96)}{6} =\frac{0.0584409}{6} =0.00974015 \quad\text{and}\quad -\frac{\phi(1.96)}{6} =-0.00974015 $$ parameter units. They cancel, so the combined stage-1 contribution is zero.

Here $\rho=\sqrt{36/100}=0.6$. The final-stage contribution is $$ B_2=\frac{0.6}{10}E[Z_1\,1\{|Z_1|<1.96\}]=0, $$ because the retained interval is symmetric around zero. The total bias is therefore zero under this model and rule, although the estimate on a particular stopped path need not be close to zero. For example, interim values $2.2$ and $-2.1$ give stage-1 estimates $2.2/6=0.3667$ and $-2.1/6=-0.3500$ parameter units, respectively.

A useful diagnostic is to check both the probabilities and the signs implied by the stopping rule. For a one-sided efficacy boundary, the early contribution should be positive at $\theta=0$; for a one-sided futility boundary, it should be negative. Contributions from the continuing stage should reflect the selection imposed by earlier analyses. Symmetric stopping can yield zero total bias by cancellation, not because each selected estimate is unbiased.

The main alternative is to use the estimate from a fixed analysis as if stopping had not occurred. That ignores selection and, generally, does not give the expectation of the stopped estimator. A second distinction is between conditional and unconditional assessment: $B_k$ describes the stage’s weighted share of overall bias, whereas $B_k/P(T=k)$ describes bias conditional on stopping there. Neither quantity alone is the overall bias; the contributions must be summed across stages.

These calculations depend on the assumed joint distribution and the exact stopping rule. They do not by themselves establish that the canonical approximation is accurate, nor do they provide an interval estimate. If the information values, analysis boundaries, or event definition are wrong, the resulting bias contributions can be wrong even when the integrals are evaluated correctly. For a design with several looks, unequal increments are handled by using the actual $I_k$ values in every covariance and stage-specific standard-error factor, rather than replacing them with an equal-spacing approximation.

Common Mistake

Wrong reasoning: “The early estimate is biased upward by $B_1$, so the stopped estimator’s bias is $B_1$.” This overlooks the paths that continue and their stage-specific contribution.
Correction: Calculate $B_k=E_\theta[(\hat\theta_k-\theta)1\{T=k\}]$ for every possible stopping stage, then sum them. Divide by $P_\theta(T=k)$ only when the question asks for bias conditional on stopping at stage $k$.
Another warning: Do not substitute equal information spacing into the correlation formula. At two looks, the correlation is $\sqrt{I_1/I_2}$, determined by cumulative information, not by the number of analyses.

Key takeaway: Unequal increments affect stopped-estimator bias through both the stage-specific factor $1/\sqrt{I_k}$ and the dependence among analysis statistics. Define the stopping events, use the actual information ratios, calculate every stage’s contribution, and only then add them.

Check Your Understanding

Answer each question, then compare with the solutions.

For $I_1=9$ and $I_2=36$, what is the canonical correlation between the two centered statistics?
With an early efficacy boundary $c>0$ and continuation to the final look otherwise, is the early-stage contribution positive or negative? Why?
Explain the difference between $B_2$ and the bias conditional on stopping at stage 2.
For symmetric early stopping at $|Z_1|\geq c$, why can the stage-1 contribution be zero even though the estimate is selected by stopping?
In the efficacy example, identify the unequal information increments and state the overall bias to four significant figures.

Solutions. (1) $\rho=\sqrt{9/36}=0.5$. (2) It is positive: $B_1=\phi(c)/\sqrt{I_1}>0$, because early efficacy stopping selects large positive statistics. (3) $B_2$ is the probability-weighted contribution $E_\theta[(\hat\theta_2-\theta)1\{T=2\}]$; conditional bias is $B_2/P_\theta(T=2)$. (4) The upper- and lower-tail contributions are equal in magnitude and opposite in sign, so they cancel. (5) The increments are $25$ and $75$; the overall bias is $0.008099$ parameter units.