Computing Continuation Probabilities From Interim Data Models

From an Interim Data Model to a Continuation Pattern

An interim decision is made arm by arm, but the arm-specific test statistics are not generally independent when the arms use the same control group. The same control observations enter every treatment-versus-control comparison. Consequently, calculating a probability for a particular collection of arms to continue requires the joint distribution of the interim statistics, not just their separate marginal distributions.

The objective here is to calculate probabilities under a specified data-generating scenario: for example, the probability that arms 1 and 3 continue while arm 2 stops if the true treatment effects take specified values. This is a probability over repeated interim data sets under that model. It is not a probability that the model or a null hypothesis is true.

Model and assumptions: At the interim analysis, let \(n_C\) observations be available in a shared control group and \(n_i\) observations in treatment arm \(i\). Assume independent patients across groups, normally distributed outcomes with a common known variance \(\sigma^2\), and fixed sample sizes. Let \(\mu_C\) and \(\mu_i\) be the group means, with treatment effect \(\delta_i=\mu_i-\mu_C\). The interim statistic is \(Z_i=(\bar{X}_i-\bar{X}_C)/\{\sigma\sqrt{1/n_i+1/n_C}\}\). Under these assumptions, the vector of treatment-versus-control statistics has a multivariate normal distribution.

A known variance makes this model exact. With an estimated variance, the normal model can be an approximation, whose adequacy depends on the design and sample sizes. For binary or other outcomes, the corresponding joint model and covariance must be derived for the chosen estimator; the common-control covariance formula below is not automatically exact for every outcome model.

The Covariance Created by Shared Control

Each statistic has variance 1. For two distinct treatment arms \(i\) and \(j\), the treatment-group means are independent of each other and of the control mean. The only shared random component in the two mean differences is \(-\bar{X}_C\). Its variance is \(\sigma^2/n_C\), so

\operatorname{Cov}(\bar{X}_i-\bar{X}_C,\bar{X}_j-\bar{X}_C) =\operatorname{Var}(\bar{X}_C) =\frac{\sigma^2}{n_C}.

Dividing by the standard errors of the two statistics gives their correlation. The covariance is positive: a control mean that happens to be low makes both treatment-control differences larger.

\rho_{ij} = \operatorname{Corr}(Z_i,Z_j) = \frac{1/n_C} {\sqrt{(1/n_i+1/n_C)(1/n_j+1/n_C)}}.

The mean of \(Z_i\) under the specified scenario is \(m_i=\delta_i/\{\sigma\sqrt{1/n_i+1/n_C}\}\). Thus the joint model is \(Z\sim N(m,R)\), where \(m\) is the vector of these means and \(R\) is the correlation matrix with diagonal entries 1 and off-diagonal entries \(\rho_{ij}\). The effect values, allocation sizes, and control size all matter: changing any of them can change the probability of a continuation pattern.

Worked Example: Two Arms With a Shared Control

Suppose a hypothetical continuous-outcome trial has \(n_1=50\), \(n_2=75\), and \(n_C=100\) at the interim analysis. Outcomes are independent and normally distributed with common known variance, and both treatment effects are zero in the scenario being evaluated. Each arm continues when \(Z_i>0\). Find the probability that both continue and the probability of each of the other three patterns.

Under this scenario the two statistics have zero means. The shared-control correlation is

\rho_{12} = \frac{1/100} {\sqrt{(1/50+1/100)(1/75+1/100)}} = \frac{0.01000}{\sqrt{0.03000\times0.02333}} \approx 0.3780.

For a zero-mean bivariate normal pair with correlation \(\rho\), the probability that both statistics are positive is \(1/4+\arcsin(\rho)/(2\pi)\). Therefore,

P(Z_1>0,Z_2>0) = \frac14+\frac{\arcsin(0.377964)}{2\pi} \approx \frac14+\frac{0.3876}{6.2832} \approx 0.3117.

The distribution is symmetric about zero, so the probability that both stop, \(P(Z_1\leq0,Z_2\leq0)\), is also 0.3117. Each marginal probability of continuation is 0.5. Hence the probability that arm 1 continues and arm 2 stops is \(0.5-0.3117=0.1883\); the reverse pattern has the same probability. The four probabilities sum to \(0.3117+0.3117+0.1883+0.1883=1.0000\).

Representing and Calculating General Patterns

Let \(S\) be the set of arms that continue, and let \(S^c\) be the set that stop. If arm \(i\) continues when \(Z_i>a_i\), the event for this exact pattern is \(\{Z_i>a_i \text{ for } i\in S;\; Z_j\leq a_j \text{ for } j\in S^c\}\). Its probability is the multivariate-normal probability over that rectangular region, using the model mean vector and correlation matrix. Different arms may have different thresholds.

For two arms, for example, the probability that arm 1 continues and arm 2 stops is

P(Z_1>a_1,Z_2\leq a_2) = P(Z_1>a_1)-P(Z_1>a_1,Z_2>a_2).

The joint upper-tail probability can be calculated from a bivariate normal distribution function, by numerical integration, or by a validated multivariate-normal probability routine. For more arms, the same event definition applies, but numerical integration or simulation is usually more practical than a closed-form expression.

A useful special case is the zero-mean bivariate model with both thresholds equal to zero. Its quadrant formula follows from the geometry of a bivariate normal distribution. Write \(Z_1=U\) and \(Z_2=\rho U+\sqrt{1-\rho^2}V\), where \(U\) and \(V\) are independent standard normal variables. Their joint density is rotationally symmetric in the \((U,V)\) plane. The angular region satisfying both inequalities \(Z_1>0\) and \(Z_2>0\) has angle \(\pi/2+\arcsin(\rho)\); dividing by the full angle \(2\pi\) gives

P(Z_1>0,Z_2>0) = \frac{\pi/2+\arcsin(\rho)}{2\pi} = \frac14+\frac{\arcsin(\rho)}{2\pi}.

This derivation also makes clear why the product of the marginal probabilities is wrong when \(\rho\neq0\): it would assume independent angles, even though the common control induces dependence.

Worked Example: An Exact Pattern Among Three Arms

Consider a hypothetical three-arm trial with 100 observations in each treatment group and 100 in the shared control group. Assume the normal known-variance model, zero treatment effects, and a continuation threshold of zero for every arm. Find the probability that arm 1 continues while arms 2 and 3 stop.

All three statistics have mean zero. For any distinct pair, the shared-control correlation is

\rho_{ij} = \frac{1/100}{\sqrt{(1/100+1/100)(1/100+1/100)}} = \frac{0.01000}{0.02000} =0.5000.

The event of interest is \(Z_1>0,\;Z_2\leq0,\;Z_3\leq0\). Begin with \(P(Z_1>0)=0.5\), subtract the events in which arm 1 and arm 2 both continue and in which arm 1 and arm 3 both continue, then add back the event in which all three continue:

\begin{aligned} P(Z_1>0,Z_2\leq0,Z_3\leq0) ={}&P(Z_1>0)-P(Z_1>0,Z_2>0)\\ &-P(Z_1>0,Z_3>0)+P(Z_1>0,Z_2>0,Z_3>0). \end{aligned}

Each pairwise probability is \(1/4+\arcsin(0.5)/(2\pi)=1/3\). For a zero-mean trivariate normal vector, the all-positive probability is \(1/8+\{\arcsin(\rho_{12})+\arcsin(\rho_{13})+\arcsin(\rho_{23})\}/(4\pi)\). Here this equals \(1/8+3(\pi/6)/(4\pi)=1/4\). Substitution gives

0.5000-\frac13-\frac13+0.2500 = 0.08333.

Thus this particular pattern has probability \(0.08333\), or about 8.33%. By symmetry, each pattern with exactly one continuing arm has the same probability. The chance all three continue is 0.2500, and the chance all three stop is also 0.2500. These results provide useful checks on a calculation of all eight patterns.

Allowing Nonzero Effects and Nonzero Boundaries

The joint-normal calculation is not limited to a global-null scenario or to zero thresholds. Under a specified alternative, the means \(m_i\) shift, and a boundary \(a_i\) must be compared with that shifted distribution. Equivalently, standardize around the scenario mean: \(X_i=Z_i-m_i\) has zero mean and the same correlation matrix, while the continuation condition becomes \(X_i>a_i-m_i\). This translation is often helpful for checking signs and interpreting the calculation.

Worked Example: A Shifted Interim Model

Use the sample sizes and correlation from the two-arm example: \(n_1=50\), \(n_2=75\), and \(n_C=100\), with \(\sigma=10\) outcome units. Consider the hypothetical scenario \(\delta_1=10\sqrt{0.03000}\approx1.732\) outcome units and \(\delta_2=0\). Suppose the continuation boundaries are \(a_1=1\) and \(a_2=0\). Find the probability both arms continue.

The standard errors are \(10\sqrt{0.03000}=1.732\) outcome units for arm 1 and \(10\sqrt{0.02333}\approx1.528\) outcome units for arm 2. Thus the standardized means are \(m_1=1.732/1.732=1\) and \(m_2=0/1.528=0\). Subtracting the means gives thresholds \(a_1-m_1=0\) and \(a_2-m_2=0\) for the centered variables. The correlation remains approximately 0.3780, so the bivariate quadrant calculation gives

P(Z_1>1,Z_2>0) = P(X_1>0,X_2>0) = \frac14+\frac{\arcsin(0.377964)}{2\pi} \approx 0.3117.

This probability is conditional on the specified model inputs—sample sizes, variance, effects, and boundaries—in the sense that it is evaluated under that scenario. It is not a probability conditional on particular realized interim statistics. Conditioning on already observed interim data would require a model for future increments and is a different calculation.

Numerical Methods, Diagnostics, and Common Pitfalls

For arbitrary means, thresholds, and three or more arms, the required probability is typically evaluated numerically. A practical workflow is to specify the scenario and continuation rule, construct the full mean vector and correlation matrix, define the inequalities for the exact pattern, and then evaluate the resulting multivariate-normal probability. A simulation estimate can also be formed by repeatedly drawing \(Z\sim N(m,R)\) and counting the draws in the pattern region; its Monte Carlo standard error is approximately \(\sqrt{\hat{q}(1-\hat{q})/B}\) for estimated probability \(\hat{q}\) from \(B\) draws.

Check the covariance structure. In the equal-variance shared-control model, the off-diagonal correlations should be positive and less than 1. A negative correlation or a value above 1 signals an error in the covariance or standardization.
Check marginal probabilities. Summing all pattern probabilities that include arm \(i\) must reproduce \(P(Z_i>a_i)\) under that arm's marginal normal distribution.
Check the partition. The probabilities of all mutually exclusive continuation patterns must sum to 1, up to numerical integration or simulation error.
Check numerical precision. Results near a threshold can be sensitive to rounding; use unrounded correlations and adequate integration accuracy, then report a sensible number of significant figures.

A common mistake is to multiply marginal continuation probabilities as if the arms were independent. In the first example, that calculation gives \(0.5\times0.5=0.2500\) for both arms continuing, whereas the correct joint probability is 0.3117. The shared-control correlation makes simultaneous large treatment-control statistics more common than independence would suggest in that zero-mean, zero-boundary case. Independence is appropriate only if the joint model actually gives zero correlations (or if an approximation is explicitly justified); it should not be assumed merely because treatment groups are separately randomized.

Other approaches answer different questions. Bonferroni bounds can provide conservative probability bounds, but do not calculate the exact pattern probability. A multivariate-normal routine or numerical integration directly targets the joint event. Simulation is flexible and useful as an independent check, but rare patterns may require many draws for adequate precision. Finally, these calculations depend on the assumed interim data model: departures from normality, variance misspecification, dependence across patients, or a different estimand can affect the resulting probabilities.

Key takeaway: A continuation pattern is a region in the joint distribution of the interim test statistics. With a shared control, construct the covariance induced by that common data before calculating the region probability; separate marginal probabilities alone are not enough.

Check Your Understanding

For each problem, state the calculation and compare it with the solution provided.

In a two-arm zero-mean model with zero boundaries and correlation \(\rho=0\), what is the probability both arms continue? Solution: The quadrant formula gives \(1/4+\arcsin(0)/(2\pi)=0.2500\), matching the product of the two marginal probabilities only because the statistics are independent.
In the three-arm example, what is the probability that no arm continues? Solution: By joint sign symmetry under the zero-mean model and zero boundaries, the all-negative probability equals the all-positive probability, which is 0.2500.
Why does increasing the shared-control sample size generally reduce the correlation between two treatment-control statistics when the treatment sample sizes are held fixed? Solution: The shared covariance contains \(1/n_C\), which decreases as \(n_C\) increases. The standard-error factors also change, so the correlation is determined by the full formula, but it tends toward zero as the common-control contribution becomes negligible relative to the treatment contributions.
With a nonzero mean \(m_i\), what boundary applies to the centered statistic \(X_i=Z_i-m_i\)? Solution: The original rule \(Z_i>a_i\) becomes \(X_i>a_i-m_i\), since subtracting \(m_i\) from both sides preserves the inequality.