Why Reporting Details Matter
A conditional or predictive power value is not a design feature that can be interpreted on its own. It is a probability calculated from the interim evidence, a model for future data, a definition of success, and assumptions about the treatment effect. Changing any of these can change the answer while leaving the reported percentage looking equally precise. A useful report therefore exposes the ingredients behind the number, not just the number itself.
The earlier material on conditional and predictive power distinguishes a calculation under a fixed effect from one that averages over uncertainty about the effect. Here the emphasis is on making either calculation reproducible. In particular, “power was 60%” is incomplete: a reader needs to know what was observed, what future data were assumed, and what event counted as success.
What a Reproducible Report Specifies
The following information lets another analyst understand and, in principle, reproduce the calculation. Give the actual values used rather than relying on labels such as “current information” or “assumed effect.”
- Interim data: identify the analysis time and the statistic or sufficient summaries used. For a canonical normal analysis, report the observed standardized statistic \(Z(t)=z\), the information fraction \(t\), and the data cut-off or interim look. State whether the trial had crossed any earlier stopping boundary.
- Future information: state the planned remaining information and how future observations are modeled. Identify assumptions such as independent future increments, a constant standardized treatment effect, and the variance or information schedule. If nuisance parameters are estimated or integrated over, say how.
- Success criterion: define the event precisely. For example, it could be crossing a specified final efficacy boundary, or crossing any of several future boundaries. State whether success means statistical significance, a clinical threshold, or another rule; these are not interchangeable.
- Effect assumption or distribution: for conditional power, give the effect value and its scale. For predictive power, give the prior and the resulting posterior distribution or weights used in the calculation. State whether the distribution is conditional on interim data.
- Computation and uncertainty: report the formula or algorithm, software or numerical method, integration tolerances or simulation size, and random-number seed when relevant. Separate numerical error from uncertainty in the effect assumptions and future-data model.
For a simple one-sided final-look calculation, the canonical model makes these ingredients explicit. Let \(t\) be the interim information fraction, \(z\) the observed interim \(Z\)-statistic, \(\theta\) a fixed standardized effect on the information scale, and \(c\) the final efficacy boundary. Assume a canonical normal model with independent information increments and a constant effect. Conditional on \(Z(t)=z\), the final statistic has a normal distribution with mean \(\sqrt{t}\,z+(1-t)\theta\) and variance \(1-t\). The conditional power is therefore
The reported value is specific to that model and event. If there are multiple future looks, success is usually the event of crossing at least one applicable boundary, so a single-final-look formula does not by itself describe the required probability.
Why the Conditional Formula Has This Form
A short derivation also helps detect mismatches between a reported formula and the intended model. Write the canonical statistic at information fraction \(u\) as \(Z(u)=\{B(u)+\theta u\}/\sqrt{u}\), where \(B(u)\) is standard Brownian motion. Given \(Z(t)=z\), the observed Brownian value is \(B(t)=\sqrt{t}z-\theta t\). The future increment \(B(1)-B(t)\) is independent of \(B(t)\) and has mean zero and variance \(1-t\). Substituting into the final statistic gives
Thus the conditional mean is \(\sqrt{t}z+(1-t)\theta\), and the conditional variance is \(1-t\). Standardizing the boundary \(c\) yields the displayed conditional-power formula. This derivation requires \(0<t<1\), the canonical normal model, and a fixed effect \(\theta\); departures from those assumptions require a different calculation.
Worked Example: Reporting Conditional Power at One Final Look
Suppose a hypothetical trial has reached \(t=0.60\), with interim statistic \(z=1.50\). It has not met an earlier stopping rule, and the planned one-sided final success criterion is \(Z(1)\ge 2.00\). For this calculation, assume a fixed standardized effect \(\theta=2.50\), independent future information increments, and the canonical normal model.
First calculate the conditional final mean and standard deviation:
The distance from this mean to the success boundary, in conditional standard deviations, is
Therefore \(CP(2.50)=\Phi(0.2560)\approx 0.6010\). Under the stated fixed-effect model, the chance of meeting this final boundary after the observed interim result is about 60.1%. This is not the probability that the treatment effect equals 2.50, nor does it account for uncertainty about that assumption.
Predictive Power and Effect Uncertainty
Predictive power averages the conditional success probability over a distribution for the effect after observing the interim data. If \(\pi(\theta\mid D)\) is the posterior distribution of \(\theta\) given interim data \(D\), then, for the one-final-look event above,
A report must identify the distribution being averaged over. Naming only a prior family is not enough if the posterior also depends on the interim likelihood, parameterization, and other model choices. For a discrete posterior, report the possible effect values and their posterior weights. For a continuous posterior, report its specification and the numerical integration or simulation procedure.
Worked Example: Averaging Conditional Power Over Posterior Weights
Use the interim data, final boundary, and canonical model from the preceding example. Suppose, hypothetically, that the posterior distribution at the interim look assigns probabilities 0.25, 0.50, and 0.25 to \(\theta=1.50,\ 2.50,\ 3.50\), respectively. These are posterior weights, not unupdated prior weights.
The conditional mean is \(\sqrt{0.60}(1.50)+(0.40)\theta=1.1619+0.40\theta\). For \(\theta=1.50\), the standardized boundary distance is
For \(\theta=2.50\), the distance is approximately 0.2560 and \(CP(2.50)\approx0.6010\). For \(\theta=3.50\), it is
Weight the component probabilities by their posterior probabilities:
The predictive probability is about 59.2% under this posterior and this success rule. Reporting the posterior values and weights makes the calculation auditable; reporting only “predictive power 59.2%” would conceal what drove the result.
Multiple Looks and Numerical Reporting
When success can occur at more than one future look, define the event over the entire remaining path. Let future looks occur at information times \(t_1,\ldots,t_J\), with \(t<t_1<\cdots<t_J\), and let \(b_j\) be the efficacy boundary at look \(j\). Conditional power is the probability that \(Z(t_j)\ge b_j\) for at least one future look, conditional on the interim data and the assumed effect. If the analysis includes other stopping rules, state how they affect continuation and the event. A terminal-only calculation can overstate or understate the probability for a multi-look criterion because it ignores the specified pathwise rule.
For a numerical calculation, identify whether probabilities were obtained from a closed-form calculation, numerical integration, or simulation. With simulation, report the number of predictive paths, the random-number seed and generator, and how each path represented the posterior uncertainty and future increments. Give a Monte Carlo standard error or interval when the simulation error is consequential. Such an interval quantifies simulation noise under the chosen model; it does not include uncertainty from choosing a different prior, effect model, or boundary.
Worked Example: Reporting a Multi-Look Simulation
Consider a hypothetical trial with interim information fraction \(t=0.50\) and observed \(z=0.80\). Its prespecified remaining success rule is to cross \(Z(0.75)\ge2.20\) or \(Z(1.00)\ge2.00\). Suppose the analyst uses a stated posterior distribution for \(\theta\), samples an effect from that distribution for each path, then simulates future canonical-model increments conditional on the interim statistic. In a hypothetical run of 50,000 paths using a fixed, reported pseudorandom generator and seed, 29,610 paths cross at least one of the two boundaries.
The estimated predictive probability is
Treating the simulated path indicators as independent Bernoulli draws, the Monte Carlo standard error is
A normal-approximation 95% interval for simulation error alone is \(0.5922\pm1.96(0.00220)\), or approximately 0.5879 to 0.5965. A reproducible report would also give the posterior distribution, interim value, model, look times, both boundaries, path-generation procedure, number of paths, and seed. It should describe the interval as Monte Carlo precision, not as a 95% interval for the true treatment effect or for all sources of predictive uncertainty.
Diagnostics, Comparisons, and Limitations
A transparent report makes it easier to find errors before a probability is used in a decision. Check that the interim look and information fraction correspond to the stated analysis; that the effect is on the scale required by the formula; and that the sign and direction of the boundary match the alternative hypothesis. For numerical work, verify that posterior weights sum to one, probabilities lie between zero and one, and simulation estimates stabilize as the number of paths increases. Compare a simulation result with a simple analytic result in a setting where both are available. Large discrepancies can indicate a boundary implemented incorrectly, omitted interim conditioning, incorrect future information, or a coding error.
Conditional power and predictive power answer different questions. Conditional power fixes an effect value and asks what follows under that assumption. Predictive power averages over a stated distribution and therefore reflects both the interim data and that distribution. Neither is a substitute for the prespecified type I error or efficacy boundary: a high estimated power does not itself establish statistical significance, and a low value does not show that the null hypothesis is true. These probabilities are planning or decision summaries conditional on models, not guarantees of trial success.
Common limitations include sensitivity to the assumed effect or posterior, misspecification of the future-data model, and reliance on an asymptotic normal approximation where it is inaccurate. A highly precise simulation can still give a misleading answer if its assumptions are poorly justified. Reports should therefore separate numerical precision from model sensitivity and avoid presenting extra decimal places as substantive certainty.
Check Your Understanding
Answer each question using the reporting principles and calculations above.
- In the one-final-look example, what does the reported 60.1% conditional power condition on, and what does it not represent?
- Why must a predictive-power report identify posterior weights rather than just state that a prior was used?
- For 29,610 successes in 50,000 simulated paths, give the estimated predictive probability and approximate Monte Carlo standard error.
- A report gives \(t=0.60\), \(z=1.50\), and \(\theta=2.50\), but does not state the success boundary. What essential information is missing?
Solutions. (1) It conditions on the observed interim statistic and information fraction, the canonical future-data model, the fixed effect \(\theta=2.50\), and the final boundary \(Z(1)\ge2.00\). It does not represent uncertainty about \(\theta\) or guarantee success. (2) The posterior distribution determines how conditional probabilities are weighted; different posterior distributions can produce different predictive probabilities from the same interim statistic. (3) The estimate is \(29{,}610/50{,}000=0.5922\), and \(SE_{\mathrm{MC}}=\sqrt{0.5922(0.4078)/50{,}000}\approx0.00220\). (4) The boundary and success event are missing, so the event whose probability is being calculated is undefined.