I-SPY 2 Graduation Rules And Predictive Signatures

Why Graduation Is a Prediction

In I-SPY 2, an investigational neoadjuvant treatment is evaluated using pathologic complete response (pCR) and baseline biomarker information. The aim of a graduation rule is not to declare that a treatment has already proved efficacy. It is to decide whether the phase II evidence makes a specified future phase III success sufficiently plausible to support further development in a biomarker-defined group.

This distinction matters because a small adaptive phase II trial and a later confirmatory trial answer different questions. Phase II data update beliefs about treatment effects. A predictive calculation then asks what might happen if a phase III trial were run under a specified design and success criterion. I-SPY 2 is known for using a Bayesian predictive-probability graduation rule, commonly described as requiring at least an 85% predictive probability of success in one or more biomarker signatures. The exact future-trial assumptions and success definition must be taken from the applicable protocol and analysis plan; the threshold is not a universal statistical standard.

From Biomarker Signature to Future-Trial Outcome

A signature classifies patients using baseline biomarker information. In a treatment comparison, the relevant question is whether the treatment’s effect, relative to control, is promising in that group. A biomarker can be prognostic if outcomes differ across its levels regardless of treatment. It is predictive of treatment benefit when the comparative treatment effect varies across biomarker-defined groups. A high pCR rate in a signature by itself does not show that the treatment works better there; the treatment must be compared with control.

I-SPY 2 uses biomarker signatures to make treatment-effect estimates clinically targeted rather than relying only on an average effect across all enrolled patients. Bayesian models can estimate effects across several signatures while allowing some information to be shared across groups. The extent and form of that borrowing depend on the model. Borrowing can stabilize estimates when subgroup data are sparse, but it also means that a signature-specific estimate is not necessarily based only on patients in that signature.

Let \(D\) denote the accumulated phase II data, and let \(\theta_s\) denote the treatment-effect parameters for signature \(s\). Let \(Y_{\mathrm{III}}\) represent the future phase III data, generated under a specified design. If \(A\) is the event that the future trial meets its success criterion, then the signature-specific predictive probability is

Here, \(\pi(\theta_s\mid D)\) is the posterior distribution of the effect parameters given the phase II data and model. The inner probability accounts for the variability of a future trial given a possible true effect; the integral averages that probability over uncertainty about the effect itself. Thus, predictive probability incorporates both uncertainty in the treatment effect and sampling variability in the proposed future trial.

A graduation rule compares this probability with a prespecified threshold. In the commonly described I-SPY 2 rule, a treatment could graduate if the predictive probability of phase III success reached at least 85% in at least one signature. This is a rule for deciding whether the available evidence justifies progression. It does not imply that every signature has an 85% probability, that the treatment has an 85% probability of benefiting an individual patient, or that the future trial will succeed.

How the Predictive Probability Is Calculated

A practical calculation can use posterior predictive simulation. For each simulation, draw effect parameters from the posterior distribution based on the current data, generate a possible phase III data set under the proposed design, and apply the phase III success criterion. The proportion of simulated trials that succeed estimates the predictive probability. The simulation must reflect the planned allocation, sample size, endpoint, analysis, and success criterion; changing any of these can change the result.

The simulation has its own numerical error. If \(M\) independent predictive trials are simulated and \(K\) succeed, the estimate is \(\widehat{\operatorname{PP}}=K/M\). Conditional on the model and posterior, its approximate Monte Carlo standard error is \(\sqrt{\widehat{\operatorname{PP}}(1-\widehat{\operatorname{PP}})/M}\). This quantifies simulation noise, not uncertainty about whether the model or future-trial assumptions are correct.

Worked Examples

What the Rule Does—and Does Not—Establish

The graduation rule connects a phase II signal to a concrete development question: would the treatment likely satisfy a defined phase III criterion in a particular signature? This is more decision-focused than simply asking whether a phase II posterior mean is positive. It also makes the target population explicit. A treatment may be promising for a narrower biomarker-defined group even if its average effect across the full population is less compelling.

However, graduation is not confirmatory efficacy evidence. The predictive probability is conditional on the accumulated data, model, prior specification, and assumptions about the future trial. It forecasts a possible future result; it does not observe that result. A phase III trial can fail because the effect is smaller than expected, the future population differs, adherence or treatment delivery changes, or sampling variation produces an unfavorable estimate. The clinical importance of any effect, safety, and outcomes beyond pCR also require consideration. A graduation decision is therefore evidence to proceed with development, not a substitute for confirmatory testing.

The calculation is meaningful only if its inputs are credible. At minimum, the analysis needs a defined endpoint and signature, a suitable model for the outcome and treatment comparison, a prespecified future-trial design, and a clear success criterion. For a binary pCR endpoint, the model must appropriately represent the response probabilities and randomized treatment comparison. The biomarker information must be measured before treatment and used consistently to assign patients to signatures. Missing data, treatment discontinuation, or changes in endpoint assessment can undermine the interpretation if they are not handled as specified in the analysis plan.

Several diagnostics are useful. Check that predictive simulations reproduce plausible pCR rates and treatment differences, that the future trial’s sample size and allocation are implemented correctly, and that the success rule is applied exactly as written. Repeat the calculation with enough simulations that Monte Carlo error is small relative to the distance from the threshold. If a result lies close to 85%, report its simulation precision and examine whether modest changes in plausible model assumptions alter the decision. Calibration checks and sensitivity analyses can reveal fragility, but they cannot prove that the model will predict a future trial accurately.

Alternatives and Limitations

A fixed phase II rule might base progression on a posterior probability that the treatment effect exceeds a chosen benchmark, or on a conventional estimate and confidence interval. Such rules can be simpler to communicate, but they do not necessarily express the chance that a specified future phase III trial will succeed. Predictive probability is directly tied to a future design, at the cost of depending on more assumptions about that design and the predictive model.

A single pooled analysis across all patients is another alternative. It can summarize an overall treatment effect, but may obscure genuine differences across signatures. Conversely, examining many signatures creates opportunities for unstable subgroup patterns. Bayesian borrowing can improve precision when groups share information, but inappropriate borrowing can pull a signature estimate toward effects in other groups. Graduation should therefore be interpreted alongside the signature definitions, modeling strategy, and planned validation—not as proof that the signature is a validated treatment-selection test.