This page separates reported trial results from statistical interpretation. Numerical results are taken from the ClinicalTrials.gov record. The registry provides the official trial record.
1. Trial at a Glance
I-SPY 2 was a completed single-group diagnostic study evaluating diffusion-weighted magnetic resonance imaging in patients with breast cancer receiving neoadjuvant chemotherapy. The primary registered endpoint was pathologic complete response (pCR), assessed at surgery, with changes in apparent diffusion coefficient evaluated as predictors of pCR.
| Feature | I-SPY 2 |
|---|---|
| Trial name | I-SPY 2 |
| Brief title | DWI in Assessing Treatment Response in Patients With Breast Cancer Receiving Neoadjuvant Chemotherapy |
| Phase | NA |
| Status | COMPLETED |
| Study design | Single-group diagnostic study; no masking |
| Allocation | NA |
| Primary purpose | DIAGNOSTIC |
| Enrollment | 406 |
| Arms | 1 |
| Intervention | Diffusion-weighted magnetic resonance imaging (procedure) |
| Lead sponsor | American College of Radiology Imaging Network |
| Sponsor type | NETWORK |
| Start | 2012-08-27 |
| Primary completion | 2018-07-19 |
| ClinicalTrials.gov | NCT01564368 |
2. Clinical Question
The central statistical question was whether changes measured by diffusion-weighted MRI could discriminate between patients who subsequently achieved a pathologic complete response at surgery and those who did not.
Population
Patients with breast cancer receiving neoadjuvant chemotherapy.
Intervention
Diffusion-weighted magnetic resonance imaging.
Reference outcome
Pathologic complete response assessed at surgery using histopathologic analysis and the Residual Cancer Burden system.
Primary question
Can changes in apparent diffusion coefficient distinguish patients with pCR from patients without pCR?
3. Trial Design
There was one study arm: diffusion-weighted MRI. The statistical comparisons shown in the registry are therefore not randomized treatment-arm comparisons. Instead, the MRI-derived measurements were evaluated for their ability to discriminate the binary pathologic outcome of pCR at surgery.
4. Primary Endpoint
| Endpoint | Time frame | Registry definition |
|---|---|---|
| Pathologic Complete Response (pCR) | Surgery | Pathologic complete response (pCR) is defined as the lack of all signs of cancer in tissue samples removed during surgery after Neoadjuvant treatment for Breast cancer. ie., no residual invasive disease in either breast or axillary lymph nodes after neoadjuvant therapy (ypT0/is, ypN0). Histopathologic analysis was performed using the Residual Cancer Burden system. |
The registered primary endpoint is binary because each participant ultimately falls into a pCR or non-pCR category for the analysis. The MRI measurements are continuous predictors, while the reference outcome is the pathologic classification obtained at surgery.
5. MRI Predictors and Analysis Windows
The statistical analyses evaluated percentage changes in apparent diffusion coefficient (ADC) and functional tumor volume (FTV) relative to baseline. The registry identifies three MRI change windows for the primary ADC analyses.
| Measurement window | ADC change used in analysis | Reference outcome |
|---|---|---|
| Early Treatment Change | (ADC1 - ADC0) / ADC0 | pCR |
| Mid-Treatment Change | (ADC2 - ADC0) / ADC0 | pCR |
| Post-Treatment Change | (ADC3 - ADC0) / ADC0 | pCR |
For the secondary FTV analyses, the corresponding changes were defined using FTV1, FTV2, and FTV3 relative to FTV0. These measurements were evaluated as predictors of the same binary pCR outcome.
| Measurement window | FTV change used in analysis | Reference outcome |
|---|---|---|
| Early Treatment Change | (FTV1 - FTV0) / FTV0 | pCR |
| Mid-Treatment Change | (FTV2 - FTV0) / FTV0 | pCR |
| Post-Treatment Change | (FTV3 - FTV0) / FTV0 | pCR |
6. Statistical Methodology
Receiver Operating Characteristic curves
The registry states that receiver operating characteristic curves and their corresponding area under the curve (AUC) were estimated for the changes in ADC and FTV, using pCR as the reference outcome.
An ROC curve evaluates the tradeoff between sensitivity and specificity as a continuous predictor is examined across possible thresholds. In this setting, the MRI-derived change is the predictor and pCR is the binary reference outcome.
Conceptually, the AUC can be interpreted as the probability that the predictor assigns a higher value to a randomly selected participant with pCR than to a randomly selected participant without pCR, subject to the direction in which the predictor is defined.
Wald / z-test
The registry reports a Z-test for the AUC analyses. The statistical method is Wald / z-test, within the categorical-data method category.
The reported hypothesis was framed around detecting a difference of 0.15 between the null AUC of 0.5 and an alternative AUC of 0.65. The registry's analysis notes state that the ROC/AUC analysis was designed to detect this difference.
The reported analysis therefore evaluates whether discrimination differs from the null reference represented by an AUC of 0.5, with the alternative design value specified as 0.65.
One-sided confidence intervals
The primary AUC analyses report 95% one-sided confidence intervals, with an upper confidence limit recorded in the registry. These are different from the more familiar two-sided 95% confidence intervals that provide both a lower and upper bound.
7. Primary Results: Pathologic Complete Response
ClinicalTrials.gov posts three statistical analyses for the registered primary pCR endpoint. Each uses a Z-test to evaluate the AUC for an MRI-derived change measure against pCR at surgery.
Early Treatment Change
ADC change AUC
95% one-sided CI upper: 0.61 · P = 0.484
Test: (ADC1 - ADC0) / ADC0 · Reference: pCR
The reported AUC of 0.53 indicates that the early treatment change in ADC had an estimated discrimination measure close to the null AUC of 0.5. In an ROC framework, an AUC of 0.5 represents no discrimination beyond the null reference.
The AUC is a measure of discrimination; it does not tell us how much an individual patient's ADC changed, what threshold should be used clinically, or the probability that a particular patient will achieve pCR.
The reported 95% one-sided confidence interval has an upper limit of 0.61. Because only the upper limit is reported, the registry does not provide the corresponding lower confidence limit in the ClinicalTrials.gov record.
The P-value of 0.484 measures evidence against the specified null framework; it does not measure the magnitude or clinical importance of the AUC. A P-value should therefore not be interpreted as a probability that the null hypothesis is true.
The analysis is also based on a particular MRI measurement window and a binary pathologic reference outcome. Its interpretation depends on how those measurements and outcome classifications were obtained.
Mid-Treatment Change
ADC change AUC
95% one-sided CI upper: 0.68 · P = 0.017
Test: (ADC2 - ADC0) / ADC0 · Reference: pCR
The estimated AUC of 0.60 indicates modest discrimination between participants with and without pCR based on the mid-treatment change in ADC. The estimate is above the null reference of 0.5.
This AUC does not mean that 60% of patients achieved pCR, nor does it mean that the predictor correctly classifies exactly 60% of patients. AUC summarizes discrimination across possible thresholds rather than performance at one selected cutoff.
The reported 95% one-sided confidence interval has an upper limit of 0.68. Since the registry reports only the upper limit, a lower confidence limit should not be constructed from the ClinicalTrials.gov record.
The P-value of 0.017 describes the evidence under the reported hypothesis-testing framework. It is not an effect-size measure and does not quantify the probability that the MRI predictor is clinically useful.
The result also needs to be distinguished from the other MRI windows. A statistically reported result for the mid-treatment measurement does not automatically establish that the early or post-treatment measurements have the same performance.
Post-Treatment Change
ADC change AUC
95% one-sided CI upper: 0.69 · P = 0.013
Test: (ADC3 - ADC0) / ADC0 · Reference: pCR
The reported AUC of 0.61 indicates modest discrimination between pCR and non-pCR participants using post-treatment change in ADC. It is above the null AUC of 0.5 specified in the analysis framework.
The estimate is not a sensitivity, specificity, positive predictive value, negative predictive value, or probability of pCR. It is an aggregate measure of discrimination over ROC thresholds.
The 95% one-sided confidence interval has an upper limit of 0.69. The ClinicalTrials.gov record does not include a lower confidence limit, so the precision of the estimate should be described using the information actually reported rather than an inferred two-sided interval.
The P-value of 0.013 is evidence from the specified statistical test; it does not measure the size of the AUC or the clinical value of the imaging measure.
As with the other primary analyses, the result is tied to the pCR reference outcome at surgery and to the specified post-treatment MRI change. It should not be generalized automatically to other endpoints or measurement schedules.
| Primary analysis | MRI predictor | AUC | 95% one-sided CI upper | P-value |
|---|---|---|---|---|
| Early Treatment Change | (ADC1 - ADC0) / ADC0 | 0.53 | 0.61 | 0.484 |
| Mid-Treatment Change | (ADC2 - ADC0) / ADC0 | 0.60 | 0.68 | 0.017 |
| Post-Treatment Change | (ADC3 - ADC0) / ADC0 | 0.61 | 0.69 | 0.013 |
8. Secondary Results: Functional Tumor Volume
Three secondary analyses evaluated functional tumor volume change as a predictor of pCR. Each analysis used the same ROC/AUC framework and a Z-test.
| Measurement window | FTV predictor | AUC | 95% one-sided CI upper | P-value |
|---|---|---|---|---|
| Early Treatment Change | (FTV1 - FTV0) / FTV0 | 0.68 | 0.75 | <0.001 |
| Mid-Treatment Change | (FTV2 - FTV0) / FTV0 | 0.63 | 0.71 | <0.001 |
| Post-Treatment Change | (FTV3 - FTV0) / FTV0 | 0.68 | 0.75 | <0.001 |
The registry's analysis notes state that the FTV analyses were designed around detecting a difference of 0.15 between an AUC under the null hypothesis of 0.5 and an alternative AUC of 0.65. The ClinicalTrials.gov record also state that the assumed number of pCR non-responders would be approximately 2 for these analyses.
The FTV AUC estimates of 0.68, 0.63, and 0.68 describe discrimination for early, mid-treatment, and post-treatment FTV changes, respectively. These are AUC estimates, not percentages of patients correctly diagnosed and not pCR rates.
All three reported P-values are <0.001. This indicates that the registry's statistical tests provide strong evidence under their specified testing framework against the corresponding null framework. The P-values do not establish the magnitude of clinical benefit or identify a particular imaging threshold.
The confidence information consists of one-sided 95% upper limits of 0.75, 0.71, and 0.75. No lower limits are reported in the ClinicalTrials.gov record.
Because these are secondary analyses and multiple MRI-derived predictors and measurement windows were evaluated, the results should be interpreted as a set of related analyses rather than as three completely independent confirmatory experiments.
9. Predictive Model Analysis
The seventh posted statistical analysis evaluated a predictive model combining change in tumor ADC, change in tumor volume, and other variables. The analysis compared an optimized model with ΔADC alone.
Optimized model vs ΔADC alone
Hypothesis type: Equivalence
Registry analysis note: AUC (optimized) = AUC (ΔADC)
| Feature | Reported information |
|---|---|
| Outcome | Determine the accuracy of predictive models including covariates for combined measurement of change in tumor ADC value, change in tumor volume, and other variables |
| Time frame | baseline and mid-treatment |
| Outcome unit | probability |
| Groups compared | Optimized Model vs ΔADC Alone |
| Method | z-test; normalized as Wald / z-test |
| P-value | 0.032 |
| Hypothesis type | Equivalence |
| Equivalence margin | No equivalence margin was used |
| Analysis note | AUC (optimized) = AUC (ΔADC) |
The analysis compares an optimized predictive model with ΔADC alone. The registry records an equivalence hypothesis and a P-value of 0.032, but it also states that no equivalence margin was used.
That distinction is important. In a conventional equivalence design, the statistical question is not simply whether a difference is statistically significant. The question is whether the estimated difference lies within a prespecified interval of acceptable differences.
Because the registry does not provide such a margin, the result cannot be translated into a conventional statement such as "the two models were shown to be equivalent within X." The ClinicalTrials.gov record does not support specifying such an X.
The reported analysis note, AUC (optimized) = AUC (ΔADC), describes the equality being assessed but does not provide separate AUC estimates in the ClinicalTrials.gov record.
10. Statistical Methods Explained
What does an AUC of 0.60 mean?
An AUC of 0.60 means the predictor has discrimination above the null reference of 0.5, under the direction of the predictor used to construct the ROC curve. It does not mean that 60% of participants are correctly classified, because classification accuracy depends on the threshold selected and on the distribution of outcomes.
Why use an ROC curve for pCR?
pCR is a binary reference outcome, whereas ADC and FTV changes are continuous measurements. An ROC curve is a natural way to examine how a continuous predictor separates the two outcome categories across possible thresholds. The AUC then summarizes discrimination across those thresholds.
Why was a Z-test used?
The registry reports a Z-test for the AUC analyses and normalizes the method as a Wald / z-test. A Wald-type statistic generally compares an estimated quantity with a null value after scaling the difference by an estimated standard error. In this trial's reported framework, that approach is used for the AUC hypothesis tests.
The precise variance calculation for an AUC depends on the underlying ROC analysis. The formula above is the general statistical structure of a Wald-type test rather than a reconstruction of an unreported calculation from this registry record.
Why is 0.5 the null AUC?
An AUC of 0.5 corresponds to the null discrimination reference in which the predictor does not distinguish the two outcome categories better than chance under the ROC formulation. The registry specifically describes detecting a difference of 0.15 between an AUC under H0 of 0.5 and an alternative AUC of 0.65.
Why does a P-value not measure the size of the AUC?
The AUC itself is the effect measure. The P-value addresses compatibility of the observed data with the specified null hypothesis under the statistical model. A small P-value can occur for a relatively modest effect when information is sufficiently large, while a larger P-value can occur for a potentially meaningful estimate when precision is limited.
Why does the one-sided confidence interval matter?
The registry reports one-sided 95% confidence intervals with upper limits of 0.61, 0.68, and 0.69 for the three primary ADC analyses. A one-sided interval provides a bound in one direction rather than the two-sided lower-and-upper range commonly seen in standard confidence intervals. The missing lower limits should not be inferred.
How should the equivalence analysis be interpreted?
Equivalence requires a prespecified margin that defines an acceptable difference. Here, the registry explicitly states that no equivalence margin was used. Therefore, the P-value of 0.032 cannot by itself establish conventional margin-based equivalence.
11. Multiplicity and Multiple MRI Analyses
The registry contains seven posted statistical analyses: three primary analyses for ADC change, three secondary analyses for FTV change, and one predictive-model analysis. The three primary analyses evaluate different treatment-response windows, while the secondary analyses evaluate a second imaging measure.
| Analysis family | Number reported | Role |
|---|---|---|
| ADC change vs pCR | 3 | Primary |
| FTV change vs pCR | 3 | Secondary |
| Optimized model vs ΔADC alone | 1 | Secondary / predictive-model analysis |
Multiple analyses create an important interpretive issue: the probability of obtaining at least one small P-value can increase when many hypotheses are tested. The ClinicalTrials.gov record identifies the individual hypothesis types but do not provide an overall multiplicity-adjustment procedure for the seven posted analyses.
12. Analysis Populations and Missing Data
The ClinicalTrials.gov record reports an enrollment of 406 and serious adverse events of 0/406 for the diffusion-weighted MRI arm. They do not provide separate analysis-population definitions, exclusions, missing-data handling rules, imputation procedures, or the number of participants contributing to each statistical analysis.
That limitation matters particularly for diagnostic analyses. ROC and AUC estimates depend on the observations available for both the predictor and the reference outcome. If MRI measurements or surgical pathology classifications are unavailable for some participants, the effective analysis sample can differ from total enrollment.
13. Safety
The ClinicalTrials.gov record reports serious adverse events by arm as follows:
| Arm | Serious adverse events affected / at risk |
|---|---|
| Diffusion Weighted-MRI | 0/406 |
Serious adverse events
Diffusion Weighted-MRI arm
This is a descriptive safety result. It does not establish that the intervention carries zero risk in a broader population, because the registry value describes the serious-adverse-event count among the 406 participants represented in the registry-reported arm-level record.
14. What the AUC Does — and Does Not — Mean
An AUC of 0.61, for example, summarizes the discrimination of the specified post-treatment ADC-change predictor with respect to the binary pCR reference outcome. It does not mean that 61% of patients experienced pCR or that the predictor is correct for 61% of individual patients.
The reported confidence limit describes statistical uncertainty around the AUC under the specified analysis framework. For the post-treatment ADC analysis, the ClinicalTrials.gov record gives a 95% one-sided upper limit of 0.69.
The P-value addresses the hypothesis test rather than the magnitude of discrimination. For post-treatment ADC change, the reported P-value is 0.013; for mid-treatment change it is 0.017. Neither P-value is a measure of the AUC itself.
The three primary estimates correspond to different MRI change windows. The early, mid-treatment, and post-treatment analyses therefore answer related but distinct predictive questions. An estimate from one window should not be substituted for another.
15. Interpreting the Three Primary AUC Results Together
| Window | AUC | P-value | Statistical reading |
|---|---|---|---|
| Early Treatment Change | 0.53 | 0.484 | Estimate close to the null AUC of 0.5; the reported test does not provide strong evidence against the null framework. |
| Mid-Treatment Change | 0.60 | 0.017 | Estimate above 0.5 with a reported P-value of 0.017. |
| Post-Treatment Change | 0.61 | 0.013 | Estimate above 0.5 with a reported P-value of 0.013. |
The statistical pattern is therefore not uniform across measurement timing. The early-treatment ADC-change analysis produced an AUC of 0.53, whereas the mid-treatment and post-treatment analyses produced AUCs of 0.60 and 0.61. The ClinicalTrials.gov record supports describing these as different observed levels of discrimination rather than collapsing them into one summary estimate.
The FTV analyses provide a second perspective: their AUC estimates were 0.68, 0.63, and 0.68 across the corresponding windows, with all three reported P-values <0.001. These secondary analyses therefore provide a distinct imaging-predictor result rather than simply duplicating the ADC analysis.
16. Limitations and Interpretation Issues
- Single-group design: there is one diffusion-weighted MRI arm and no randomized comparator arm. The statistical analyses therefore address diagnostic discrimination rather than comparative treatment efficacy.
- Diagnostic reference outcome: the imaging predictors are evaluated against pCR at surgery. AUC measures discrimination for that reference outcome and should not automatically be interpreted as a measure of other clinical outcomes.
- Multiple measurement windows: early, mid-treatment, and post-treatment MRI changes were separately analyzed. Their estimates should not be treated as interchangeable.
- Multiple analyses: seven statistical analyses are posted, including primary ADC analyses, secondary FTV analyses, and a predictive-model analysis. The ClinicalTrials.gov record does not specify a global multiplicity-adjustment procedure.
- One-sided confidence intervals: the registry reports only upper limits for the 95% one-sided intervals.
- Equivalence margin: the predictive-model analysis is labelled as equivalence, but the registry explicitly states that no equivalence margin was used. Conventional margin-based equivalence should therefore not be inferred.
- Analysis populations: the ClinicalTrials.gov record does not report the number of participants included in each statistical analysis or any separate exclusions from the enrollment population.
- Missing data: no missing-data or imputation method is provided in the ClinicalTrials.gov record.
- Safety scope: the ClinicalTrials.gov record is limited to serious adverse events by arm, reported as 0/406.
- AUC interpretation: AUC summarizes discrimination across thresholds and does not specify the sensitivity, specificity, predictive value, or clinical utility at any particular cutoff.
17. Why This Trial Matters Statistically
I-SPY 2 is useful as a statistical teaching case because it shows how an imaging biomarker can be evaluated against a binary clinical or pathological reference outcome without using a randomized treatment comparison.
| Concept | How it appears in I-SPY 2 |
|---|---|
| Diagnostic study design | Single-group study with diagnostic primary purpose |
| Binary reference outcome | pCR assessed at surgery |
| Continuous predictor | Changes in ADC and FTV |
| ROC analysis | Used to evaluate discrimination between pCR and non-pCR |
| AUC | Reported as the effect measure for the primary and secondary predictor analyses |
| Wald / z-test | Reported statistical method for the AUC analyses |
| Null AUC | 0.5 |
| Alternative design AUC | 0.65 |
| Superiority hypothesis | Used for the primary ADC and secondary FTV analyses |
| Equivalence hypothesis | Used for the optimized model vs ΔADC analysis |
| Equivalence margin | No equivalence margin was used in the reported model comparison |
| Multiple analyses | Seven statistical analyses posted |
18. Statistical Concepts in This Trial
Learn more about the methods used in this trial:
19. Related Statistical Calculators
20. Sources
- ClinicalTrials.gov: NCT01564368 — I-SPY 2.
- Linked publication: PubMed PMID 38180338.
- Linked publication: PubMed PMID 30179110.
- Linked publication: PubMed PMID 32548294.
- Linked publication: PubMed PMID 30350329.
- Linked publication: PubMed PMID 35314635.
Continue through the Clinical Biostats knowledge graph
Connect this trial's diagnostic endpoints and statistical methods to deeper tutorials, then to appropriate statistical calculators and analysis workflows.
21. Record Summary
I-SPY 2 provides a focused example of statistical evaluation of imaging-derived predictors against a binary pathological reference outcome. The completed study enrolled 406 participants in a single diffusion-weighted MRI arm. Its registered primary endpoint was pathologic complete response at surgery, defined using histopathologic analysis and the Residual Cancer Burden system.
The three primary analyses evaluated percentage change in ADC across early, mid-treatment, and post-treatment measurement windows. The corresponding AUC estimates were 0.53, 0.60, and 0.61, with reported P-values of 0.484, 0.017, and 0.013, respectively. Three secondary FTV analyses produced AUC estimates of 0.68, 0.63, and 0.68, each with a reported P-value of <0.001.
A separate predictive-model analysis compared an optimized model with ΔADC alone under an equivalence hypothesis and reported a P-value of 0.032. Crucially, the registry states that no equivalence margin was used, which limits how that result can be interpreted under conventional equivalence-testing logic.
The statistical story is therefore centered on ROC discrimination, AUC estimation, Wald / z-testing, one-sided confidence intervals, multiple related analyses, and the distinction between superiority-style discrimination testing and equivalence hypotheses. The study is not a randomized treatment comparison, so its statistical results should be understood as evidence about the predictive performance of the evaluated MRI measurements with respect to pCR.