BATTLE-2 And Biomarker Hypothesis Testing

Why BATTLE-2 Is a Biomarker-Testing Case Study

A biomarker-guided trial asks more than whether a treatment works on average. It asks whether treatment effects differ according to a measured biological feature. That distinction matters in advanced non-small-cell lung cancer (NSCLC), where a treatment might help patients with one molecular profile but not another. A promising pattern in a small subgroup, however, can arise through sampling variability, multiple comparisons, or features of the trial’s adaptive allocation.

BATTLE-2 was a randomized phase II study in previously treated advanced NSCLC. It evaluated four regimens: erlotinib; erlotinib plus MK-2206; MK-2206 plus selumetinib; and sorafenib. Its primary endpoint was disease control at 8 weeks. The design used biomarker information and accumulating outcomes in an adaptive randomization strategy. The scientific challenge was to learn whether biomarker-defined groups responded differently to the treatment options, not simply to identify which subgroup happened to have the largest observed response.

The broad lesson is that adaptive assignment and biomarker hypothesis testing answer related but distinct questions. Adaptation can change the probabilities with which participants receive treatments as evidence accumulates. It does not, on its own, establish that a biomarker is predictive or ensure that a subgroup estimate is precise.

What Counts as a Biomarker Hypothesis?

Definition: A prognostic biomarker is associated with a patient’s outcome regardless of treatment choice. A predictive biomarker identifies variation in the treatment effect: the difference between treatments depends on the biomarker value. Evidence that outcomes differ between biomarker groups under one treatment alone does not establish prediction.

For a binary endpoint such as disease control by 8 weeks, let \(B=1\) denote biomarker-positive status and \(B=0\) biomarker-negative status. Let \(A=1\) denote an experimental treatment and \(A=0\) its comparator. Write \(p_{ab}\) for the probability of disease control for treatment \(a\) and biomarker group \(b\). On the risk-difference scale, the treatment effect in the positive group is \(p_{11}-p_{01}\), and the effect in the negative group is \(p_{10}-p_{00}\). A predictive-biomarker hypothesis concerns the difference between these treatment effects:

\Delta_{\text{int}}=(p_{11}-p_{01})-(p_{10}-p_{00}).

The null hypothesis of no interaction on this scale is \(H_0:\Delta_{\text{int}}=0\); a two-sided alternative is \(H_a:\Delta_{\text{int}}\ne 0\). The scale must be stated: interaction on the risk-difference scale is not generally the same as interaction on the odds-ratio or hazard-ratio scale. A trial’s planned analysis should specify the endpoint, biomarker definition, treatment contrast, interaction scale, and direction or sidedness of the test before outcomes are examined.

With four treatment arms, there are multiple possible contrasts, and a biomarker may have more than two categories or be represented by a continuous measurement. Every additional treatment comparison, biomarker, cutoff, or endpoint creates further opportunities to observe an apparently favorable pattern by chance. A hypothesis selected after examining the results is exploratory unless its selection and testing are handled in the design and analysis.

From Adaptive Allocation to Valid Inference

In an adaptive randomized design, allocation probabilities may depend on information accrued during the trial. In BATTLE-2, biomarker information and emerging outcomes informed allocation among the treatment options. The resulting data are still generated by randomization, but the allocation mechanism is more complex than a fixed equal-probability assignment. In particular, the number of participants in a biomarker-treatment cell can be small or unbalanced, and later assignments may reflect earlier outcomes.

A standard two-sample test that assumes fixed, independent groups with a particular allocation scheme should not be applied automatically to adaptive-trial data. The analysis must respect the actual randomization and adaptation procedures, as well as any prespecified statistical model. For confirmatory claims, investigators should evaluate the operating characteristics of the full design: how often it would falsely identify a biomarker effect under relevant null scenarios, and how often it would detect meaningful interaction effects. Simulation can reproduce the adaptive assignment and analysis rules to assess these properties.

The following simple derivation illustrates the interaction test for a different, hypothetical setting: four independent binomial groups with equal fixed sample sizes. It clarifies what an interaction contrast measures, but it is not a substitute for the design-aware analysis of an adaptive trial.

Justification for the standard error: If the four cell estimates are independent sample proportions, the estimated variance of each proportion is approximately \(p_{ab}(1-p_{ab})/n\). The interaction contrast adds two proportions and subtracts two, so its estimated variance is the sum of their variances: each coefficient is \(+1\) or \(-1\), and its square is \(1\). Thus \(SE(\widehat{\Delta}_{\text{int}})\) is the square root of that sum. This approximation relies on independent binomial observations and sufficiently adequate cell sizes; adaptive assignment or very sparse cells can invalidate this simple calculation.

Worked Example: Testing a Hypothetical Biomarker Interaction

All numbers in this example are hypothetical and are not BATTLE-2 results. Suppose a fixed randomized study has 40 patients in each of four treatment-by-biomarker cells. Disease-control proportions are 0.60 for experimental treatment and biomarker-positive, 0.35 for comparator and biomarker-positive, 0.40 for experimental treatment and biomarker-negative, and 0.50 for comparator and biomarker-negative. Assume patients’ binary outcomes are independent within and across cells and each cell follows a binomial model.

The experimental-treatment difference is \(0.60-0.35=0.25\), or 25 percentage points, among biomarker-positive patients. Among biomarker-negative patients it is \(0.40-0.50=-0.10\), or a 10-percentage-point lower disease-control rate. The interaction estimate is therefore:

\widehat{\Delta}_{\text{int}}=(0.60-0.35)-(0.40-0.50)=0.35.

The estimated interaction standard error is:

SE=\sqrt{\frac{0.60(0.40)+0.35(0.65)+0.40(0.60)+0.50(0.50)}{40}} =\sqrt{\frac{0.9575}{40}} \approx 0.1547.

For a large-sample two-sided test of \(H_0:\Delta_{\text{int}}=0\), the test statistic is \(z=0.35/0.1547\approx 2.262\). The approximate two-sided normal p-value is \(0.02369\). At a prespecified \(\alpha=0.05\), this hypothetical test would reject the no-interaction null. The result is evidence of a difference in treatment effects on the risk-difference scale in this setting; it is not proof that the biomarker is clinically useful or that the effect will replicate.

Using the displayed standard error, an approximate 95% Wald confidence interval is:

0.35\pm 1.96(0.1547) =0.35\pm 0.303212 =(0.04679,\ 0.6532).

The interval is for the interaction in disease-control probability, expressed as a difference of percentage-point differences. Under repeated use of this method and its assumptions, 95% of intervals constructed this way would cover the true interaction. The positive interval suggests a larger treatment effect in the biomarker-positive group in these hypothetical data, but its wide upper bound also shows substantial uncertainty about the magnitude.

Small Subgroups and the Difference Between Signal and Validation

Adaptive allocation can make a trial more responsive to emerging evidence, but it cannot manufacture information in a rare biomarker subgroup. When few participants contribute to a cell, a single outcome can noticeably change the estimated response rate. Interaction estimates are especially demanding because they combine information from all four cells; precision can be poor even when the overall sample size looks substantial.

Worked Example: A Large-Looking Subgroup Difference With Wide Uncertainty

All numbers here are hypothetical. Suppose 20 biomarker-positive patients receive each of two treatments, and disease control occurs in 13 of 20 on the experimental treatment and 8 of 20 on the comparator. The estimated difference is \(13/20-8/20=0.25\), or 25 percentage points. Assume independent binomial outcomes and use a large-sample standard error for this difference.

SE=\sqrt{\frac{0.65(0.35)}{20}+\frac{0.40(0.60)}{20}} =\sqrt{0.023375} \approx 0.1529.

The test statistic for a zero difference is \(z=0.25/0.1529\approx 1.635\), giving an approximate two-sided p-value of 0.102. The approximate 95% interval is \(0.25\pm1.96(0.1529)\), or \((-0.0497,\ 0.5497)\). Although the observed difference is 25 percentage points, these data are compatible with a small disadvantage as well as a substantial benefit. Also, this within-subgroup comparison alone does not test whether treatment effects differ between biomarker groups.

A biomarker is not validated merely because one subgroup has a small p-value and another does not. “Significant here, not significant there” is not itself a test that the effects differ. The relevant evidence is a direct interaction estimate and its uncertainty, interpreted alongside the prespecified hypothesis, multiplicity, clinical importance, and design. For treatment selection, independent confirmation or other strong prespecified validation evidence is generally needed; a phase II signal is often a basis for further study, not a final clinical rule.

How to Read BATTLE-2 Without Overclaiming

BATTLE-2 is informative because it put biomarker hypotheses and adaptive randomization into a difficult clinical setting: patients with previously treated advanced NSCLC, several targeted regimens, and molecularly defined groups that could be small. Its results did not establish a validated biomarker rule that reliably selected one of the study treatments. In particular, an observed response pattern in a biomarker group must be distinguished from a confirmed treatment-by-biomarker interaction.

A careful reading separates three claims:

Overall treatment comparison: whether the outcomes differ across treatment arms in the study population.
Subgroup outcome description: whether outcomes observed in one biomarker group differ from those in another, without necessarily attributing the difference to treatment.
Predictive-biomarker evidence: whether the randomized treatment effect differs by biomarker status, assessed with an interaction contrast or an appropriate prespecified model.

These are not interchangeable. A biomarker can be prognostic without predicting differential treatment benefit. An overall treatment comparison can conceal genuine heterogeneity, while a subgroup-only comparison can suggest heterogeneity that is actually noise. The correct analysis depends on the scientific estimand and the randomized design.

Diagnostics, Alternatives, and Limitations

Before interpreting a biomarker finding, check whether the biomarker measurement and cutoff were specified before outcome analysis, whether each treatment-by-biomarker cell has adequate data, and whether the model and test reflect the randomization scheme. Examine estimates and intervals, not only p-values. Look for influential individual outcomes, sparse or empty cells, imbalance created by adaptation, and discrepancies between the planned analysis and the analysis actually performed.

The main alternatives answer different questions. An overall treatment comparison is usually more efficient for an average effect but does not establish that treatment effects vary by biomarker. Separate tests within each biomarker group are easy to explain but do not directly test an interaction and multiply the testing burden. A prespecified interaction test directly targets differential effects, but it may have low power when biomarker-positive patients are uncommon. A model using a continuous biomarker can avoid an arbitrary dichotomizing cutoff, but requires a prespecified functional form and careful interpretation.

For adaptive designs, fixed-design formulas can serve as intuition, not as an automatic analysis recipe. Allocation probabilities may depend on accumulated evidence, subgroup sample sizes may be unequal, and the adaptation itself can affect the distribution of estimates and test statistics. A design-aware analysis should use the protocol’s assignment and analysis rules, with simulation where needed to quantify error rates and power. Even a statistically persuasive interaction may have limited clinical relevance if the absolute benefit is small, the assay is unreliable, or the finding is not replicated.

Common Mistake

Pitfall: “The experimental treatment had a low p-value in biomarker-positive patients but not in biomarker-negative patients, so the biomarker predicts benefit.” This reasoning compares two significance labels rather than testing whether the treatment effects differ. The correction is to estimate the treatment-by-biomarker interaction directly, report its uncertainty and scale, and account for multiplicity and the adaptive design. A subgroup signal is hypothesis-generating until the evidence supports a reproducible, clinically meaningful treatment difference.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

In a binary disease-control analysis, what does a treatment-by-biomarker interaction test that an overall treatment comparison does not?
Why is “significant in one biomarker group, not significant in the other” not a valid interaction test?
Suppose the treatment effects are 0.18 in the biomarker-positive group and 0.16 in the biomarker-negative group on the risk-difference scale. What is the estimated interaction, and what does it indicate?
Name two reasons a simple fixed-allocation standard error may be inappropriate for an adaptive randomized trial.

Solutions. (1) It tests whether the treatment effect differs by biomarker status, rather than whether there is an average treatment difference. (2) The two subgroup tests have separate uncertainties; differing significance labels do not test the difference between effects. A direct interaction contrast is required. (3) The interaction is \(0.18-0.16=0.02\), a 2-percentage-point difference in treatment effects. Its uncertainty would still be needed to judge evidence for interaction. (4) Adaptive assignment may produce unequal, outcome-dependent allocation probabilities, and small or sparse biomarker-treatment cells may undermine the approximations. The analysis must reflect the actual design.