This page provides an independent statistical analysis and educational interpretation of publicly reported results. ClinicalTrials.gov provides the official trial registry record.
1. Trial at a Glance
CheckMate-037 was a completed, randomized phase 3 parallel trial in participants with unresectable or metastatic melanoma. The trial compared BMS-936558 with Investigator's Choice of dacarbazine or carboplatin plus paclitaxel.
| Feature | CheckMate-037 |
|---|---|
| Trial name | CheckMate-037 |
| ClinicalTrials.gov identifier | NCT01721746 |
| Phase | Phase 3 |
| Status | COMPLETED |
| Condition | Unresectable or Metastatic Melanoma |
| Allocation | RANDOMIZED |
| Design model | PARALLEL |
| Masking | NONE |
| Primary purpose | TREATMENT |
| Enrollment | 405.0 |
| Lead sponsor | Bristol-Myers Squibb |
| Sponsor type | INDUSTRY |
| Start | 2012-12-21 |
| Primary completion | 2016-02-16 |
2. Clinical Question
The registered study question compared BMS-936558 with Investigator's Choice in advanced melanoma patients that had progressed following anti-CTLA-4 therapy. The two primary endpoints were objective response rate (ORR) and overall survival (OS).
Population
Advanced melanoma patients with unresectable or metastatic melanoma that had progressed following anti-CTLA-4 therapy.
Intervention
BMS-936558. The posted OS analysis identifies the treatment as nivolumab 3 mg/kg (IV).
Comparator
Investigator's Choice of dacarbazine or carboplatin plus paclitaxel.
Primary question
How does BMS-936558 compare with Investigator's Choice with respect to ORR and OS?
3. Trial Design
BMS-936558 / Nivolumab
- BMS-936558
- Posted OS analysis identifies the treatment as nivolumab 3 mg/kg (IV)
- Biological intervention
Investigator's Choice
- Dacarbazine
- Or carboplatin plus paclitaxel
- Drug interventions
4. Endpoints
The registry identifies two primary endpoints: objective response rate, a binary endpoint, and overall survival, a time-to-event endpoint. The registry also contains posted analyses for progression-free survival and PD-L1-related analyses.
| Endpoint | Type | Registered definition / time frame |
|---|---|---|
| Objective Response Rate (ORR) | Binary | Objective response rate (ORR) per Independent Review Committee (IRC) is defined as the number of participants with a best overall response (BOR) of complete response (CR) or partial response (PR) divided by the number of randomized participants using RECIST 1.1. Time frame: From date of randomization to the date of objectively documented progression, date of death, or the date of subsequent therapy (Up to approximately 38 months). |
| Overall Survival (OS) | Time-to-event | Overall Survival (OS) was defined the time between the date of randomization to the date of death. For participants without documentation of death, OS was censored on the last date the participant was known to be alive. Unit of measure (months) is the median survival time. Time frame: Up to 96 months. |
Posted secondary endpoints with statistical analyses
| Endpoint | Time frame | Effect measure |
|---|---|---|
| Progression Free Survival (PFS) | From the date of randomization to the date of the first documented progression or death (Up to approximately 38 months) | Hazard ratio |
| Objective Response Rate (ORR) by Baseline PD-L1 Expression | From date of randomization to the date of objectively documented progression or the date of subsequent therapy (Up to approximately 38 months) | Odds ratio |
| Overall Survival (OS) by PD-L1 Positive | Up to 96 months | Hazard ratio |
| Overall Survival (OS) by PD-L1 Negative | Up to 96 months | Hazard ratio |
5. Statistical Methodology
Overall survival: stratified Cox proportional-hazards model
The posted primary OS analysis used a stratified Cox proportional-hazards model with treatment group as a single covariate. The analysis was based on all randomized participants and compared nivolumab with Investigator's Choice.
The analysis notes specify a stratified Cox proportional-hazards model using BRAF status, prior anti-CTLA-4 benefit, and PD-L1 status from the IVRS source as stratification factors.
Hazard ratio
The primary OS effect measure was a hazard ratio. The reported estimate compares the estimated hazard of death under nivolumab with the estimated hazard under Investigator's Choice. An HR below 1 corresponds to a lower estimated instantaneous event rate under nivolumab within the fitted model.
The hazard ratio is a relative, model-based measure. It is not an absolute survival probability, a median survival difference, or the fraction of participants who benefit.
Kaplan-Meier estimation
OS and PFS are time-to-event endpoints. A Kaplan-Meier analysis would estimate the event-free or survival distribution while retaining information from participants whose outcomes are censored before the event occurs. The registry's OS definition explicitly specifies censoring at the last date a participant was known to be alive when death was not documented.
Stratified analysis
Stratification allows the treatment comparison to account for specified categorical factors without requiring those factors to have a single common regression coefficient in the same way as ordinary covariates. For the posted primary OS analysis, the stratification factors were BRAF status, prior anti-CTLA-4 benefit, and PD-L1 status.
Binary response analysis
ORR is defined as the proportion of randomized participants whose best overall response is complete response or partial response according to RECIST 1.1 and Independent Review Committee assessment. The posted trial data identify ORR as a primary binary endpoint, but the ClinicalTrials.gov record does not provide a formal primary ORR comparison or an ORR effect estimate.
6. Primary Results
Overall Survival
Hazard ratio for death
95% CI: 0.68–1.08 · Two-sided 95% confidence interval
Analysis population: all randomized participants
The posted OS analysis compared nivolumab 3 mg/kg (IV) with Investigator's Choice of dacarbazine or carboplatin plus paclitaxel using a stratified Cox proportional-hazards model. The hypothesis type recorded for the analysis was superiority.
An OS hazard ratio of 0.86 means that, under the fitted Cox model, the estimated instantaneous hazard of death for nivolumab was 0.86 times that for Investigator's Choice. Expressed as a simple relative interpretation, 0.86 corresponds to an estimated 14% lower hazard, not a 14% lower probability of death.
The estimate does not mean that 14% of participants benefited, that survival increased by 14%, or that every participant experienced the same reduction in hazard. A hazard ratio describes a relative event rate within a time-to-event model.
The two-sided 95% CI of 0.68–1.08 describes the statistical uncertainty around the estimated hazard ratio under the analysis framework. Because the interval spans 1, the estimate is compatible with both a lower and a higher hazard under nivolumab relative to Investigator's Choice within the range represented by the interval.
The p-value is not reported in the ClinicalTrials.gov record. A p-value, when available, would address evidence against a specified null hypothesis; it would not measure the magnitude or clinical importance of the hazard ratio.
Interpretation also depends on the Cox model and its proportional-hazards assumption. If hazards are not approximately proportional over time, a single hazard ratio can compress a more complex time-varying treatment effect into one summary measure. Censoring and the use of randomized participants also matter to the interpretation of the estimate.
Objective Response Rate
The registry defines ORR as the number of randomized participants with a best overall response of complete response or partial response divided by the number of randomized participants, using RECIST 1.1 and Independent Review Committee assessment. For a randomized binary endpoint of this type, an analysis would ordinarily compare the response proportions between the two treatment groups and quantify uncertainty with a confidence interval around an appropriate effect measure.
The absence of a registry-reported formal statistical comparison does not mean that the endpoint is uninformative. It means that the available trial data do not provide the numerical ingredients needed to characterize the randomized ORR treatment effect without adding information from another source.
ORR is also conceptually different from OS. ORR summarizes whether a prespecified response state was achieved; OS measures time from randomization to death and incorporates censoring for participants not known to have died.
7. Secondary Results
Progression Free Survival
Hazard ratio for progression or death
95.1% CI: 0.78–1.36 · Two-sided confidence interval
Analysis population: all randomized participants
The secondary PFS analysis compared nivolumab with Investigator's Choice using a stratified Cox proportional-hazards model. The analysis time frame was from randomization to the first documented progression or death, up to approximately 38 months.
An HR of 1.03 means that the estimated instantaneous hazard of progression or death under nivolumab was 1.03 times the corresponding estimated hazard under Investigator's Choice in the fitted model.
The 95.1% CI of 0.78–1.36 is relatively broad around the point estimate and includes 1. The interval therefore represents uncertainty that encompasses both a lower and a higher hazard under nivolumab relative to the comparator.
The ClinicalTrials.gov record does not report a p-value for this PFS analysis. A p-value would not itself describe the size of the treatment effect; the hazard ratio and its confidence interval provide the principal quantitative description reported here.
As with the OS model, interpretation of a Cox hazard ratio requires attention to the proportional-hazards assumption, censoring, and the analysis population. The endpoint is also a composite time-to-event endpoint because the first documented progression or death can define the event.
ORR by Baseline PD-L1 Expression
| Baseline PD-L1 expression | Odds ratio | 95% CI |
|---|---|---|
| <5% PD-L1 expression | 1.13 | 0.44–3.16 |
| >=5% PD-L1 expression | 5.49 | 1.92–19.08 |
Both analyses used the population of all PD-L1 evaluable participants for ORR. The effect measure was the odds ratio, described in the analysis notes as the ratio of nivolumab over Investigator's Choice. The ClinicalTrials.gov record does not report the statistical method used to generate these odds ratios.
An odds ratio compares the odds of the binary outcome between treatment groups. An OR of 1.13 indicates that the estimated odds of response under nivolumab were 1.13 times those under Investigator's Choice in the <5% PD-L1 expression analysis.
An OR of 5.49 indicates that the estimated odds of response under nivolumab were 5.49 times those under Investigator's Choice in the >=5% PD-L1 expression analysis. This is an odds ratio, not a risk ratio and not a 5.49-fold increase in the response probability.
The confidence intervals show substantial uncertainty, particularly for the <5% analysis, where the 95% CI is 0.44–3.16. The >=5% analysis has a 95% CI of 1.92–19.08, which is also wide.
These are subgroup analyses based on PD-L1 evaluable participants rather than the complete randomized population. Differences between subgroup estimates should not automatically be interpreted as evidence that treatment effects differ by PD-L1 expression. A formal interaction analysis would ordinarily be needed to assess effect modification.
Overall Survival by PD-L1 Status
| PD-L1 subgroup | OS HR | 95% CI | Model note |
|---|---|---|---|
| PD-L1 positive | 0.71 | 0.50–1.01 | Unstratified Cox proportional-hazards model |
| PD-L1 negative | 1.03 | 0.75–1.41 | Unstratified Cox proportional-hazards model |
The PD-L1 positive analysis used all PD-L1 positive evaluable participants for OS; the PD-L1 negative analysis used all PD-L1 negative evaluable participants for OS. Both analyses used a two-sided 95% confidence interval and were recorded under a superiority hypothesis.
The PD-L1-positive OS estimate of 0.71 corresponds to an estimated hazard under nivolumab that was 0.71 times that under Investigator's Choice in that subgroup. The 95% CI of 0.50–1.01 indicates uncertainty around the subgroup estimate.
The PD-L1-negative OS estimate of 1.03 corresponds to an estimated hazard under nivolumab that was 1.03 times that under Investigator's Choice in that subgroup. Its 95% CI of 0.75–1.41 also includes 1.
A difference between these two point estimates should not be treated as proof that PD-L1 status modifies the treatment effect. Subgroup comparisons require consideration of interaction and the reduced precision that accompanies analysis within smaller populations.
8. Safety Results
The ClinicalTrials.gov record reports serious adverse events by treatment arm as affected participants divided by participants at risk. The registry-derived record does not provide a separate serious-adverse-event statistical comparison.
| Arm | Serious adverse events, affected / at risk |
|---|---|
| Nivolumab | 191/268 |
| Investigator's Choice (Dacarbazine or Carboplatin+Paclitaxel) | 37/102 |
These figures describe the number affected relative to the number at risk in each reported arm. They should not be interpreted as a randomized efficacy estimate or combined with the OS hazard ratio. Safety and efficacy address different outcomes and can involve different analysis populations.
9. Analysis Populations and Stratification
| Analysis | Population specified in the posted analysis | Role |
|---|---|---|
| Primary OS | All randomized participants | Primary time-to-event analysis |
| Secondary PFS | All randomized participants | Secondary time-to-event analysis |
| ORR by PD-L1 expression | All PD-L1 evaluable participants for ORR | Secondary binary endpoint analysis |
| OS by PD-L1 positive | All PD-L1 positive evaluable participants for OS | Secondary subgroup survival analysis |
| OS by PD-L1 negative | All PD-L1 negative evaluable participants for OS | Secondary subgroup survival analysis |
The primary OS model was stratified by BRAF status, prior anti-CTLA-4 benefit, and PD-L1 status. The analysis notes identify treatment group as a single covariate in the stratified Cox model.
Conceptually, a stratified Cox model permits different baseline hazards across the specified strata while estimating a common treatment coefficient across those strata.
10. Statistical Methods Explained
Why was a Cox proportional-hazards model used?
OS and PFS are time-to-event outcomes, so the analysis must account not only for whether an event occurred but also for when it occurred and for censoring. The Cox model provides a way to estimate a relative hazard between randomized treatment groups without requiring a particular parametric survival distribution.
What does an OS hazard ratio of 0.86 mean?
It means that the fitted model estimates the instantaneous hazard of death under nivolumab at 0.86 times the hazard under Investigator's Choice. It does not mean that 14% fewer participants died, nor does it translate directly into a 14% increase in survival time.
Why is the confidence interval important?
The point estimate is only one summary of the treatment comparison. The 95% CI of 0.68–1.08 communicates the uncertainty around the OS hazard-ratio estimate. Because it includes 1, the data represented by this interval are compatible with a lower or higher hazard under nivolumab relative to Investigator's Choice.
Why does PD-L1 subgroup analysis require caution?
Subgroup analyses partition the study population into smaller groups. Smaller analysis populations generally provide less precise estimates. More importantly, seeing different estimates in two subgroups does not by itself establish that the treatment effect differs between them. Effect modification is normally assessed using an interaction test or another prespecified comparison of treatment effects.
What is the difference between an odds ratio and a hazard ratio?
An odds ratio is an effect measure for a binary outcome such as response. A hazard ratio is a time-to-event measure describing the relative instantaneous event rate under a survival model. They answer different statistical questions and should not be interpreted interchangeably.
Why is censoring important for OS?
The registry defines OS as time from randomization to death and specifies that participants without documentation of death are censored at the last date they were known to be alive. Censoring allows such participants to contribute the follow-up information that is available without treating an unobserved death as if it had occurred at the censoring date.
Why should the primary ORR result not be reconstructed here?
The ClinicalTrials.gov record identifies ORR as a primary binary endpoint and says results were posted, but it does not provide a formal ORR statistical analysis, effect estimate, confidence interval, or p-value. Reconstructing those quantities would require adding information beyond the ClinicalTrials.gov record. A rigorous trial-results page should distinguish a posted endpoint from a formally reported statistical comparison.
11. Understanding the Primary OS Result
Relative effect
HR 0.86 is a relative model-based comparison of the estimated instantaneous hazard of death between randomized treatment groups.
Precision
The 95% CI of 0.68–1.08 quantifies uncertainty around the OS hazard-ratio estimate under the specified model.
Analysis population
The posted primary OS analysis used all randomized participants, preserving treatment assignment as the basis of the comparison.
Stratification
The Cox model was stratified by BRAF status, prior anti-CTLA-4 benefit, and PD-L1 status.
The most important statistical distinction is between the magnitude of an estimate and the uncertainty around that estimate. An HR of 0.86 is the point estimate. The interval 0.68–1.08 describes the range of values represented by the specified 95% confidence procedure. Neither quantity alone describes individual patient outcomes.
12. PD-L1 Analyses: How to Read the Numbers
The trial data contain both binary ORR analyses and time-to-event OS analyses divided by baseline PD-L1 status. This creates a useful teaching example because the effect measures are different even though the treatment comparison is the same.
| Analysis | PD-L1 group | Effect |
|---|---|---|
| ORR | <5% expression | OR 1.13 (95% CI 0.44–3.16) |
| ORR | >=5% expression | OR 5.49 (95% CI 1.92–19.08) |
| OS | Positive | HR 0.71 (95% CI 0.50–1.01) |
| OS | Negative | HR 1.03 (95% CI 0.75–1.41) |
The estimates should be read within their respective endpoint definitions. The ORRs are binary response comparisons among PD-L1 evaluable participants, while the OS estimates describe time to death among PD-L1 evaluable participants. Comparing the numerical size of an OR with the numerical size of an HR would not be statistically meaningful.
13. Multiplicity and Multiple Analyses
the ClinicalTrials.gov record identifies two primary endpoints and six posted statistical analyses overall, with one primary-endpoint analysis and one primary analysis containing an estimate and confidence interval. The primary endpoints were ORR and OS, while the posted statistical analyses also cover PFS and PD-L1-related analyses.
| Endpoint / analysis | Role in the ClinicalTrials.gov record | Statistical measure |
|---|---|---|
| Objective Response Rate | Primary | Binary; formal statistical analysis not reported |
| Overall Survival | Primary | HR 0.86, 95% CI 0.68–1.08 |
| Progression Free Survival | Secondary | HR 1.03, 95.1% CI 0.78–1.36 |
| ORR by PD-L1 expression | Secondary | OR 1.13 / 5.49 |
| OS by PD-L1 positive | Secondary | HR 0.71, 95% CI 0.50–1.01 |
| OS by PD-L1 negative | Secondary | HR 1.03, 95% CI 0.75–1.41 |
Multiple endpoints and subgroup analyses create a distinction between descriptive estimates and confirmatory hypothesis testing. The ClinicalTrials.gov record does not provide an alpha-spending scheme, multiplicity-adjustment procedure, interim-analysis plan, or p-values for the six posted analyses. Those features therefore should not be inferred.
14. What the Hazard Ratio Does — and Does Not — Mean
The primary OS HR of 0.86 means that the fitted model estimated the instantaneous hazard of death under nivolumab at 0.86 times the corresponding hazard under Investigator's Choice.
It does not mean that 14% of participants survived, that 14% of participants were cured, or that every participant experienced a 14% reduction in their individual probability of death.
The 95% CI of 0.68–1.08 expresses uncertainty around the estimated OS hazard ratio. It is not a prediction interval for individual patients and does not state that 95% of future patients will have hazard ratios inside this interval.
No p-value for the primary OS analysis is contained in the ClinicalTrials.gov record. Even when a p-value is reported in a clinical trial, it should be interpreted as evidence relative to a specified null hypothesis rather than as a measure of treatment-effect size.
15. Design Features That Affect Interpretation
Randomization
The allocation was randomized, creating the structural basis for comparing outcomes according to assigned treatment.
Parallel design
The trial used a parallel design with two arms rather than a crossover or factorial structure.
Open-label design
Masking was listed as NONE. This is relevant when considering outcomes that can be influenced by knowledge of treatment assignment.
Stratified survival model
The primary OS analysis accounted for BRAF status, prior anti-CTLA-4 benefit, and PD-L1 status through model stratification.
The ClinicalTrials.gov record does not identify a non-inferiority margin, factorial design, crossover procedure, Bayesian analysis, interim-analysis scheme, imputation method, or missing-data strategy. These topics are therefore not treated as features of the CheckMate-037 analysis on this page.
16. Trial Timeline
Trial start
The ClinicalTrials.gov record lists 2012-12-21 as the study start date.
Randomized comparative design
the ClinicalTrials.gov record identifies a phase 3 randomized, parallel study with two arms comparing BMS-936558 with Investigator's Choice.
Primary completion
The ClinicalTrials.gov profile lists 2016-02-16 as the primary completion date.
Results posted
the ClinicalTrials.gov record identifies the study as completed, with results posted and six statistical analyses posted.
17. Why This Trial Matters Statistically
CheckMate-037 is a useful teaching case because the ClinicalTrials.gov record combine randomized treatment comparison, binary response assessment, time-to-event endpoints, stratified Cox modeling, hazard ratios, odds ratios, and biomarker-defined subgroup analyses.
| Concept | How it appears in CheckMate-037 |
|---|---|
| Randomization | The trial used RANDOMIZED allocation. |
| Parallel design | The design model was PARALLEL with two arms. |
| Binary endpoint | ORR was a registered primary endpoint. |
| Time-to-event endpoint | OS was a registered primary endpoint and PFS was a posted secondary endpoint. |
| Kaplan-Meier framework | Time-to-event endpoints are naturally interpreted using survival-function estimation with censoring. |
| Hazard ratio | OS and PFS treatment effects were reported as hazard ratios. |
| Cox model | The primary OS and secondary PFS analyses used Cox proportional-hazards models. |
| Stratified analysis | The primary OS Cox model was stratified by BRAF status, prior anti-CTLA-4 benefit, and PD-L1 status. |
| Odds ratio | PD-L1-defined ORR analyses reported odds ratios. |
| Subgroup analysis | OS and ORR were analyzed according to PD-L1 status or expression. |
| Analysis populations | Primary OS and PFS used all randomized participants, while PD-L1 analyses used evaluable subgroup populations. |
18. Important Limitations and Interpretation Issues
- Incomplete primary ORR analysis: ORR is a registered primary endpoint and results are posted, but the ClinicalTrials.gov record does not provide a formal ORR comparison, effect estimate, confidence interval, or p-value.
- Hazard-ratio interpretation: Cox HRs are model-based relative measures and depend on the assumptions underlying the model, including the proportional-hazards framework.
- Confidence-interval width: subgroup estimates can be substantially less precise than the primary randomized comparison because they are based on restricted analysis populations.
- Subgroup heterogeneity: different estimates in PD-L1 subgroups do not, by themselves, demonstrate treatment-effect modification.
- Analysis populations: the primary OS analysis used all randomized participants, whereas the PD-L1 analyses used PD-L1 evaluable populations. These populations should not be treated as identical.
- Safety denominators: the registry-reported serious-adverse-event data use 268 and 102 as the respective at-risk denominators, while the overall enrollment is 405. These should not be silently substituted for one another.
- Missing design details: the ClinicalTrials.gov record does not report a non-inferiority margin, Bayesian method, interim-analysis plan, imputation strategy, or explicit multiplicity-adjustment procedure.
- Open-label design: masking was listed as NONE. Knowledge of treatment assignment can be relevant to interpretation of outcomes susceptible to assessment or management influences.
19. A Statistical Reading of the Entire Evidence Set
The six registry-reported statistical analyses illustrate why clinical-trial interpretation should not reduce a study to one number. The primary OS analysis gives a hazard ratio of 0.86 with a 95% CI of 0.68–1.08. The secondary PFS analysis gives an HR of 1.03 with a 95.1% CI of 0.78–1.36. The PD-L1-specific ORR analyses use odds ratios, while the PD-L1-specific OS analyses return to hazard ratios.
Those estimates are not directly interchangeable. ORR is binary; OS and PFS are time-to-event outcomes. The primary OS estimate comes from all randomized participants and a stratified Cox model, whereas the PD-L1 analyses restrict attention to biomarker-evaluable populations. Consequently, a statistically careful reading keeps the endpoint, population, effect measure, model, and confidence interval attached to every number.
This structure prevents a hazard ratio from being mistaken for an odds ratio, a subgroup estimate from being treated as the primary analysis, or a confidence interval from being interpreted as an individual-level prediction.
20. Sources
- ClinicalTrials.gov: NCT01721746.
- Linked PubMed publication: PubMed 28671856.
- Linked PubMed publication: PubMed 25795410.
Continue through the Clinical Biostats statistical pathway
Use the related tutorials and calculators to explore the statistical methods that appear in randomized oncology trials, including survival analysis, confidence intervals, Cox models, hazard ratios, odds ratios, randomization, and stratified analysis.
21. Related Tutorials
Learn more about the methods used in this trial:
22. Related Calculators
23. Record Summary
CheckMate-037 provides a compact example of several important clinical-trial statistical concepts. The study was a randomized phase 3 parallel trial with two arms and no masking, enrolling 405 participants. Its registered primary endpoints were ORR, a binary response endpoint, and OS, a time-to-event endpoint. The registry-reported formal primary analysis is the OS comparison, which used all randomized participants and a stratified Cox proportional-hazards model with BRAF status, prior anti-CTLA-4 benefit, and PD-L1 status as stratification factors.
The primary OS hazard ratio was 0.86 with a two-sided 95% CI of 0.68–1.08. Secondary analyses included PFS, ORR by baseline PD-L1 expression, and OS according to PD-L1-positive or PD-L1-negative status. These analyses demonstrate why treatment effects must always be interpreted together with their endpoint definitions, analysis populations, statistical models, and uncertainty intervals.