← Clinical Trials
Unresectable Melanoma Phase 2 Binary Endpoint NCT01927419

Study of Nivolumab (BMS-936558) Plus Ipilimumab Compared With Ipilimumab Alone in the Treatment of Previously Untreated, Unresectable, or Metastatic Melanoma: Complete Statistical Analysis

This randomized phase 2 trial compared nivolumab plus ipilimumab with ipilimumab alone in previously untreated, unresectable or metastatic melanoma. The primary result concerns investigator-assessed objective response among participants with BRAF wild-type melanoma.

Trial start: 2013-08-23  ·  Enrollment: 142  ·  Sponsor: Bristol-Myers Squibb
Scope of this record

This page provides an independent statistical analysis and educational interpretation of publicly reported results. ClinicalTrials.gov provides the official trial registry record. Numerical trial results on this page are restricted to the results posted in the ClinicalTrials.gov record; calculations added by Clinical Biostats are labelled as such.

1. Trial at a Glance

This page covers the registry results for Study of Nivolumab (BMS-936558) Plus Ipilimumab Compared With Ipilimumab Alone in the Treatment of Previously Untreated, Unresectable, or Metastatic Melanoma (NCT01927419), a randomized phase 2 study. The primary outcome is Objective Response Rate (ORR) - BRAF Wild-type (WT) Participants.

142
Enrollment
Phase 2
2
Arms
In the primary outcome
-49.5 pts
Risk difference
Ipilimumab
-64.5 to -34.4 pts
95% CI
Two-sided
FeatureRegistry value
ClinicalTrials.gov identifierNCT01927419
PhasePhase 2
ConditionUnresectable Melanoma, Metastatic Melanoma
AllocationRandomized
Design modelParallel
MaskingQuadruple
Enrollment142
Arms in the primary outcome2
Lead sponsorBristol-Myers Squibb
Results postedYes

2. Clinical Question

Population

Participants with previously untreated, unresectable or metastatic melanoma. The primary endpoint population was all randomized participants with BRAF wild-type melanoma; the registry record also reports results for BRAF mutant participants.

Intervention

Nivolumab plus ipilimumab. Participants received nivolumab 1 mg/kg and ipilimumab 3 mg/kg intravenously every 3 weeks for 4 doses, followed by nivolumab 3 mg/kg intravenously every 2 weeks until a listed stopping condition.

Comparator

Ipilimumab alone, with placebo-matching nivolumab during the combination-treatment schedule and afterward. Ipilimumab was given at 3 mg/kg intravenously every 3 weeks for 4 doses.

Primary question

Among BRAF wild-type participants, what percentage had a best overall response of complete or partial response, as assessed by investigators using RECIST 1.1? The registry record compares that percentage between the nivolumab-plus-ipilimumab arm and the ipilimumab arm.

The central statistical question is about the difference in observed response proportions between randomized arms, rather than a comparison of response duration or quality of life. Response was assessed beginning 12 weeks after randomization, at the stated intervals, and then until disease progression. The record gives arm-specific response percentages, confidence intervals, and a reported sponsor analysis. It also includes a simple unadjusted recalculation based on the posted counts.

Randomization supports interpreting the arm contrast as a comparison of assigned treatment strategies in the trial population, subject to the analysis and missing-data details. It does not establish that every participant would have the same outcome, nor does a response percentage describe how long a response lasted. The registry record does not report a complete statistical analysis plan or further details about handling missing response assessments.

3. Trial Design

The study was a randomized, quadruple-masked, parallel-group phase 2 trial sponsored by Bristol-Myers Squibb. The registry record lists enrollment of 142 and a completed status. It lists a study start date of 2013-08-23 and a primary completion date of 2014-07-24. The condition entries are unresectable melanoma and metastatic melanoma.

Participants were assigned to nivolumab plus ipilimumab or ipilimumab alone. In the combination arm, part 1 used nivolumab 1 mg/kg plus ipilimumab 3 mg/kg intravenously every 3 weeks for 4 doses. Part 2 used nivolumab 3 mg/kg intravenously every 2 weeks until documented disease progression, toxicity, withdrawal of consent, or study completion. In the comparator arm, placebo-matching nivolumab was paired with ipilimumab during part 1, followed by placebo-matching nivolumab on the part 2 schedule described in the record. The registry lists quadruple masking but does not describe all roles covered by that label in the record.

The primary outcome was objective response rate among BRAF wild-type participants. The corresponding arm denominators were 73 for nivolumab plus ipilimumab and 37 for ipilimumab. A separate BRAF mutant response analysis used denominators of 22 and 10, respectively. The record also lists progression-free survival in each BRAF subgroup and change from baseline in EORTC Overall Quality of Life C30 score as secondary outcomes. Four secondary outcomes are posted, and the record contains six analyses.

The registry record does not report the original sample-size justification, the allocation ratio planned by the protocol, or the trial’s original power assumptions. The participant totals for individual analyses should be read as their analysis populations, rather than assumed to equal the overall enrollment.

4. Participant Flow and Accounting

The registry reports how participants moved through each stage of the study. All counts below are exactly as posted; missing counts are not inferred from other stages.

Before assignment: 142 participants were randomized, and 140 participants received treatment.

Pre-Treatment Period

StageNivolumab + IpilimumabIpilimumab
STARTED9547
COMPLETED9446
NOT COMPLETED11

Reasons for not completing this period:

ReasonNivolumab + IpilimumabIpilimumab
Adverse event unrelated to study drug01
Participants no longer meeting study criteria10

Treatment Period

StageNivolumab + IpilimumabIpilimumab
STARTED9446
COMPLETED00
NOT COMPLETED9446

Reasons for not completing this period:

ReasonNivolumab + IpilimumabIpilimumab
Disease progression1720
Study drug toxicity4810
Death01
Adverse event unrelated to study drug63
Participant request to discontinue124
Withdrawal by Subject11
Maximum Clinical Benefit62
Other reasons34
Not Reported11

Analysis population for the primary outcome, as defined in the registry: All randomized BRAF wild-type participants .

5. Baseline Characteristics

The table reproduces the baseline characteristics posted in the registry, which describe the participants analysed at baseline and help show whether the randomized groups were comparable.

Population: All randomized participants

CharacteristicUnitNivolumab + IpilimumabIpilimumabTotal
Participants analysed9547142
Age, ContinuousYears63.3 (SD 11.0)64.5 (SD 10.2)63.7 (SD 10.7)
Age, Customized: Younger than 65 yearsParticipants482068
Age, Customized: 65 years and older to younger than 75 yearsParticipants352257
Age, Customized: 75 years and olderParticipants12517
Sex: Female, Male: FemaleParticipants321547
Sex: Female, Male: MaleParticipants633295
Ethnicity (NIH/OMB): Hispanic or LatinoParticipants101
Ethnicity (NIH/OMB): Not Hispanic or LatinoParticipants8243125
Ethnicity (NIH/OMB): Unknown or Not ReportedParticipants12416
Race (NIH/OMB): American Indian or Alaska NativeParticipants000
Race (NIH/OMB): AsianParticipants101
Race (NIH/OMB): Native Hawaiian or Other Pacific IslanderParticipants000
Race (NIH/OMB): Black or African AmericanParticipants000
Race (NIH/OMB): WhiteParticipants9247139
Race (NIH/OMB): More than one raceParticipants000
Race (NIH/OMB): Unknown or Not ReportedParticipants202

6. Primary Endpoint

The registry lists the primary outcome as Objective Response Rate (ORR) - BRAF Wild-type (WT) Participants, measured over From 12 weeks after Randomization, assessed every 6 weeks up to Week 49 of study treatment and then every 12 weeks until disease progression (up to approximately 76 months).

ElementRegistry entry
Result typeNumber
Measure of spread95% Confidence Interval
UnitPercentage of participants
Endpoint classificationBinary Endpoint

The primary endpoint is objective response rate, defined as the percentage of participants whose best overall response was complete response or partial response. Investigators assessed response using RECIST 1.1 criteria. In the registry description, complete response means that target and nontarget lesions have disappeared and selected lymph nodes have returned to normal size; partial response means at least a 30% decrease in the sum of the longest diameters of target lesions relative to baseline.

Response rate reduces each participant’s tumor-response assessment to a yes-or-no outcome for this analysis: whether the participant met the complete- or partial-response definition. The arm-level percentage then expresses how common that outcome was in each analysis population. This endpoint is used to compare the frequency of qualifying responses between randomized groups; it does not, by itself, describe response duration, time to progression, survival, or quality of life.

The specified primary population was all randomized BRAF wild-type participants. Assessments began 12 weeks after randomization, continued every 6 weeks up to Week 49 of study treatment, and then every 12 weeks until disease progression, with the timeframe listed as up to approximately 76 months. The registry record identifies investigator assessment and the response criteria, but does not report further detail on adjudication, missing assessments, or the precise analysis rules for participants without an evaluable response.

7. Results Posted in the Registry

Risk difference: Ipilimumab

-49.5

95% CI: -64.5 to -34.4 percentage points

Recalculated from the posted summary statistics

Primary outcome by arm, with 95% confidence intervals
Primary outcome by arm with 95% confidence intervals0.015.530.946.461.977.4Nivolumab + Ipilimumab (n=73)Ipilimumab (n=37)% of participants (95% CI)
Treatment effect versus the control arm
Forest plot of risk difference (percentage points) vs control (95% CI)Ipilimumab-70-55-40-25-9.805.16risk difference (percentage points) vs control (95% CI)

8. Secondary Efficacy Endpoints

The registry also posts results for secondary outcomes. For each outcome below, the endpoint type, the test applied to the posted summary statistics, the result and its interpretation are given in turn. Secondary endpoints are usually not adjusted for multiple testing, so they support rather than confirm the primary finding.

Secondary outcomeTypeEffect measureEstimate (95% CI)Planning size, 80% power
Progression-Free Survival (PFS) - BRAF Wild-type (WT) ParticipantsTime-to-Event AnalysisHazard ratio0.36 (0.21 to 0.59)31 events
Objective Response Rate (ORR) - BRAF Mutant ParticipantsBinary EndpointRisk difference-44.5 pts (-72.4 to -16.6)17 per arm
Progression-Free Survival (PFS) - BRAF Mutant ParticipantsTime-to-Event AnalysisHazard ratio0.36 (0.14 to 0.97)31 events

Progression-Free Survival (PFS) - BRAF Wild-type (WT) Participants

Treatment effect versus the control arm
Forest plot of hazard ratio (as reported) vs control (95% CI)Nivolumab + Ipilimumab vs Ipi…0.190.270.380.550.791.13hazard ratio (as reported) vs control (95% CI)

Objective Response Rate (ORR) - BRAF Mutant Participants

Treatment effect versus the control arm
Forest plot of risk difference (percentage points) vs control (95% CI)Ipilimumab-78-61-45-28-115.80risk difference (percentage points) vs control (95% CI)

Progression-Free Survival (PFS) - BRAF Mutant Participants

Treatment effect versus the control arm
Forest plot of hazard ratio (as reported) vs control (95% CI)Nivolumab + Ipilimumab vs Ipi…0.120.190.300.470.741.17hazard ratio (as reported) vs control (95% CI)

The secondary outcomes extend the comparison beyond the primary BRAF wild-type response question. They include progression-free survival and objective response rate in both BRAF subgroups, as well as change from baseline in the EORTC Overall Quality of Life C30 score. The registry record lists four secondary outcomes, but does not report a result for the quality-of-life outcome. The posted response and progression-free survival estimates generally point in the same direction as the primary comparison, while the subgroup estimates are less precise.

These outcomes help describe different dimensions of the randomized comparison: response frequency, time to progression or death, and quality of life. They are related to, but do not duplicate, the primary endpoint. A response percentage does not establish how long a response lasts, while progression-free survival incorporates time to progression or death. A quality-of-life measure would address a different patient-reported domain, but its result is not reported in the registry record.

The secondary estimates should carry less inferential weight than the primary endpoint because multiple outcomes and subgroup comparisons are listed and no multiplicity adjustment is reported in the registry record. General statistical explanation: when many hypotheses are tested, the chance of finding at least one apparently unusual result can increase if each test is considered separately at its nominal threshold. Unadjusted secondary p-values, where available through recalculation, should therefore be viewed as exploratory rather than as independent confirmation. The registry does not provide a complete hierarchy or testing strategy, so it is not possible to determine from the record which secondary comparisons, if any, were intended to confirm a claim.

Subgroup sizes also matter. The BRAF mutant analyses include smaller populations than the BRAF wild-type analyses, and their intervals are correspondingly important context for any apparent pattern. The common direction across endpoints is descriptive; it does not remove uncertainty, address multiplicity, or establish an effect on the unreported quality-of-life outcome.

Secondary outcomes registered for this trial, for reference (only those above were recalculated):

9. Statistical Interpretation

Clinical Biostats interpretation

For BRAF wild-type participants, the posted objective response percentage was 60.3% in the nivolumab-plus-ipilimumab arm and 10.8% in the ipilimumab arm. The 95% confidence intervals were 48.1% to 71.5% and 3.0% to 25.4%, respectively. The unadjusted recalculation reports p < 0.001. These results describe a higher observed response proportion in the combination arm in this trial population. They do not establish how a particular participant would respond.

The posted sponsor analysis reports a mean difference of 49.5, with a 95% confidence interval from 31.4 to 61.8, and an odds ratio of 12.52, with a 95% confidence interval from 3.79 to 52.55. Its analysis method is listed as Fisher Exact. No sponsor p-value is listed for these analyses. The recalculated risk difference is oriented as ipilimumab compared with nivolumab plus ipilimumab, so its negative direction corresponds to the smaller response proportion in the ipilimumab arm. The separate descriptions of the sponsor analysis and recalculation use different reported measures and should not be conflated.

The secondary results cover response and progression-free survival in the BRAF subgroups. The direction of the posted estimates generally favors the combination arm, but each subgroup has a smaller analysis population than the overall enrollment, and some confidence intervals are broad. The BRAF mutant response estimate, in particular, is based on 22 and 10 participants. Progression-free survival estimates also have uncertainty, and a median confidence limit is not reported for the combination arm in either subgroup. These features make subgroup patterns less definitive than a large, precise estimate would be.

Statistical significance is not a measure of clinical importance or certainty about every aspect of the comparison. A small p-value indicates that the observed response contrast is not readily compatible with a null contrast under the assumptions of the unadjusted test; it does not account for every possible source of bias, nor does it settle questions about duration, quality of life, or harms. The registry record does not report adjustment for multiple outcomes, so the secondary analyses should be interpreted cautiously.

The trial’s randomized design makes the between-arm comparison informative for the enrolled population, but the analysis details available in the registry are incomplete. In particular, the registry record does not report the full prespecified analysis plan, the handling of missing outcome data, or whether any analyses were interim. Those omissions limit assessment of how closely the posted analyses follow the original plan. The results should therefore be read as registry-posted estimates with the stated analysis populations and intervals, not as a complete account of all trial evidence.

10. Statistical Methods Explained

The primary outcome is binary at the participant level: a participant either met the stated complete- or partial-response definition or did not. The registry reports the proportion responding in each arm and 95% confidence intervals for those proportions. A separate unadjusted recalculation compares the arm proportions. General statistical explanation: a risk difference expresses the absolute separation between two event proportions, with its sign depending on which arm is subtracted from which. The recalculation uses ipilimumab as the comparison arm against the combination arm, which is why the reported direction is negative when the response proportion is lower with ipilimumab.

What does the response confidence interval describe?

General statistical explanation: a confidence interval expresses the uncertainty of an estimate under the method used to construct it. A wider interval means the data are compatible with a broader range of values than a narrower interval, all else equal. The arm-specific intervals describe uncertainty around each response percentage; the interval for a between-arm contrast addresses uncertainty in that contrast. Overlap or non-overlap of two arm-specific intervals is not a substitute for a direct comparison interval or test. The primary sponsor analysis reports a 95% interval for its difference estimate, while the recalculation also gives an interval for the unadjusted contrast.

Why is Fisher’s exact method listed?

The sponsor’s primary analyses are recorded as using Fisher Exact, with reported results for a mean difference and an odds ratio. General statistical explanation: Fisher’s exact test is a method for comparing categorical outcomes that uses the observed counts and can be suitable when cell counts are limited. An odds ratio describes the relative odds of the event between groups; it is not the same measure as a difference in response percentages. The registry record does not provide the complete calculation details behind each reported interval or a sponsor p-value for these analyses.

The recalculated p-value is explicitly unadjusted. General statistical explanation: a p-value measures how unusual data at least as incompatible with a specified null hypothesis would be under the test’s assumptions. It is not the probability that the null hypothesis is true, and it does not measure the size or practical importance of an effect. Because the record has multiple outcomes and analyses, a single unadjusted p-value does not account for the full set of comparisons.

How are the progression-free survival estimates different?

Progression-free survival is a time-to-event outcome, defined in the record as time from randomization to progression or death. It is reported separately for BRAF wild-type and BRAF mutant participants. The registry gives medians and, for the comparison, hazard ratios from an unstratified Cox proportional hazard analysis. General statistical explanation: a median time-to-event estimate is the time by which half of the relevant population is estimated to have experienced the event, with the estimate depending on follow-up and censoring. A hazard ratio compares event rates over follow-up under the fitted model; it is not a ratio of medians and does not directly state how many months are gained.

For BRAF wild-type participants, the reported hazard ratio is 0.36, with a 95% confidence interval from 0.21 to 0.59. For BRAF mutant participants, it is 0.36, with a 95% confidence interval from 0.14 to 0.97. General statistical explanation: a hazard ratio below 1 indicates a lower estimated event hazard in the numerator group relative to the comparison group, according to the model and its coding. The record labels the comparisons as nivolumab plus ipilimumab versus ipilimumab. No p-value is reported for either sponsor hazard-ratio analysis. The proportional-hazards assessment and any alternative modeling details are not reported in the registry record.

What limits the statistical interpretation?

The primary population is explicitly identified as all randomized BRAF wild-type participants, but further analysis-population rules are not reported in the registry record. The record does not specify how missing response assessments were handled, whether all randomized participants were included in each analysis, or whether sensitivity analyses were performed. It also does not describe the allocation concealment process, stratification factors, or a complete prespecified statistical plan. These are not grounds for assuming a particular procedure; they are limits on what can be checked from the registry entry.

The reported arm sizes and uncertainty intervals matter when judging precision. The primary comparison uses 73 and 37 participants, and subgroup analyses use smaller groups. General statistical explanation: with fewer observations, estimates can be more sensitive to individual outcomes, and intervals may be wide. This is especially relevant to subgroup estimates and to odds-ratio intervals that extend broadly. The intervals are estimates of sampling uncertainty under their methods, not a guarantee that the true value lies within a particular interval.

11. Confidence Intervals and Precision

The primary response estimates are 60.3% for nivolumab plus ipilimumab and 10.8% for ipilimumab. Their 95% confidence intervals are 48.1% to 71.5% and 3.0% to 25.4%, respectively. The reported sponsor mean-difference interval is 31.4 to 61.8, and the sponsor odds-ratio interval is 3.79 to 52.55. The recalculated unadjusted comparison has a confidence interval that supports a contrast favoring the combination arm in direction, when interpreted with the comparison orientation stated for that calculation.

General statistical explanation: an interval combines the point estimate with a range reflecting sampling uncertainty under a particular method. It does not capture every uncertainty about study conduct, analysis choices, or generalizability. The arm-specific intervals and the contrast interval answer related but distinct questions. The odds-ratio interval is notably broad, so the exact magnitude on that scale is less precisely determined than the direction suggested by its reported bounds.

For progression-free survival, the BRAF wild-type hazard ratio is 0.36 with a 95% confidence interval from 0.21 to 0.59; the BRAF mutant estimate is 0.36 with a 95% confidence interval from 0.14 to 0.97. The latter interval extends close to 1, reflecting greater uncertainty about the subgroup comparison. The median progression-free survival estimates are 58.41 months and 4.3 months in the wild-type arms, and 8.61 months and 2.73 months in the mutant arms. The upper confidence limit for the combination-arm median is not reported in the registry record in either subgroup.

These intervals should be considered alongside the analysis populations. The subgroup denominators are 73 and 37 for BRAF wild-type response, and 22 and 10 for BRAF mutant response. Smaller populations generally offer less precision as a general statistical principle, although the exact interval width also depends on observed outcomes and the method used. The uncertainty displayed by each estimate is not a measure of clinical importance.

12. Planning a Similar Trial: Sample Size

This is a planning illustration, not a re-evaluation of the original trial. It asks: if a new study wanted to detect a difference like the one observed, or a smaller one, how large would it need to be? It is deliberately not “observed power” of the finished trial, which is not a meaningful quantity.

Effect to detectParticipants per arm, 80% powerParticipants per arm, 90% power
100% of the observed effect1418
75% of the observed effect2736
50% of the observed effect6384
Sample size needed versus the size of the effect to detect
Sample size needed versus size of effect to detect10205010020040%60%80%100%80% power90% powersize of effect to detect, as % of the effect observed (log vertical scale: participants per arm)
Method and assumptions. Two-sided α = 0.05, 1:1 allocation, pooled-variance normal approximation for two proportions. Counts were reconstructed from the posted percentages and arm sizes, so small rounding differences are possible. Calculated for the comparison: Ipilimumab.

The sample-size figures are planning illustrations based on the observed effect, not the trial’s own design justification. For the binary response comparisons, the listed figures are participants per arm. For the progression-free survival comparisons, they are total events across the comparison. The rows describe planning for 100%, 75%, or 50% of the observed effect; the columns describe participants per arm for 80% power and participants per arm for 90% power for binary outcomes, and total events for the corresponding power levels for time-to-event outcomes.

In plain language, the 80% power column gives the illustrative sample size or event count that would yield an 80% chance of detecting the specified effect under the stated assumptions, if that effect were present. The 90% column uses a higher target chance of detection and consequently lists a larger requirement. These are planning calculations, not outcomes observed in the trial. This is not observed power.

The listed assumptions are a two-sided significance level of 0.05, equal allocation between the two arms, and a normal approximation. The target effect is the stated fraction of the observed effect. For time-to-event planning, the calculation uses the Schoenfeld method and gives total events; other planning figures give participants per arm. As the target effect becomes smaller, more participants or events are needed to distinguish it from random variation under the same assumptions. These illustrations do not show that the original trial was designed with those assumptions or that its design had a particular power; the original design assumptions are not reported in the registry record.

13. How the Sample Size Is Derived

Power is the probability that a study of a given size rejects the null hypothesis when a specific true effect exists. It depends on four things: the effect to detect, the variability of the outcome, the significance level (here a two-sided α of 0.05, so z1−α/2 = 1.960) and the desired power (z1−β = 0.842 for 80% and 1.282 for 90%).

Formula

n = [ z1−α/2 √(2p̄(1−p̄)) + z1−β √(p0(1−p0) + p1(1−p1)) ]2 / (p1 − p0)2

n is the number of participants in each arm, p0 and p1 are the control and treatment proportions, and p̄ is their average.

Worked calculation for this trial

  1. Control proportion p0 = 0.603; treatment proportion p1 = 0.108; average p̄ = 0.355.
  2. First term: z1−α/2 × √[2p̄(1−p̄)] = 1.96 × 0.6769 = 1.3267.
  3. Second term: z1−β × √[p0(1−p0) + p1(1−p1)] = 0.842 × 0.5795 = 0.4878.
  4. n = (1.3267 + 0.4878)2 / (-0.4946)2 = 13.5, rounded up to 14 participants per arm for 80% power.

The smaller-effect rows in the table above repeat the same calculation with the effect multiplied by 0.75 and 0.5 (for ratios, applied to the logarithm of the ratio).

R code

alpha <- 0.05
z_a   <- qnorm(1 - alpha / 2)          # two-sided
p0 <- 0.602739726027397                          # control proportion
p1 <- 0.108108108108108                          # treatment proportion
n_per_arm <- function(power, frac = 1) {
  q1 <- p0 + (p1 - p0) * frac               # smaller effect = fraction of the observed difference
  pbar <- (p0 + q1) / 2
  z_b <- qnorm(power)
  ceiling(round((z_a * sqrt(2 * pbar * (1 - pbar)) + z_b * sqrt(p0 * (1 - p0) + q1 * (1 - q1)))^2 / (q1 - p0)^2, 6))
}
for (f in c(1, 0.75, 0.5))
  cat(sprintf("%3.0f%% of observed effect: n per arm = %s (80%% power), %s (90%% power)\n", 100 * f,
              format(n_per_arm(0.80, f), big.mark = ","), format(n_per_arm(0.90, f), big.mark = ",")))

# Check against R's built-in function (unrounded n per arm)
cat("power.prop.test:", round(power.prop.test(p1 = p0, p2 = p1, power = 0.80)$n, 1), "\n")

# Simulation: how often does the z-test reject at the 80%-power sample size?
set.seed(1927419)
n    <- n_per_arm(0.80)
reps <- 20000
x0 <- rbinom(reps, n, p0); x1 <- rbinom(reps, n, p1)
ph <- (x0 + x1) / (2 * n)
z  <- ((x1 - x0) / n) / sqrt(ph * (1 - ph) * 2 / n)
cat(sprintf("Simulated power at n = %s per arm: %.3f (%d replicates)\n", format(n, big.mark = ","), mean(abs(z) > z_a, na.rm = TRUE), reps))

Output from running the code

100% of observed effect: n per arm = 14 (80% power), 18 (90% power)
 75% of observed effect: n per arm = 27 (80% power), 36 (90% power)
 50% of observed effect: n per arm = 63 (80% power), 84 (90% power)
power.prop.test: 13.5 
Simulated power at n = 14 per arm: 0.830 (20000 replicates)

The simulation draws many pairs of event counts at the planned sample size and counts how often the two-proportion z-test rejects the null. It should land close to the target power; at very small required sample sizes it can fall below it, because the normal approximation is rough there.

Very small required sample. The formula gives a very small requirement because the observed effect is very large relative to the variability. Normal-approximation formulas are unreliable at such sizes, and the simulation above shows the power actually achieved. An exact or simulation-based design would be needed in practice.

14. Analyses Reported by the Sponsor

MethodEstimate95% CIP-value
Fisher ExactMean Difference (Final Values) 49.531.4 to 61.8not reported
Fisher ExactOdds Ratio (OR) 12.523.79 to 52.55not reported

For the primary BRAF wild-type response comparison, the sponsor’s recorded analyses report a mean difference of 49.5 with a 95% confidence interval from 31.4 to 61.8, and an odds ratio of 12.52 with a 95% confidence interval from 3.79 to 52.55. The listed method is Fisher Exact, and the sponsor p-values are not reported in the registry record. The posted arm percentages are 60.3% and 10.8%.

The simple unadjusted recalculation uses the posted response counts and compares ipilimumab with nivolumab plus ipilimumab. Its contrast therefore uses the reverse arm order from a combination-minus-ipilimumab presentation. Its p-value is p < 0.001. A risk difference and an odds ratio describe different scales, so their numerical values should not be compared as if they were the same quantity. The sponsor difference is reported as a positive mean difference, while the recalculated risk difference follows its stated comparison orientation. Both descriptions point to a higher observed response proportion in the combination arm.

The registry record does not flag a discrepancy between the sponsor and recalculated results. The sponsor analysis is the trial’s reported analysis; the recalculation is a simple unadjusted check from the posted summary counts, not a replacement for a full prespecified analysis. Further details about the sponsor’s implementation and analysis population are not reported in the registry record.

15. Multiplicity, Missing Data and Analysis Populations

The registry record lists one primary outcome and four secondary outcomes, and records six analyses. The primary endpoint is objective response rate among BRAF wild-type participants. Secondary analyses cover progression-free survival and response in BRAF subgroups and change from baseline in the EORTC Overall Quality of Life C30 score. A complete prespecified testing hierarchy, a familywise error strategy, and any multiplicity adjustment are not reported in the registry record.

This matters because results from multiple analyses are not independent pieces of confirmatory evidence simply because each has its own estimate or interval. General statistical explanation: when multiple hypotheses are examined, the probability of at least one nominally significant finding can rise if no adjustment or prespecified hierarchy is used. The primary unadjusted recalculated p-value is evidence about its own comparison under the test assumptions, but it does not account for the other analyses. The posted secondary results should consequently be interpreted with appropriate caution.

Several analysis details needed to assess bias and robustness are also not reported in the registry record. These include the full analysis populations for each outcome, rules for missing or unevaluable response assessments, censoring details for progression-free survival, sensitivity analyses, and whether any interim analyses occurred. The record identifies the primary population as all randomized BRAF wild-type participants, but does not state enough to establish how every participant with missing or incomplete follow-up was handled. The original sample-size justification and original statistical analysis plan are likewise not reported.

The absence of these details does not prove that a particular method was or was not used. It limits independent assessment of whether the posted analyses followed a prespecified approach and how sensitive results may be to missing-data or analytic choices. The record’s reported methods and estimates remain interpretable on their stated terms, but should not be expanded into claims about unreported procedures.

16. Safety Results

The registry reports serious adverse events as affected participants divided by participants at risk.

ArmParticipants with serious adverse eventsParticipants at riskPercentage
Nivolumab + Ipilimumab699473.4%
Ipilimumab274658.7%
Participants with at least one serious adverse event
Serious adverse events by armNivolumab + Ipilimumab73.4%Ipilimumab58.7%% of participants with at least one serious adverse event

Deaths

ArmDeaths (participants affected of at risk)
Nivolumab + Ipilimumab44 of 94
Ipilimumab29 of 46

Other (non-serious) adverse events

Participants with at least one non-serious event at or above the registry's 5% reporting threshold.

ArmParticipants affected of at risk
Nivolumab + Ipilimumab90 of 94
Ipilimumab45 of 46

Most frequent serious adverse event terms

Event termOrgan systemNivolumab + IpilimumabIpilimumab
Malignant neoplasm progressionNeoplasms benign, malignant and unspecified (incl cysts and polyps)10 of 948 of 46
ColitisGastrointestinal disorders13 of 942 of 46
DiarrhoeaGastrointestinal disorders10 of 945 of 46
PyrexiaGeneral disorders6 of 944 of 46
PneumonitisRespiratory, thoracic and mediastinal disorders7 of 942 of 46
PneumoniaInfections and infestations4 of 943 of 46
Autoimmune colitisGastrointestinal disorders3 of 942 of 46
DyspnoeaRespiratory, thoracic and mediastinal disorders3 of 941 of 46
Atrial fibrillationCardiac disorders2 of 941 of 46
Adrenal insufficiencyEndocrine disorders3 of 940 of 46

Most frequent non-serious adverse event terms

Event termOrgan systemNivolumab + IpilimumabIpilimumab
FatigueGeneral disorders57 of 9434 of 46
DiarrhoeaGastrointestinal disorders54 of 9424 of 46
NauseaGastrointestinal disorders40 of 9425 of 46
PruritusSkin and subcutaneous tissue disorders47 of 9417 of 46
RashSkin and subcutaneous tissue disorders45 of 9417 of 46
CoughRespiratory, thoracic and mediastinal disorders30 of 9420 of 46
PyrexiaGeneral disorders32 of 9417 of 46
ConstipationGastrointestinal disorders32 of 9414 of 46
HeadacheNervous system disorders35 of 9411 of 46
Decreased appetiteMetabolism and nutrition disorders26 of 9417 of 46

Serious events with the highest arm-level rate: malignant neoplasm progression (17.4%), colitis (13.8%), diarrhoea (10.9%), pyrexia (8.7%), pneumonitis (7.4%).

Do not overextend the safety comparison. Serious-adverse-event counts are safety observations, not efficacy outcomes, and should not be combined with the primary result into a single benefit-risk number.

The registry reports serious adverse event figures for 94 participants at risk in the nivolumab-plus-ipilimumab group and 46 in the ipilimumab group. Serious adverse events affected 69 and 27 participants, respectively. Other adverse event figures list 90 of 94 participants affected in the combination group and 45 of 46 in the ipilimumab group. The record also lists 44 deaths in the combination group and 29 in the ipilimumab group.

These are counts of participants, not a comparison adjusted for follow-up time, and the record does not provide enough detail to attribute individual events or deaths to treatment. The listed serious adverse event terms include malignant neoplasm progression (17.4%), colitis (13.8%), diarrhoea (10.9%), pyrexia (8.7%), pneumonitis (7.4%), and pneumonia (6.5%). Those term percentages should be read as registry-reported safety summaries, not as a complete account of each event’s severity, timing, or causal assessment.

Safety counts and efficacy estimates address different questions. The figures alone do not establish a comparative causal effect on serious events or mortality, and they do not provide individualized risk estimates. Further definitions, exposure-adjusted summaries, event timing, and attribution details are not reported in the registry record.

17. What the Result Does, and Does Not, Mean

The primary registry result is a comparison of investigator-assessed objective response proportions among randomized BRAF wild-type participants. The observed percentages are higher in the nivolumab-plus-ipilimumab arm than in the ipilimumab arm, and the unadjusted recalculation reports p < 0.001. The sponsor’s posted mean-difference and odds-ratio analyses describe the same primary response comparison using different measures. Together, these estimates describe the outcomes recorded in this trial; they do not determine the outcome for any individual participant.

A response endpoint is not interchangeable with progression-free survival, overall survival, response duration, or quality of life. The registry also reports progression-free survival estimates, but no overall survival result is listed among the outcomes summarized in the record, and the quality-of-life outcome result is not reported in the registry record. The response percentages therefore cannot answer questions about those other outcomes.

The trial was randomized and masked, which are design features relevant to reducing bias in a comparison. However, the available record does not include the complete statistical plan or important analysis details such as missing-data handling and multiplicity control. The BRAF mutant results arise from smaller analysis groups, and their uncertainty intervals should be considered rather than focusing only on point estimates. Secondary findings also carry less weight when adjustment for multiple comparisons is not reported.

Statistical evidence is not the same as a clinical recommendation. These results do not say that a treatment works or does not work for a particular person, do not establish the balance of benefits and harms for an individual, and do not replace a complete clinical evidence assessment. They describe the comparisons and uncertainty documented in the registry record for this study population and these specified outcomes.

18. Why This Trial Matters Statistically

This trial illustrates why the endpoint, analysis population, and effect measure all matter when reading a randomized comparison. The primary question is narrowly defined: the percentage of BRAF wild-type participants meeting investigator-assessed response criteria. The result is not a generic measure of every possible clinical outcome. Its interpretation depends on the RECIST-based response definition, the stated assessment schedule, and the participants included in the analysis.

The record also demonstrates that one comparison can be expressed on different scales. A difference in response proportions describes absolute separation, while an odds ratio describes relative odds. The sponsor reports both measures, and the unadjusted recalculation uses a risk difference with a particular arm order. Understanding the measure and its direction prevents a sign or scale from being mistaken for disagreement about which arm had the larger observed response percentage.

Precision deserves equal attention to the point estimate. Confidence intervals communicate uncertainty, and subgroup estimates based on smaller analysis populations may be less stable. The PFS hazard ratios and medians also answer distinct statistical questions: the hazard ratio compares event hazards under a model, while the median summarizes a time-to-event distribution. They should not be substituted for each other.

Finally, the registry entry provides enough detail to describe several estimates but not enough to reconstruct the full analysis plan. Multiplicity, missing-data handling, and original design assumptions are not reported. This distinction between posted results and unreported analytic details is important: statistical interpretation should be specific about what the estimates support while avoiding claims about procedures or outcomes that the record does not establish.

19. Limitations

The registry entry has limitations for interpreting the results beyond the displayed estimates. It does not report a full statistical analysis plan, the original sample-size justification, interim-analysis details, or multiplicity procedures. Missing-data rules and the handling of participants without an evaluable response are also not reported in the registry record. These omissions prevent a complete assessment of prespecification and robustness.

The primary analysis is restricted to BRAF wild-type participants, rather than all enrolled participants. The record gives 73 and 37 participants for the primary arms, while the BRAF mutant response analysis includes 22 and 10. These subgroup populations are important context for precision and generalizability. The registry record does not report further information on how BRAF status was determined or how participants with unknown status were handled.

For progression-free survival, the record reports medians and hazard ratios from an unstratified Cox proportional hazard analysis, but does not report the proportional-hazards assessment, censoring details, or p-values for the sponsor hazard-ratio analyses. The upper confidence limits for the combination-arm median estimates are not reported in the registry record. The quality-of-life outcome is listed, but its result is not reported in the registry record.

Safety summaries include participant counts and selected serious adverse event terms, but do not provide enough information to evaluate exposure-adjusted event rates, timing, attribution, or all event categories. The completed status and enrollment count do not by themselves resolve these analytic limitations. These points define what can be assessed from the registry record; they do not imply that an unreported procedure was omitted from the trial.

20. Related Tutorials

21. Related Statistical Calculators

22. Sources

Continue through the Clinical Biostats statistical pathway

Use this trial's endpoint as a starting point for deeper study of effect measures, confidence intervals, and sample-size planning.

23. Record Summary

NCT01927419 is a completed, randomized, quadruple-masked, parallel-group phase 2 study sponsored by Bristol-Myers Squibb. It compared nivolumab plus ipilimumab with ipilimumab alone in previously untreated, unresectable or metastatic melanoma, with registry enrollment of 142.

The primary outcome was investigator-assessed objective response rate among BRAF wild-type participants. The posted response percentages were 60.3% for the combination arm and 10.8% for ipilimumab; the unadjusted recalculation reports p < 0.001. The sponsor reports a mean difference and an odds ratio with 95% confidence intervals. Secondary outcomes include response and progression-free survival in BRAF subgroups and a quality-of-life measure; the latter result is not reported in the registry record.

The posted estimates describe the trial’s analyzed populations and have varying precision. The registry record does not report the complete statistical plan, multiplicity adjustment, original design assumptions, or several missing-data and survival-analysis details.

Clinical Biostats