What Phase 3 Confirmation Means
Before Phase 3, evidence from earlier development is used to choose a dose or regimen for confirmatory testing. Phase 3 then asks whether the prespecified dose produces an effect consistent with the justification for choosing it, under the conditions and on the endpoint defined for the trial. Confirmation is not simply finding a small p-value, nor is it a search for whichever tested dose looks best after the results are known.
The dose justification should specify in advance what evidence would support the choice. Relevant elements can include the expected direction and magnitude of the treatment effect, a clinical relevance threshold, uncertainty around the estimate, and acceptable safety. These are program-specific criteria, not universal numerical rules. The confirmatory analysis should estimate the effect for the selected dose using the prespecified endpoint, population, treatment comparison, and analysis plan.
For illustration, let \(\Delta_d\) denote the true treatment effect for dose \(d\), defined as the selected-dose outcome minus the control outcome on a specified scale. In examples below, positive values mean improvement, and the effect is expressed in endpoint units. The estimate is \(\widehat{\Delta}_d\), with standard error \(SE\). The treatment contrast must match the estimand: for example, the specified population, endpoint, time point, and handling of intercurrent events. A numerical estimate for a different endpoint or estimand is not interchangeable evidence.
Criteria, Estimates, and Uncertainty
A useful dose-justification plan separates questions that are often mistakenly collapsed into one. Is the estimated effect in the expected direction? Is its magnitude compatible with the expected or clinically relevant range? How uncertain is it? Does the safety profile satisfy the prespecified constraints? For instance, a criterion might require a positive effect estimate of at least a prespecified target, a confidence interval excluding no effect, and no breach of a defined safety limit. Another program may require the confidence interval to exceed a clinically meaningful threshold. These rules make different claims, so the chosen rule must be explicit.
A confidence interval describes uncertainty in the treatment-effect estimate under the analysis model. Under a normal approximation, a two-sided 95% confidence interval is formed by estimate plus or minus 1.96 standard errors. This approximation is appropriate when the estimator’s sampling distribution is adequately close to normal and its standard error reflects the planned analysis. The exact method may instead use a t distribution, a model-based interval, or another prespecified procedure, depending on the endpoint and design.
The reason for this interval is that, under the assumed sampling model, the standardized estimation error is approximately standard normal. The central 95% of a standard normal distribution lies between -1.96 and 1.96. Rearranging that statement gives an interval around the observed estimate:
The 95% refers to the long-run coverage of the procedure under its assumptions; it is not a 95% probability that the fixed true effect lies in this particular interval. A confidence interval that excludes zero supports a nonzero effect at the corresponding two-sided significance level, under the specified analysis. It does not by itself show that the effect is clinically important. Likewise, an interval that includes zero does not establish that the true effect is zero.
Worked Example: Checking a Prespecified Effect Rule
Suppose a hypothetical Phase 3 trial compares a selected dose with placebo on a continuous endpoint where higher values indicate improvement. Before unblinding, the team specifies that the dose’s efficacy justification is supported if the estimated improvement is at least 0.30 endpoint units and the two-sided 95% confidence interval excludes zero. The separate safety criterion is that the prespecified safety limit is not exceeded. The estimated treatment difference is 0.42 units, with \(SE=0.12\) units; the safety analysis meets its criterion.
The effect estimate exceeds the target:
The 95% confidence interval is:
The interval is wholly above zero, and the estimate exceeds 0.30 units. Together with the stated safety result, the hypothetical trial meets the specified rule. The interval’s lower limit is below 0.30 units, however. Thus this rule supports an estimate meeting the target and evidence of a positive effect; it does not establish that the true effect exceeds 0.30 units with 95% confidence. If the intended criterion were that the lower confidence limit exceed 0.30 units, this result would not meet it.
Statistical Evidence Is Not the Whole Dose Justification
A hypothesis test can be useful for a prespecified confirmatory question, such as whether the selected dose is superior to control. For a two-sided test of no effect, the test statistic is the estimated contrast minus the null value, divided by its standard error. The p-value is the probability, assuming the null hypothesis and analysis model, of obtaining a statistic at least as extreme as the observed one. It is not the probability that the null hypothesis is true.
The p-value does not summarize effect magnitude, clinical relevance, or safety. A very small effect can be statistically significant in a sufficiently large study; an important estimated effect can be imprecise in a smaller study. Dose confirmation therefore requires reading the estimate and its uncertainty against the prespecified criteria, rather than treating the test result as the complete justification.
Worked Example: A Small Estimate with a Nonsignificant Test
Consider a hypothetical selected-dose comparison on an endpoint measured in units where positive values favor treatment. The estimated effect is 0.020 units and its standard error is 0.01778 units. Suppose the prespecified primary test is two-sided superiority over zero at \(\alpha=0.05\), using a normal approximation.
The test statistic is:
The two-sided p-value is \(2P(Z\geq1.125)\approx0.261\), where \(Z\sim N(0,1)\). Since \(0.261>0.05\), this result does not reject \(H_0:\Delta_d=0\) at the prespecified level. The corresponding 95% confidence interval is:
The interval includes zero, so this result does not provide statistically significant evidence of superiority under the stated test. It also does not prove that the dose has no effect: effects across the interval remain compatible with the data and model. If the prespecified clinical target were, for example, an estimate of at least 0.05 units, the observed estimate would fall short of that target as well. Whether the overall dose rationale is supported must be judged using the actual prespecified criteria, not by relabeling this nonsignificant result as proof of no benefit.
Confirmation Versus Post Hoc Dose Selection
The distinction between confirming a selected dose and selecting a dose after observing Phase 3 results is about the question being answered. A confirmatory analysis evaluates a dose and contrast specified in advance. A post hoc search instead examines the available results and chooses a dose because it appears most favorable. The latter can be useful for generating a hypothesis, but the apparent winner is subject to selection: its estimate may look unusually good partly because it was chosen from several observed estimates.
If more than one dose is intended to support a confirmatory claim, the doses and comparisons should be prespecified, with an appropriate plan for multiplicity and interpretation. A prespecified multi-dose design is not the same as an unplanned switch to whichever dose has the largest observed effect. Nor does a larger point estimate establish that one dose is superior to another; that requires an appropriate direct comparison and its uncertainty.
Worked Example: Why a Larger Post Hoc Estimate Does Not Replace the Selected Dose
In a hypothetical trial, 60 mg was selected before Phase 3. The prespecified efficacy rule requires an estimate of at least 0.30 units and a 95% confidence interval above zero; a separate safety rule requires the observed rate of a specified adverse event to be no more than 15%. At 60 mg, the estimated improvement is 0.36 units with \(SE=0.13\), and the event rate is 10%. A 90 mg arm, which was not the dose selected for the confirmatory claim, has an estimated improvement of 0.50 units with \(SE=0.14\), and an event rate of 18%.
For the selected 60 mg dose:
The estimate exceeds 0.30 units, the interval excludes zero, and the stated adverse-event rate is below the 15% limit. Thus, on the hypothetical criteria given, 60 mg meets the specified rule.
For context, the nominal interval for 90 mg is \(0.50\pm1.96(0.14)=[0.2256,\ 0.7744]\) units. Its point estimate is larger, but the event rate exceeds the stated safety limit. More importantly, choosing 90 mg now because its estimate is larger would change the question from “does the selected dose meet its prespecified justification?” to “which observed dose looks best in these data?” The latter is a post hoc selection and cannot inherit the original confirmatory interpretation. These summaries also do not, by themselves, establish that 90 mg is more efficacious than 60 mg; that requires a prespecified or otherwise appropriately interpreted direct comparison.
Assumptions, Diagnostics, and Limits
The interpretation of dose-specific estimates depends on the design and analysis being fit for the prespecified question. In a randomized, controlled trial, randomization supports comparison of treatment groups in expectation, but it does not repair a mismatched estimand, a poorly measured endpoint, or an analysis that departs from the plan without justification. The standard error and interval must correspond to the specified analysis, and any multiplicity from testing several doses or endpoints must be handled as planned.
Before concluding that the selected dose was confirmed, check whether the endpoint, visit, treatment comparison, population, and intercurrent-event strategy match the prespecified estimand. Review missing data, adherence, treatment implementation, and the amount of information contributing to the estimate. Examine whether the model fits the observed data and whether influential observations or departures from its assumptions materially affect the estimate or interval. If results are sensitive to plausible analysis choices, report that uncertainty rather than presenting one favorable analysis as decisive.
There are limits to what confirmation can show. A positive result at the selected dose supports the prespecified claim under the trial’s design and analysis; it does not prove optimality over untested doses or establish every possible regimen’s benefit-risk profile. Conversely, a failed criterion may reflect an imprecise estimate, a smaller-than-expected effect, or a safety constraint. The result should be described against the exact criterion that failed. A non-significant test is not evidence of equivalence, and a numerical trend across doses should not be converted into a new dose recommendation without accounting for the design and selection involved.
A practical review sequence is:
Identify the selected dose, endpoint, comparator, estimand, and criteria before interpreting observed results.
Report the dose-specific estimate with its standard error or confidence interval and units; compare each quantity with the criterion it addresses.
Apply the prespecified safety constraints and check whether data quality, model assumptions, and planned analyses support the interpretation.
State whether the selected dose met its rule. Label unplanned dose comparisons as exploratory, and do not claim that a larger estimate proves a superior dose.
The central pitfall is to treat the largest observed effect as confirmation of the dose that should be used. That reasoning confuses an outcome-driven selection with a prespecified test, ignores uncertainty and safety, and may overstate evidence after examining multiple options. The correction is to evaluate the selected dose against the criteria set before the results were examined, and to describe any new dose hypothesis as requiring appropriate further evaluation.
Check Your Understanding
Answer each question using the distinction between a prespecified dose criterion and a post hoc dose choice.
- A protocol requires a dose-effect estimate of at least 0.25 units and a 95% confidence interval excluding zero. The estimate is 0.31 units and the interval is [0.02, 0.60]. Does it meet both efficacy criteria?
- A trial reports \(p=0.08\) for a two-sided superiority test at \(\alpha=0.05\). Does this prove there is no treatment effect?
- Why can the dose with the largest observed effect not automatically replace the dose selected before Phase 3?
- A prespecified rule requires the lower 95% confidence limit to exceed 0.20 units. The interval is [0.18, 0.55] units. Is the rule met, even if the point estimate is 0.36 units?
- What should be checked if the reported effect estimate uses a different endpoint or visit from the one in the dose-justification plan?
Solutions. (1) Yes. The estimate, 0.31 units, is at least 0.25, and the interval excludes zero. This meets those stated criteria, without establishing that the true effect exceeds 0.25 units. (2) No. The result does not reject the null at the 0.05 level, but it does not prove the effect is zero. (3) Selecting the largest observed estimate after examining results is outcome-driven and can overstate evidence; it also does not establish a direct difference between doses or satisfy the original dose’s prespecified claim. (4) No. The lower limit, 0.18 units, is below the required 0.20 units; the point estimate does not override the rule. (5) Check whether the analysis answers the prespecified estimand. A different endpoint or visit is not automatically interchangeable and should not be presented as direct confirmation of the original criterion.