What Does a Probability of Success Refer To?
“Probability of success” (PoS) is not a complete description of a quantity until success, the population or program, the time horizon, and the available information have been specified. Success might mean meeting a trial’s primary endpoint, progressing to the next development stage, obtaining approval, or achieving a commercial threshold. These are different events and generally have different probabilities.
A PoS is a probability assigned to a defined event under a stated probability model or reference class. It is not an intrinsic property of a drug candidate that is independent of assumptions. A forecast for a new program may depend on its stage, indication, design, available evidence, and the population of comparable programs used to inform the forecast. The introductory material on probability-of-success decision rules gives the broader decision context; here the emphasis is on avoiding errors in interpreting and calculating the probability itself.
That statement does not claim that the program will succeed with certainty, nor that 62% of its possible outcomes are already known. It summarizes uncertainty under the stated model and information. Changing the endpoint, analysis rule, design, evidence, or assumptions can change the probability.
Historical Attrition Is Not Automatically an Individual Forecast
A historical attrition rate is a summary of outcomes in a defined cohort. It can be useful for forecasting, but it does not automatically equal the probability of success for a particular program. The cohort’s eligibility criteria, development stage, therapeutic area, calendar period, and definition of success all matter. A historical rate can also combine programs with very different evidence and design characteristics.
For a sequence of stages, the probability of reaching the final stage can be written as a product of conditional transition probabilities. Each factor describes the chance of advancing from one stage given that the program reached the relevant preceding stage. This is not the same as assuming that all stage outcomes are independent.
Worked Example: Interpreting a Historical Development Pathway
Suppose a hypothetical historical cohort contains 100 programs that entered Phase I. Of these, 60 advanced to Phase II, 30 of those 60 advanced to Phase III, and 18 of those 30 ultimately met a defined approval-related success criterion. Calculate the observed overall proportion and the stage-transition proportions.
The overall cohort proportion is \(18/100=0.18\), or 18%. The observed conditional transition proportions are \(60/100=0.60\) from Phase I entry to Phase II, \(30/60=0.50\) from Phase II to Phase III, and \(18/30=0.60\) from Phase III to the defined final outcome.
The product reproduces the cohort’s observed 18% overall proportion because the denominators are nested: each stage’s numerator is the next stage’s denominator. It may serve as a reference for a new program only if the historical cohort and the new program are relevantly comparable, and if the outcome definitions and time horizon match. It is not automatically an 18% forecast for every Phase I program. A program with different evidence, design, or context may warrant a different forecast.
The distinction is especially important when a rate is reported without its denominator or starting point. “The approval rate is 60%” could refer to programs entering Phase III, programs entering Phase I, or a selected subset with a particular type of evidence. Those rates answer different questions. A forecast must use the rate whose conditioning population matches the program and decision being considered, or justify an adjustment.
Multiplication Requires Attention to Dependence
For two events \(A\) and \(B\), the general multiplication rule is
provided \(P(A)>0\), so that \(P(B \mid A)\) is defined. This rule does not require independence. Independence is the additional condition \(P(B \mid A)=P(B)\), when those conditional probabilities are defined. Only under independence can the joint probability be calculated as \(P(A)P(B)\).
For three events, the corresponding chain-rule factorization is
As written, this factorization requires \(P(A)>0\) and \(P(A \cap B)>0\), so both conditional probabilities are defined. If a conditioning event has probability zero, the conditional-probability factorization is not defined as written. Independence is not required for this chain-rule factorization; multiplying only the marginal probabilities \(P(A)P(B)P(C)\), by contrast, would require mutual independence.
Worked Example: Combining Dependent Program Events
Consider two hypothetical events for a development program: \(A\), a prespecified clinical efficacy criterion is met, and \(B\), the program has adequate operational execution for the planned evidence package. Suppose \(P(A)=0.60\), \(P(B \mid A)=0.80\), and \(P(B \mid A^c)=0.25\), where \(A^c\) means that the efficacy criterion is not met. Calculate the probability that both events occur and compare it with the result from an independence assumption.
The probability that both events occur is
To assess independence, first calculate the marginal probability of \(B\) by splitting on whether \(A\) occurs:
If one incorrectly multiplied the marginals as if the events were independent, the result would be \(P(A)P(B)=(0.60)(0.58)=0.348\), not 0.48. The supplied conditional probabilities show why independence is inappropriate: \(P(B \mid A)=0.80\), whereas \(P(B)=0.58\). The example’s values are hypothetical; in an actual analysis, the dependence structure must be justified from relevant data or assumptions.
A common source of dependence is shared information or a shared cause: for example, the same program characteristics may influence several transition events. Treating events as independent can either understate or overstate a joint probability. The direction is not fixed; it depends on the relationship between the events. When stages are represented as a sequential pathway, conditional transition probabilities are often the clearest way to express the calculation.
The diagnostic question is not simply “Can these probabilities be multiplied?” It is: “Which probabilities are these, and what are they conditional on?” A multiplication of successive conditional probabilities may be a valid chain-rule calculation. A multiplication of marginal probabilities silently imposes independence unless that assumption is explicitly justified.
Conditional Power and Posterior Predictive Probability
Interim analyses often produce a probability that a trial will ultimately succeed. Two quantities that can look similar have different interpretations. Conditional power commonly means the probability of final trial success conditional on the interim data, calculated under a specified assumption about the unknown treatment effect. A common version plugs in the interim estimate as if it were the true effect. Other conditional-power calculations use a prespecified effect instead, so the assumed effect must always be stated.
A posterior predictive probability instead averages the probability of success over the posterior distribution of the unknown treatment effect, given the interim data. It incorporates uncertainty about that effect as well as future sampling variability. Neither quantity is “the” probability of success without qualification: they answer different questions and can differ substantially.
Worked Example: Plug-In Conditional Power and Posterior Prediction
Suppose, hypothetically, that the final estimated treatment effect \(X\), measured in endpoint units, will meet the success rule if \(X>0.50\). Conditional on a fixed true effect \(\theta\), assume \(X \mid \theta \sim N(\theta,0.25^2)\). Interim evidence gives a posterior distribution \(\theta \mid \text{interim data} \sim N(0.40,0.30^2)\). Calculate plug-in conditional power using the posterior mean as the assumed true effect, and calculate the posterior predictive probability of success.
For plug-in conditional power, set \(\theta=0.40\). The standard deviation of the final estimate under this fixed-effect assumption is 0.25 endpoint units. Therefore,
This is about a 34.5% chance of meeting the rule under the plug-in effect assumption and the stated future-sampling model. It does not average over uncertainty in \(\theta\).
For the posterior predictive probability, integrate over the posterior uncertainty in \(\theta\). Under the stated normal model, the predictive distribution of \(X\) has mean 0.40 and variance \(0.30^2+0.25^2=0.1525\), because the future sampling variation and posterior uncertainty are combined. Its standard deviation is \(\sqrt{0.1525}\approx0.3905\) endpoint units. Thus,
The posterior predictive probability is about 39.9% under these assumptions. It is higher here because averaging over uncertainty about \(\theta\) puts some probability on effects above the posterior mean; this direction of difference is not guaranteed in every model or decision rule. The calculation assumes the stated normal likelihood, posterior distribution, and success threshold. Changing any of them can change either probability.
A frequent interpretive error is to report a plug-in conditional power as though it had integrated all uncertainty in the treatment effect. The reverse error is to call a posterior predictive probability “conditional power” without explaining the calculation. At minimum, report the assumed effect or posterior distribution, the future-data model, the success rule, and whether uncertainty in the effect was integrated.
A Short Justification of the Chain Rule
The general multiplication rule follows directly from the definition of conditional probability. If \(P(A)>0\), then \(P(B \mid A)=P(A \cap B)/P(A)\). Multiplying both sides by \(P(A)\) gives the two-event rule. Applying the same definition to the event \(A \cap B\), when \(P(A \cap B)>0\), yields the three-event factorization.
The positivity conditions matter because division by a zero probability is undefined in the ordinary definition of conditional probability. The identity does not license assigning an arbitrary value to a conditional probability given an event of probability zero. In practical program models, check that each conditioning event has positive probability under the model before using the factorization as displayed.
Diagnostics, Alternatives, and Limits
Before presenting a PoS estimate, check whether the event and reference population are aligned. A historical success proportion should have a clear denominator and outcome definition. A pathway product should use rates conditioned on reaching each stage. A joint-probability calculation should make its dependence assumptions visible. An interim forecast should say whether the treatment effect was fixed at a plug-in value or integrated over a posterior distribution.
- Warning sign: a single “industry success rate” is applied to a candidate without checking stage, indication, time period, or outcome definition. Correction: define the relevant cohort and explain why it is informative for this program.
- Warning sign: several marginal probabilities are multiplied because the events occur at different stages. Correction: use conditional transition probabilities or justify independence; stage order alone does not establish independence.
- Warning sign: an interim probability is reported without naming the assumed effect or treatment-effect uncertainty. Correction: identify whether it is a specified-effect conditional power, a plug-in calculation, or a posterior predictive probability.
Historical proportions, conditional transition models, and model-based predictive probabilities are alternatives for different purposes, not interchangeable summaries. Historical proportions are transparent descriptions of a reference cohort but may be poorly matched to a specific program. A transition model makes the conditioning structure explicit but depends on appropriate stage definitions and input estimates. A model-based prediction can use program-specific evidence, but its result depends on its likelihood, assumptions, and success criterion. No calculation removes the need to state those conditions.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- A historical cohort has 24 successes among 120 programs. What does the resulting proportion describe, and why is it not automatically the forecast for a new program?
- If \(P(A)=0.50\) and \(P(B \mid A)=0.70\), calculate \(P(A \cap B)\). Does this calculation require independence?
- For the three-event chain-rule factorization, what positivity conditions are needed for the conditional probabilities to be defined?
- In the worked interim example, which probability integrates uncertainty about the effect, and what assumptions are needed to calculate it?
- Why can multiplying marginal probabilities give a wrong joint probability even when each marginal probability was estimated correctly?
Solutions. (1) The proportion is \(24/120=0.20\), or 20%, among that defined historical cohort. A new program may differ in stage, evidence, design, or context, so the cohort rate is not automatically its forecast. (2) \(P(A \cap B)=0.50(0.70)=0.35\). This uses the conditional multiplication rule and does not require independence. (3) The displayed factorization requires \(P(A)>0\) and \(P(A \cap B)>0\). (4) The posterior predictive probability integrates uncertainty about the effect. Its calculation requires the stated posterior distribution, future-data model, and success threshold. (5) Marginal probabilities do not encode the dependence between events; multiplying them as a joint probability assumes independence, which may not hold.