From a Development Question to a Decision
“Should the program proceed?” sounds like a decision, but it is not yet precise enough to analyze. Proceed with what study, at what time, and under what constraints? What uncertainties could affect the consequences, and what does “better” mean to the people making the choice? A decision formulation makes these elements explicit before calculations or recommendations begin.
The formulation is especially important when the apparent choices include go, stop, and defer. “Defer” is not simply a cautious version of “go”: it uses time and resources differently, and may preserve the possibility of acting later. Its consequences therefore depend on a defined delay, the information or circumstances expected during that delay, and what happens afterward.
Actions, States, Timing, and Consequences
An action should describe something the decision maker can actually do. For a Phase 2 program, “go” might mean fund a specified Phase 3 trial, while “stop” means terminate the program and make no further development investment. These are more useful descriptions than vague labels such as “continue” or “be optimistic.” Feasibility matters: if manufacturing capacity, a regulatory commitment, or a budget limit rules out an action, it should not appear as an available option.
A state describes uncertainty that matters to the consequences. For example, the program’s underlying benefit–risk profile may be adequate or inadequate for the intended population. State definitions should be precise enough to distinguish the cases, and together they should be mutually exclusive and exhaustive: no two states can hold simultaneously, and every relevant possibility belongs to one of them. Evidence such as a trial result is not automatically a state. It may be information about an underlying state, or it may be an event that occurs after an action.
The decision time fixes what information is available when the action is chosen. It also establishes the relevant horizon for consequences. “Defer” needs a duration and a description of what is being waited for; “go now” and “go after the next data readout” are different actions. If a later choice will be made after new information arrives, that is a future decision. At this stage, record that feature clearly rather than hiding it inside an undefined defer option.
Finally, consequences must be measured on a scale suited to the decision. They may include development costs, time to a decision, patient benefit and harm, or financial value. A single score can simplify comparison only if its construction and interpretation are defensible. Earlier material on preference weights is relevant here: a precise-looking aggregate does not resolve uncertainty or disagreement about how consequences should be valued.
Worked Example: Turning a Vague Phase 2 Question into a Formulation
A hypothetical sponsor has completed Phase 2 and must decide at a committee meeting on 1 July whether to advance a candidate. The intended next action is to begin planning a specified Phase 3 program. The committee wants to account for uncertainty about whether the candidate’s benefit–risk profile will be adequate in the proposed population.
Actions. Define \(a_G\) as fund Phase 3 preparation now and \(a_S\) as stop development now. For this formulation, these are the feasible choices at the meeting; “wait and see” is not included unless a particular delay and its consequences are specified.
States. Define \(s_A\) as an adequate benefit–risk profile for the intended population and \(s_I\) as an inadequate profile. These states cannot both hold, and, under the stated two-category definition, one must hold. The definitions should be connected to prespecified clinical criteria rather than left to a committee’s impression after seeing the result.
Timing and consequences. The action is selected on 1 July using information available then. Consequences are assessed over a common development horizon and include investment costs, the possibility of delivering clinical benefit, and the consequences of pursuing a candidate with an inadequate profile. The formulation is now explicit, but no action is recommended yet: the consequence valuations and uncertainty assessment still need to be supplied.
Comparing Actions Without Hiding the Criterion
A consequence table is a useful first representation. Rows list actions, columns list states, and each cell records the consequence for that combination. If the consequences are expressed as utility and the decision maker uses expected utility, a probability distribution over states is also needed. For a finite set of states, the expected utility of action \(a\) is
This comparison assumes that the states cover the relevant uncertainty, the probabilities describe uncertainty at the decision time, and the utilities are on a common scale that represents the decision maker’s trade-offs. Choosing the action with the largest expected utility is appropriate only when that is the agreed criterion. For monetary consequences, using expected monetary value as the criterion also implies that money is valued linearly over the relevant range; that may not represent every organization’s preferences.
The calculation follows directly from averaging the utility across the possible states, weighted by their probabilities. If the states are mutually exclusive and exhaustive, their probabilities sum to one. For two states, \(s_1\) and \(s_2\), with probabilities \(p\) and \(1-p\), the comparison is
This is a way to make a stated criterion operational, not a method for establishing that the probabilities or utilities are correct. A ranking can change if plausible probabilities, consequence valuations, or state definitions change. Those inputs should therefore remain visible rather than being buried in a single recommendation.
Worked Example: Comparing Go, Stop, and Defer
Consider a hypothetical Phase 2 program. At the decision meeting, the committee defines two mutually exclusive and exhaustive underlying states: \(s_F\), a favorable development outlook, and \(s_U\), an unfavorable outlook. Its stated probabilities are \(P(s_F)=0.30\) and \(P(s_U)=0.70\). The committee has supplied hypothetical net monetary consequences, in millions of dollars, over the same five-year horizon. The defer values summarize the consequences of a specified six-month delay, including its cost and the residual value of retaining the opportunity; they are not just the cost of waiting.
| Action | Favorable outlook \(s_F\) ($ millions) | Unfavorable outlook \(s_U\) ($ millions) |
|---|---|---|
| Go: fund the specified next-stage program now | 70 | -50 |
| Stop: terminate development now | 0 | 0 |
| Defer: wait six months under the stated plan | 12 | 8 |
The committee uses expected net monetary value as its criterion. For go, the calculation is
For stop, \(EU(a_S)=0.30(0)+0.70(0)=0\) million dollars. For defer,
Under the stated probabilities, consequences, horizon, and expected-value criterion, defer has the largest expected value, at $9.2 million. This is a conditional conclusion about the hypothetical inputs, not evidence that deferral is generally preferable. In particular, the result depends on the defer consequences already accounting for the specified delay and what the committee expects to do afterward.
Checking Whether the Formulation Is Sound
Before comparing actions, check that each element is complete and internally consistent. A practical review asks whether the choice is genuinely available, whether it is made at a specified time, whether the state definitions cover the relevant uncertainty without overlap, and whether every consequence uses the same horizon and scale. For defer, check that the duration, purpose, cost, and endpoint are stated. If the decision after the delay is important, say so explicitly rather than presenting defer as a self-contained action with unexplained benefits.
A formulation can fail even when its arithmetic is flawless. For example, a consequence table may omit a serious harm, count a cost for go but not for defer, or use a different time horizon for each action. A state such as “Phase 3 succeeds” may also be ambiguous unless success is defined in terms of an endpoint, population, and decision-relevant criterion. These are specification problems, not calculation errors.
Worked Example: Repairing an Incomplete Defer Option
A hypothetical committee proposes three actions: “go now,” “stop,” and “defer.” It assigns defer a value of $15 million, but does not say how long to wait or what the value includes. The committee’s stated probabilities for favorable and unfavorable outlooks are 0.40 and 0.60. For illustration, suppose its values for go are $60 million and \(-$40\) million, and stop is $0 in either state.
The expected value of go is \(0.40(60)+0.60(-40)=24-24=0\) million dollars. Stop also has expected value \(0.40(0)+0.60(0)=0\) million dollars. Comparing either with an unexplained $15 million for defer is not valid: the defer value may omit the delay cost, may use a different horizon, or may assume a later action that is not available under the other options.
The repair is to define defer, for example, as “delay six months to obtain a specified assay result, incur the associated cost, then reassess under a stated process,” and to value its consequences on the same horizon and scale as go and stop. If a defensible, fully comparable hypothetical value is then $5 million in each state, its expected value is \(0.40(5)+0.60(5)=5\) million dollars. That arithmetic is valid for the supplied values, but a recommendation still requires checking that the $5 million consequence actually represents the defined defer action.
Common Mistakes, Alternatives, and Limits
- Treating a state as an action. “The trial is positive” may be an observed result, not a choice. Name the action the decision maker controls and distinguish it from uncertain conditions or future evidence.
- Leaving defer undefined. “Wait” has no stable consequence until its duration, purpose, costs, and next decision point are clear.
- Comparing unlike consequences. Dollar values over different horizons, or a utility score for one action and a clinical outcome for another, do not form a valid direct comparison without a justified conversion.
- Assuming the table makes the decision objective. The table organizes assumptions; it does not remove value judgments about benefits, harms, time, or financial consequences.
- Using a binary choice when a real option exists. A go/stop framing can conceal a feasible defer choice. Conversely, adding defer without defining it creates an apparent option that cannot be evaluated.
A simple action–state table is well suited to a choice made now under a compact set of uncertainty categories. A prose rationale can be useful for documenting context, but it often leaves actions or consequences ambiguous. A table makes omissions easier to spot. When the choice leads to later decisions that depend on new information, a static table may not capture the full structure; that situation calls for an explicit sequential representation rather than an arbitrary single value for “defer.” Even a well-formed table remains limited by the quality of its states, probabilities, consequence estimates, and valuation rule.
Check Your Understanding
Answer each question using the definitions and decision formulation in this tutorial. Brief solutions are included after the questions.
- A team lists “positive assay result” and “negative assay result” as its actions. What needs correction?
- Why must the consequences for go, stop, and defer use a common horizon?
- With two states of probabilities 0.25 and 0.75, an action has utilities 20 and \(-4\). What is its expected utility?
- A defer option is described only as “wait for more data.” Name two details needed to make it evaluable.
- What does a recommendation to choose the action with the highest expected utility depend on?
Solutions. (1) Assay results are uncertain evidence or outcomes, not actions; list choices the team controls, such as advance, stop, or wait for a defined readout. (2) Different horizons can include different costs, benefits, and time at risk, so the values would not be directly comparable. (3) \(0.25(20)+0.75(-4)=5-3=2\) utility units. (4) Specify the duration and purpose of the wait, along with its costs and what decision follows; any two of these identify missing elements. (5) It depends on the stated state probabilities, consequence utilities, completeness of the state space, and agreement that expected utility is the appropriate criterion.