Valuing a New Study With Expected Value of Information

Why Put a Value on a Study?

A targeted dose-finding study may improve a later development decision, but conducting it consumes resources and can delay a program. The relevant question is not simply whether the study will produce informative data. It is whether acting on those data is expected to improve the decision enough to justify the study’s cost and delay.

Expected value of information provides a structured way to answer that question. It combines the current uncertainty about a development-relevant state, the possible study results, and the consequences of the actions available after each result. The calculation is decision-specific: information has value when it can change an action in a way that improves expected outcomes.

Decision Model and Expected Value of Sample Information

Definition: Let $\theta$ denote an uncertain state of the world, $a$ a development action, and $U(a,\theta)$ the utility of taking action $a$ when state $\theta$ is true. Utility may be expressed in monetary units, such as net present value in millions of dollars. Without a new study, the best expected utility is $\max_a E[U(a,\theta)]$. For study result $Y=y$, the best conditional expected utility is $\max_a E[U(a,\theta)\mid Y=y]$. The expected value of sample information (EVSI) is the expected improvement from being able to choose an action after observing the study result: $$ \mathrm{EVSI} = \sum_y P(Y=y)\max_a E[U(a,\theta)\mid Y=y] - \max_a E[U(a,\theta)]. $$

The first term averages the best decision value over all possible study results. The second is the best expected value using current information alone. EVSI is a gross value: study cost, delay, and any other consequences of conducting the study have not yet been deducted.

For a discrete state and result, posterior probabilities are obtained from Bayes’ rule:

P(\theta\mid Y=y) = \frac{P(Y=y\mid\theta)P(\theta)} {\sum_{\theta'}P(Y=y\mid\theta')P(\theta')}.

The likelihood $P(Y=y\mid\theta)$ describes how likely each study result is under each state. For a dose-finding study, those likelihoods summarize how the study’s observed outcomes distinguish development-relevant possibilities, such as whether a candidate dose has a favorable benefit-risk profile. They must be justified by design assumptions, model predictions, or other appropriate evidence; they are not the posterior probabilities themselves.

Decision rule: If study cost is $C$ and the expected loss from delay is $D$, both measured at the same valuation date and in the same utility units, then the study’s net value is $\mathrm{EVSI}-C-D$. Under the stated decision model, conduct the study when this quantity is positive; a value of zero indicates indifference.

Worked Development Case

Suppose a team must decide whether to proceed with a program at a proposed dose or stop. All amounts below are hypothetical net present values in millions of dollars, assessed at the same decision date. The favorable state $F$ means the dose has a development profile worth pursuing; the unfavorable state $U$ means it does not. Proceeding yields $+\$150$ million in state $F$ and $-\$100$ million in state $U$. Stopping yields $\$0$ in either state. The team’s current probabilities are $P(F)=0.35$ and $P(U)=0.65$.

A targeted dose-finding study has two possible summaries, positive (+) and negative (−). Assume $P(+\mid F)=0.80$ and $P(+\mid U)=0.25$. Thus, the probability of a positive result is higher when the dose has a favorable profile, but a positive result is not definitive. Assume the study costs $\$18$ million and its delay has an additional expected opportunity cost of $\$5$ million.

Worked Example: Calculate the Study’s Gross and Net Value

1. Find the best current decision. Proceeding without new data has expected value $$ 0.35(150)+0.65(-100)=52.5-65=-12.5\text{ million dollars}. $$ Stopping is worth $\$0$ million, so the team currently chooses to stop. The best current expected value is therefore $\$0$ million.

2. Calculate the probability and value of a positive result. The probability of a positive result is $$ P(+)=0.35(0.80)+0.65(0.25)=0.28+0.1625=0.4425. $$ The posterior probability of a favorable state after a positive result is $$ P(F\mid +)=\frac{0.28}{0.4425}=0.632768\ldots. $$ Using the unrounded posterior, proceeding after a positive result has conditional expected value $$ \frac{0.28(150)+0.1625(-100)}{0.4425} = \frac{42-16.25}{0.4425} = \frac{25.75}{0.4425} = 58.1921\text{ million dollars}. $$ The numerator uses the joint probabilities of each state and a positive result; the denominator converts that joint expected value to a conditional one. Since stopping is worth zero, proceed after a positive result.

3. Calculate the value of a negative result. The probability of a negative result is $$ P(-)=0.35(0.20)+0.65(0.75)=0.07+0.4875=0.5575. $$ The posterior probability of a favorable state after a negative result is $0.07/0.5575=0.125561\ldots$. Proceeding then has conditional expected value $$ \frac{0.07(150)+0.4875(-100)}{0.5575} = \frac{10.5-48.75}{0.5575} = \frac{-38.25}{0.5575} = -68.6099\text{ million dollars}. $$ The best action after a negative result is therefore to stop, worth $\$0$ million.

4. Calculate EVSI and net value. The expected value after the study, before its cost and delay, is $$ 0.4425(58.1921\ldots)+0.5575(0)=25.75\text{ million dollars}. $$ Equivalently, the positive-result branch contributes $0.28(150)+0.1625(-100)=\$25.75$ million and the negative-result branch contributes zero. Because the best current value is zero, $\mathrm{EVSI}=25.75-0=\$25.75$ million. The net value is $$ 25.75-18-5=2.75\text{ million dollars}. $$ Under these assumptions, the study has positive net expected value. The result changes the preferred action in the positive-result branch, while the team would stop in the negative-result branch.

Why the Information Has Value

The study does not create value merely by reducing uncertainty. It creates value here because the positive result leads to proceeding and the negative result leads to stopping. The team avoids proceeding in the negative-result branch, where the updated expected value of proceeding is unfavorable, while retaining the opportunity to proceed when the result is positive.

The basic non-negativity result for gross EVSI follows from the ability to ignore the result and keep the current action. For any fixed action $a$ and any result $y$, the best action after that result is at least as good in conditional expectation as $a$:

\max_b E[U(b,\theta)\mid Y=y] \geq E[U(a,\theta)\mid Y=y].

Multiply by $P(Y=y)$ and sum over results. The right-hand side becomes $E[U(a,\theta)]$, because averaging the conditional expectation over all results recovers the unconditional expectation. Thus,

\sum_y P(Y=y)\max_b E[U(b,\theta)\mid Y=y] \geq E[U(a,\theta)].

This holds for every fixed action $a$, including the best current action. Therefore the expected value after observing information is at least the best value without it, and $\mathrm{EVSI}\geq 0$, provided the decision-maker can choose to disregard the information and the study has no consequences included in this gross-value calculation. Net value can, of course, be negative once cost and delay are counted.

Study Quality, Perfect Information, and Break-Even Values

Worked Example: Compare a Less Informative Study

Suppose an alternative design has $P(+\mid F)=0.60$ and $P(+\mid U)=0.20$, with the same prior probabilities and action values as in the case above. Its positive-result probability is $$ 0.35(0.60)+0.65(0.20)=0.21+0.13=0.34. $$

For a positive result, the joint expected value of proceeding is $0.21(150)+0.13(-100)=31.5-13=\$18.5$ million. The conditional expected value is $18.5/0.34=\$54.4118$ million, so the team would proceed. For a negative result, the probability is $0.66$, and the joint expected value of proceeding is $0.14(150)+0.52(-100)=21-52=-\$31$ million. The team would stop after a negative result. Therefore the gross EVSI is $\$18.5$ million. If this alternative also costs $\$18$ million and creates $\$5$ million in delay cost, its net value is $$ 18.5-18-5=-4.5\text{ million dollars}. $$ The less informative design would not be justified at those costs, even though it has positive gross information value.

Worked Example: Find the Break-Even Delay Cost

For the original design, gross EVSI is $\$25.75$ million and study cost is $\$18$ million. If $D$ is the expected delay cost in millions of dollars, the net value is $25.75-18-D=7.75-D$. The study has positive net value when $$ 7.75-D>0, $$ so the break-even delay cost is $\$7.75$ million. At that delay cost the decision-maker is indifferent; above it, the study’s net value is negative. The assumed delay cost of $\$5$ million is below the break-even value, giving the $\$2.75$ million net value calculated earlier.

A useful benchmark is the expected value of perfect information (EVPI): the expected value if the state were known before choosing an action, minus the best value using current information. In the original case, perfect information means proceeding in $F$ and stopping in $U$, for an expected value of $$ 0.35(150)+0.65(0)=52.5\text{ million dollars}. $$ The current best value is zero, so EVPI is $\$52.5$ million. The targeted study’s EVSI of $\$25.75$ million is below this upper bound. Perfect information is a benchmark, not a realistic substitute for evaluating a particular study: it cannot generally be purchased, and it does not account for the cost or delay of obtaining information.

Assumptions, Diagnostics, and Limitations

The calculations are only as credible as their inputs and decision structure. They assume that the prior state probabilities are appropriate for the decision population; the study-result probabilities correctly describe the proposed design; the listed utilities include all relevant consequences of proceeding and stopping; and the team will implement the action that maximizes expected utility after seeing the result. They also assume that study cost and delay are represented consistently and are not counted twice in the utilities.

Several checks help identify a fragile valuation:

Check probabilities. State probabilities must sum to one, and study-result probabilities conditional on each state must sum to one. The marginal probabilities of all possible results must also sum to one.
Check the decision rule in each branch. Compare the conditional expected value of every available action, including stopping, rather than assuming that a favorable-looking result automatically warrants proceeding.
Check whether the study can change the action. If the same action is optimal for every possible result, information has no decision value under the specified utilities, even if it reduces uncertainty.
Check the mapping from dose-finding results to the development state. A study may estimate dose response yet still have weak ability to distinguish the states that matter for the later decision. The result likelihoods should reflect that connection, not just statistical precision within the study.
Vary uncertain inputs. Recalculate under plausible alternative priors, study operating characteristics, utilities, costs, and delay assumptions. A positive net value in one scenario is not evidence that the decision is robust to all plausible assumptions.

EVSI is an expected value, not a guarantee that the study will produce a positive result, find an effective dose, or lead to a successful program. Its limitations include dependence on the chosen states and actions, possible misspecification of prior probabilities or study likelihoods, and uncertainty in future costs and benefits. If uncertainty about those inputs is material, present scenario or probabilistic analyses rather than relying on a single point estimate. The same study may have different value for a different decision, because utilities and available actions determine whether its results can change what the team should do.

Common Mistake

Pitfall: “The study is valuable because it reduces uncertainty, so its value is the expected improvement in precision.” Precision alone does not determine decision value. The correction is to calculate the best expected utility before the study and the expected best utility after each possible result. Only changes in decisions that improve expected outcomes contribute to EVSI; costs and delay are then deducted to assess net value.

Check Your Understanding

Answer each question using the definitions and decision logic above. Dollar amounts are hypothetical and in millions.

In the main case, what is the best current action, and what is its expected value?
For the less informative study, why is the negative-result action to stop even though the posterior probability of the favorable state is not zero?
For the original study, what is the maximum delay cost consistent with a positive net value if study cost remains $\$18$ million?
A new study result is equally likely under every state, and it does not affect utilities. What is its EVSI?

Solutions. (1) Proceeding has expected value $0.35(150)+0.65(-100)=-\$12.5$ million, while stopping is worth zero, so the best current action is to stop with expected value $\$0$ million. (2) The less informative study’s negative-result branch has joint expected value of proceeding $-\$31$ million and probability $0.66$, so its conditional value is $-31/0.66=-\$46.9697$ million. Stopping, worth zero, is better. (3) Gross EVSI minus study cost is $25.75-18=\$7.75$ million, so delay cost must be less than $\$7.75$ million for strictly positive net value; at $\$7.75$ million the team is indifferent. (4) If the result does not change the expected utilities or preferred action, the expected best value after the result equals the best value before it, so EVSI is zero.