Eliciting Patient Preferences for Benefit and Harm

From Trade-Off Responses to Preference Weights

Clinical outcomes often differ in more than one way. A treatment may improve symptom control while also increasing the chance of an adverse event. To use patient preferences in a benefit-risk assessment, it is not enough to list those outcomes: we need a systematic way to ask how people value them relative to one another. Elicitation methods do this by presenting choices and translating a respondent’s stated indifference or observed choice into a utility or preference weight.

The methods below answer related but distinct questions. A standard gamble asks how much risk of a specified worse outcome a person would accept to avoid a health state. A time trade-off asks how much time in full health is equivalent to a longer time in a specified health state. A discrete-choice experiment infers preference coefficients from repeated choices among alternatives described by attributes. These are not interchangeable scales: their values depend on the outcomes, framing, and assumptions used.

Definition: An elicited utility is a numerical representation of preference for an outcome under a specified model or elicitation task. A preference weight is a coefficient or value used to represent the relative importance of an outcome attribute. Its interpretation depends on the method and scale that produced it; it is not automatically a probability, a clinical effect size, or a universal patient value.

Standard Gamble

In a standard gamble, the respondent considers a specified health state and a gamble with two possible outcomes: full health with probability \(p\), and death with probability \(1-p\). The respondent varies \(p\) until indifferent between living in the specified state and taking the gamble. Under expected-utility theory, with full health assigned utility 1 and death assigned utility 0, the gamble’s expected utility is \(p\). At indifference, the utility of the health state therefore equals \(p\).

u_{\mathrm{state}} = p(1) + (1-p)(0) = p

This calculation assumes the respondent understands and accepts the stated probabilities and outcomes, can express a stable preference, and is indifferent at the reported probability. It also uses the specified anchors and expected-utility representation. With these anchors, the resulting utility lies between 0 and 1. A design intended to measure states considered worse than death requires a different elicitation procedure and cannot be interpreted by simply applying this two-outcome formula.

Worked Example: Standard-Gamble Indifference

Suppose, in a hypothetical interview, a patient considers a specified chronic symptom state and a gamble offering full health with probability \(p\) and death with probability \(1-p\). The interviewer varies \(p\), and the patient reports indifference at \(p=0.72\). Assume the patient understands the gamble and the full-health and death anchors are scored 1 and 0.

The expected utility of the gamble at indifference is \(0.72(1)+0.28(0)=0.72\). Thus the elicited utility of the specified state is \(u_{\mathrm{state}}=0.72\) on these anchors. This is the utility implied by this response and model, not a statement that there is a 72% chance the health state is preferred or that 72% of patients would choose it.

Time Trade-Off

In a time trade-off (TTO), a respondent compares a fixed duration \(T\) in a specified health state with a shorter duration \(x\) in full health. Both options end in death, and the respondent is asked to identify the duration \(x\) that makes the options equally preferred. If utility is constant over time within each state, utility is proportional to time, and time preference is ignored, the indifference condition is \(T u_{\mathrm{state}}=x(1)\). Therefore, the state utility is \(x/T\).

T u_{\mathrm{state}} = x \qquad\Longrightarrow\qquad u_{\mathrm{state}} = \frac{x}{T}

The ratio is meaningful only under those assumptions and for the stated durations and health state. In particular, discounting, adaptation, changing symptoms over time, or a strong response to the difference in survival duration can affect the answer. The TTO value is not a direct measurement of a treatment’s clinical effect.

Worked Example: Time Trade-Off for a Symptom State

In a hypothetical TTO task, a respondent compares 10 years in a specified symptom state followed by death with \(x\) years in full health followed by death. The respondent is indifferent when \(x=7.5\) years. Assume constant utility within each state, linear utility in time over this range, and no discounting.

Substituting the durations into the TTO equation gives \(u_{\mathrm{state}}=x/T=7.5/10=0.750\). The elicited utility is 0.750 on the full-health-equals-1 and death-equals-0 scale under the stated assumptions. If the respondent’s preferences change with duration or if time is discounted, this ratio need not represent the same utility.

Discrete-Choice Methods

A discrete-choice experiment presents alternatives defined by attributes, such as symptom improvement and adverse-event risk, and records which alternative the respondent chooses. In a conditional-logit formulation, the utility of alternative \(j\) for respondent \(i\) is represented as a systematic component \(V_{ij}\) plus an unobserved error \(\epsilon_{ij}\). For example, \(V_{ij}\) may be a linear function of the alternative’s attribute levels. If the errors are independent and identically distributed with a type-I extreme-value distribution, the choice probability has the conditional-logit form below.

P(i\text{ chooses }j) = \frac{\exp(V_{ij})}{\sum_k \exp(V_{ik})} \qquad\text{with}\qquad V_{ij}=\sum_r \beta_r x_{ijr}

Here \(x_{ijr}\) is the level of attribute \(r\) for alternative \(j\), and \(\beta_r\) is its coefficient. The model’s error scale is normalized, so coefficients are interpreted relative to that scale. Adding the same constant to every alternative’s \(V_{ij}\) for a choice task leaves its probabilities unchanged, because the common factor cancels from numerator and denominator. Multiplying coefficients alone generally changes predicted probabilities; it is not a harmless rescaling of the fitted model.

For two alternatives, the log of the odds of choosing alternative A rather than B equals \(V_{iA}-V_{iB}\). This identity makes a simple binary-choice illustration transparent. In a real study, coefficients are usually estimated jointly by maximizing the likelihood across all choice tasks, and uncertainty and design quality matter.

\log\left(\frac{P(i\text{ chooses A})}{P(i\text{ chooses B})}\right) =V_{iA}-V_{iB}

Worked Example: Coefficients From Simple Choice Contrasts

Consider hypothetical choices from respondents in two sets of binary tasks. In the first set, alternatives differ only in a benefit attribute, coded so that A has one unit more benefit than B. Fifteen of 20 choices favor A. In the second set, alternatives differ only in a harm attribute, coded so that A has one unit more harm than B. Five of 20 choices favor A. For this illustration, assume each set can be treated as repeated observations of its stated binary contrast under the same conditional-logit scale.

For the benefit contrast, the observed choice proportion for A is \(15/20=0.75\), so the observed odds are \(0.75/(1-0.75)=3\). With a one-unit difference in benefit and no other difference, the fitted coefficient is the log odds: \(\hat{\beta}_{\mathrm{benefit}}=\log(3)=1.099\).

For the harm contrast, A is chosen with proportion \(5/20=0.25\), giving odds \(0.25/0.75=1/3\). Since A has one additional unit of harm, \(\hat{\beta}_{\mathrm{harm}}=\log(1/3)=-1.099\).

Under this coding and model, one benefit unit offsets one harm unit in systematic utility, because \(-\hat{\beta}_{\mathrm{harm}}/\hat{\beta}_{\mathrm{benefit}}=1.00\) benefit units per harm unit. This is a model-based trade-off, not a probability or an absolute utility anchored to full health and death. The estimates are only illustrative: the two small contrast sets would have substantial sampling uncertainty, and broader conclusions require an appropriate choice design and model assessment.

Justification and Interpretation of Choice Coefficients

For a binary choice with alternatives A and B, let the systematic utility difference be \(\Delta V=V_A-V_B\). The conditional-logit formula gives \(P(A)=\exp(V_A)/[\exp(V_A)+\exp(V_B)]\). Dividing this probability by \(P(B)\) cancels the shared denominator, yielding \(\exp(V_A-V_B)\). Taking logarithms gives \(\log[P(A)/P(B)]=\Delta V\). If the alternatives differ by one unit in a single attribute and the model is correctly specified, the coefficient for that attribute equals the log odds of choosing the alternative with that extra unit.

When attributes are coded so that larger \(x_{\mathrm{benefit}}\) means more benefit and larger \(x_{\mathrm{harm}}\) means more harm, a reasonable directional pattern is \(\beta_{\mathrm{benefit}}>0\) and \(\beta_{\mathrm{harm}}<0\). If both coefficients are estimated on the same model scale, the ratio \(-\beta_{\mathrm{harm}}/\beta_{\mathrm{benefit}}\) gives the number of benefit units that offset one harm unit in systematic utility, provided the linear additive specification is appropriate. Its units depend on the attribute coding. It does not supply an absolute utility anchor.

The conditional-logit probabilities are sensitive to coefficient magnitude. For a two-alternative task with \(\Delta V=1\), the predicted probability of choosing the higher-utility alternative is \(\exp(1)/[1+\exp(1)]=0.731\). If the coefficient difference is doubled, so that \(\Delta V=2\), the probability becomes \(\exp(2)/[1+\exp(2)]=0.881\). This is why multiplying all coefficients by a positive constant while retaining the same normalized error scale changes predicted choices.

Choosing a Method and Checking Responses

Standard gamble, TTO, and discrete choice differ in the trade-off they ask respondents to make. Standard gamble explicitly involves risk and can be affected by risk attitudes. TTO involves survival duration and can be affected by time preference or by the way duration is framed. Discrete choice avoids asking respondents to state a numerical utility directly, but its coefficients depend on attribute definitions, level ranges, experimental design, and the assumed choice model. A choice coefficient from a discrete-choice model is not directly comparable to a standard-gamble utility.

Before interpreting elicited values, examine whether respondents understood the task and whether answers are coherent across repeated or related questions. For example, check whether a respondent gives systematically different indifference answers after reversing the order of presentation, or whether apparently dominated alternatives are chosen. In a discrete-choice analysis, inspect whether attribute coding and estimated coefficient directions make clinical sense, whether model fit is poor, and whether predictions are implausible for observed tasks. Large uncertainty, sparse choices at some levels, or strong differences across respondents can make an average coefficient a poor summary.

Framing and comprehension: probability wording, descriptions of adverse events, and order of alternatives can affect responses. Use clear, balanced descriptions and comprehension checks.
Inconsistent answers: a single anomalous response can arise from error or genuine preference complexity. Do not silently discard it; investigate patterns and report the handling rule.
Model misspecification: a linear additive utility model can miss nonlinear attribute effects or interactions. Check whether the task design and data support the assumed form.
Scale and scope: elicited weights pertain to the described outcomes, population, and task. They should not be treated as universal constants or as evidence that one clinical outcome causes another.

Common pitfall: “All the choice-model coefficients can be multiplied by the same positive constant without changing predicted choices.” This is false for the conditional-logit probabilities above: multiplying coefficients changes utility differences and generally changes probabilities. Only adding a common constant to all alternatives’ utilities within a task leaves those probabilities unchanged. Coefficients are tied to the model’s normalized error scale.

A practical elicitation plan should select the method that matches the intended interpretation, define outcomes in clinically understandable terms, and specify the anchors or attribute coding in advance. Report the task, assumptions, and units alongside the resulting values. The same numerical weight can imply different preferences if the attribute definitions or model scale differ.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

In a standard gamble between a health state and a lottery offering full health with probability \(0.64\) and death otherwise, what state utility follows under the stated anchors and indifference assumption?
A respondent is indifferent between 8 years in a health state and 6 years in full health. Under the standard TTO assumptions, calculate the state utility.
In a binary choice model, alternative A has systematic utility 0.8 and B has utility 0.2. What are the log odds of choosing A, and approximately what is its choice probability?
Why does multiplying every conditional-logit coefficient by two generally change predicted choice probabilities?
If \(\beta_{\mathrm{benefit}}=0.8\) and \(\beta_{\mathrm{harm}}=-1.2\), with benefit and harm each coded in one-unit increments, how many benefit units offset one harm unit under the additive model?

Solutions. (1) \(u_{\mathrm{state}}=0.64(1)+0.36(0)=0.640\), assuming indifference and the stated expected-utility anchors. (2) \(u_{\mathrm{state}}=x/T=6/8=0.750\), under constant utility over time and no discounting. (3) The log odds are \(V_A-V_B=0.8-0.2=0.6\). Thus \(P(A)=\exp(0.6)/[1+\exp(0.6)]=0.646\), approximately. (4) Doubling coefficients doubles utility differences, which changes the exponentials in the conditional-logit formula; unlike adding a common constant, this does not cancel. (5) The offset is \(-\beta_{\mathrm{harm}}/\beta_{\mathrm{benefit}}=1.2/0.8=1.50\) benefit units per harm unit, conditional on the additive model and the specified coding.