Why the Source of Preference Weights Matters
A benefit-risk analysis combines two different kinds of information. Clinical evidence describes what may happen under each treatment; preference weights describe how important the specified outcomes are to a decision perspective. Those perspectives need not assign the same importance to benefit and harm. A treatment with a modest benefit and a more frequent adverse effect could therefore be preferred by one group and not another, even when everyone uses the same clinical evidence.
Patients can speak to the importance of outcomes in lived experience and treatment decisions. Clinicians contribute a professional perspective informed by clinical practice and the consequences of outcomes. A population sample can represent public preferences relevant to decisions such as allocating health resources. None of these perspectives is automatically the correct one for every decision. The appropriate perspective depends on whose decision is being informed and what decision-making authority or responsibility is relevant.
The introductory material on eliciting patient preferences for benefit and harm establishes how preferences can be gathered. Here the focus is on comparing weights from different sources and tracing their consequences through a quantitative decision. The examples use a deliberately simple two-outcome score to make that comparison transparent; they are not a general multi-criteria decision model.
Make the Perspectives Comparable
A comparison is interpretable only if the elicitation tasks are sufficiently aligned. Groups should consider the same outcomes, outcome definitions, reference levels, and trade-off question. If patients value “avoiding severe nausea” while clinicians value “avoiding any adverse event,” a numerical difference between their weights may reflect different definitions rather than different preferences. Similarly, weights cannot be interpreted independently of the outcome scales and the task used to elicit them.
For the worked calculations, benefit and harm are expressed as expected scores on a shared 0-to-10 scale: a higher benefit score is better, while a higher harm-burden score is worse. Weights are nonnegative and sum to one. These are stated conventions for the examples, not a claim that every elicitation method produces weights on this scale. The clinical evidence and score construction are held fixed when comparing perspectives.
Worked Example: Summarizing Elicited Weights
Suppose a small, hypothetical elicitation exercise asks each participant to allocate 100 points between the importance of a benefit and the importance of a harm. The four patients allocate 80, 70, 60, and 70 points to benefit. The four clinicians allocate 50, 40, 30, and 50 points to benefit. The four population-sample participants allocate 60, 55, 50, and 45 points to benefit. The remaining points in each allocation go to harm.
For each source, first calculate the mean benefit allocation. For patients, the mean is \((80+70+60+70)/4=70\) points. The corresponding mean harm allocation is \(100-70=30\) points, so the illustrative patient weights are \(w_B=0.70\) and \(w_H=0.30\).
For clinicians, the mean benefit allocation is \((50+40+30+50)/4=42.5\) points, giving \(w_B=0.425\) and \(w_H=0.575\). For the population sample, the mean benefit allocation is \((60+55+50+45)/4=52.5\) points, giving \(w_B=0.525\) and \(w_H=0.475\).
These small groups are hypothetical and far too small to represent real stakeholder populations. The calculation illustrates a descriptive comparison: the patient group assigns more weight to benefit, while the clinician group assigns more to harm. The group means do not show how much individual preferences vary or establish that one group’s preferences should govern a decision.
Applying Each Perspective to the Same Evidence
For the examples below, let \(\Delta B\) denote the difference in expected benefit score between a new treatment and a comparator, and let \(\Delta H\) denote the corresponding difference in expected harm-burden score. Positive \(\Delta H\) means the new treatment has greater expected harm burden. For a perspective with benefit weight \(w_B\) and harm weight \(w_H\), define the treatment’s score difference as:
A positive \(\Delta V\) favors the new treatment under this scoring rule; a negative value favors the comparator. This is a deterministic comparison of expected scores under specified inputs. It does not, by itself, account for uncertainty in clinical estimates or preference weights.
Worked Example: Comparing Three Perspectives
In a hypothetical comparison, a new treatment increases the response rate from 50% to 58%. Define the benefit score as 10 for a response and 0 for no response. The expected benefit score increases by \(10(0.58-0.50)=0.80\) points. Suppose the treatment also increases the rate of a specified adverse event from 10% to 20%, with a harm-burden score of 10 for an event and 0 otherwise. The expected harm-burden score increases by \(10(0.20-0.10)=1.00\) point. Thus \(\Delta B=0.80\) and \(\Delta H=1.00\), in score points.
Use the illustrative weights from the preceding example. Under the patient perspective, \(w_B=0.70\) and \(w_H=0.30\):
The positive score difference favors the new treatment by 0.26 weighted score points under this perspective.
For clinicians, \(w_B=0.425\) and \(w_H=0.575\):
The negative score difference favors the comparator by 0.235 weighted score points under the clinician perspective.
For the population sample, \(w_B=0.525\) and \(w_H=0.475\):
This score difference also favors the comparator, but only slightly. The treatment ranking therefore changes across perspectives: patients favor the new treatment, while the clinician and population perspectives favor the comparator. The calculations do not imply that the patient perspective is more or less valid; they show that the decision depends on the perspective chosen.
Finding When the Ranking Reverses
A threshold calculation makes the source of a reversal explicit. In the preceding example, suppose the benefit weight is \(w_B\), and the harm weight is \(1-w_B\). The score difference is zero when neither treatment is favored by this rule:
Expanding and solving gives \(0=0.80w_B-1.00+1.00w_B=1.80w_B-1.00\), so \(w_B=1.00/1.80=0.5556\). For these particular outcome differences, a benefit weight above 0.5556 favors the new treatment; a weight below 0.5556 favors the comparator. The patient weight of 0.70 lies above the threshold, whereas the clinician weight of 0.425 and population weight of 0.525 lie below it. The population result is relatively close to the threshold, which signals that its ranking may be sensitive to small changes in inputs.
Worked Example: Checking a Near-Tie
Suppose a second hypothetical comparison has a benefit increase of 1.20 score points and a harm-burden increase of 1.00 point. The population weights are \(w_B=0.525\) and \(w_H=0.475\). Then:
The new treatment has a positive score difference of 0.155 weighted score points. To find the threshold, solve \(0=w_B(1.20)-(1-w_B)(1.00)\). Rearranging gives \(2.20w_B=1.00\), or \(w_B=0.4545\). Since 0.525 is above 0.4545, the positive score is consistent with the threshold result. This check is useful near a decision boundary: it verifies both the arithmetic and how far the selected weight lies from a reversal.
Choosing and Reporting a Perspective
A perspective should be selected because it fits the decision context, not because it produces a preferred treatment ranking. A patient-level treatment decision may call for patient preferences, while a decision about public resource allocation may require population preferences. Clinician preferences can be relevant to clinical judgment, but should not be silently substituted for the preferences of patients or the public. Where a decision has several legitimate audiences, reporting more than one perspective can make the normative choice visible.
| Perspective | Potential relevance | Question to examine |
|---|---|---|
| Patients | Individual treatment choices and outcomes experienced by people with the condition | Do participants reflect the patients facing this decision and its treatment options? |
| Clinicians | Professional decisions informed by clinical practice | Does the elicitation task capture preferences, rather than only clinical judgments about outcome frequency or severity? |
| Population sample | Public or system-level decisions affecting a broader population | Does the sample and framing represent the population relevant to the decision? |
The table identifies possible uses, not a hierarchy. The decision maker should state whose preferences are used, why that source matches the decision, how participants were recruited, and what outcome definitions and elicitation task they received. If a conclusion changes across sources, report that change directly rather than presenting one result as a purely technical answer.
Diagnostics, Alternatives, and Limitations
A weight comparison is going wrong, or at least becoming hard to interpret, when the groups did not answer the same question; when outcome labels, reference levels, or scales differ; or when the weights are applied to outcome estimates defined in a different way. Check that benefit and harm point in the stated directions, that the weights match the calculation, and that the result is consistent with the threshold implied by the inputs. Also inspect whether the estimated score difference is large relative to the distance from a reversal threshold.
A common mistake is to treat preference weights as clinical evidence. For example, a higher patient weight on harm does not mean patients experience a higher adverse-event rate. The correction is to keep the evidence and the value judgment separate: use clinical data to estimate outcome differences, and use elicited weights to represent how the specified perspective values those differences.
Another mistake is to average patient, clinician, and population weights automatically and call the result neutral. Such an average assigns an implicit role to each source, and the result may represent no actual decision perspective. If a combined perspective is required, the rationale for combining sources must be stated; otherwise, report the perspectives separately. A further pitfall is to treat a small ranking difference as decisive. A result close to zero may be reversed by modest changes in weights or clinical estimates, so its fragility should be reported.
Alternative approaches include selecting one prespecified perspective, presenting parallel results for several perspectives, or using a deliberative process to determine which perspective is appropriate for a decision. These approaches answer different governance questions; none removes the need to disclose the chosen outcomes, evidence, and weights. The simple score used here has important limitations: it considers only two outcomes, assumes the stated score scales are suitable for the calculation, and omits uncertainty in both evidence and preferences. Group means also conceal individual variation. The calculations are therefore a transparent comparison tool, not a substitute for a complete decision process.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- Why should outcome definitions and scales be aligned before comparing patient and clinician weights?
- A treatment has \(\Delta B=0.50\), \(\Delta H=0.40\), \(w_B=0.60\), and \(w_H=0.40\). Calculate \(\Delta V\) and state which option the score favors.
- For \(\Delta B=0.50\) and \(\Delta H=0.40\), find the benefit-weight threshold when \(w_H=1-w_B\).
- Why is the unexamined average of patient and population weights not automatically a neutral choice?
Solutions. (1) Otherwise, observed weight differences may reflect different outcome meanings or trade-off tasks rather than different preferences. (2) \(\Delta V=(0.60)(0.50)-(0.40)(0.40)=0.30-0.16=0.14\) weighted score points. The positive result favors the treatment. (3) Set \(0=0.50w_B-0.40(1-w_B)=0.90w_B-0.40\), giving \(w_B=0.40/0.90=0.4444\). A benefit weight above this threshold favors the treatment under the stated rule. (4) Averaging gives each source an implicit role and may describe neither group nor the decision maker’s intended perspective; the reason for combining them must be justified.