Tutorials › Model-Informed Drug Development (MIDD) › Decomposing a Benefit-Risk Score

Quantitative Benefit-Risk Assessment · Tutorial 354 of 596

Decomposing a Benefit-Risk Score

You will calculate and interpret attribute-level contributions to a treatment utility difference, including how benefits and harms combine in a hypothetical comparison.

Advanced 12 min read

What You'll Learn

  • Define an attribute-level contribution to the difference in utility between treatments.
  • Calculate contributions using a common additive utility model and attribute scales.
  • Interpret how benefits and harms add up to a total treatment comparison.
  • Check that attribute utilities, weights, and treatment comparisons are aligned.
  • Recognize why contributions can depend on the utility model and its interactions.

Why Decompose a Treatment Utility Difference?

A total utility score can summarize a treatment comparison, but it may conceal the reasons one treatment scores higher. A decomposition shows how much each attribute contributes to the overall difference: for example, whether an efficacy gain outweighs a loss in tolerability, or whether a severe-harm reduction is the main driver. The result is an accounting of the specified model, not evidence by itself that a treatment causes an outcome or that the preferences in the model are correct.

The calculation below uses an additive multi-attribute utility model. For each treatment, each attribute is represented on a utility scale, with larger values defined as more desirable. This orientation matters: an adverse-event probability may increase while its corresponding desirability utility decreases.

Definition: Let \(A\) and \(B\) denote two treatments, \(u_j(A)\) and \(u_j(B)\) their utility scores on attribute \(j\), and \(w_j\) the weight for that attribute. In an additive model, the total utility for treatment \(t\) is \(U(t)=\sum_{j=1}^{J}w_j u_j(t)\). The attribute-level contribution to the difference favoring \(B\) over \(A\) is \(C_j=w_j[u_j(B)-u_j(A)]\). Thus, \(\Delta U=U(B)-U(A)=\sum_{j=1}^{J}C_j\). A positive \(C_j\) favors \(B\); a negative \(C_j\) favors \(A\).

Conditions for a Meaningful Decomposition

The decomposition is exact when the total score is additive in the stated attribute utilities. The attributes must be defined consistently for both treatments, and each \(u_j\) must use the same scale and direction in both treatment evaluations. The weights must also be the same for \(A\) and \(B\), and they must correspond to those attribute scales. In many normalized additive models, the weights are nonnegative and sum to 1; then the total is a weighted average of the attribute utilities. Other conventions are possible, but the interpretation of the score must match the model actually used.

In benefit-risk analysis, an attribute utility might be based on a treatment’s expected outcome, such as a response probability or the probability of avoiding a serious adverse event. The mapping from outcome to utility must be applied consistently. If the attribute is “serious adverse event risk,” for example, one can either define its utility to decrease as risk increases or define the attribute as “avoiding a serious adverse event,” with utility increasing as avoidance becomes more likely. Mixing these directions would reverse the meaning of a contribution.

A difference in utility is relative to the specified comparison and scoring system. It is not generally an absolute clinical effect, and it should not be interpreted without the attribute definitions, scales, weights, and direction of comparison. In particular, a numerical contribution is meaningful only on the model’s common total-utility scale.

Worked Example: Breaking Down a Weighted Score

Suppose a hypothetical comparison uses three normalized attribute utilities, each ranging from 0 to 1, with larger values preferred. The weights are 0.50 for efficacy, 0.25 for symptom improvement, and 0.25 for avoiding serious adverse events. The utility scores are:

AttributeWeightTreatment ATreatment BUtility difference, B − A
Efficacy0.500.600.720.12
Symptom improvement0.250.700.65−0.05
Avoiding serious adverse events0.250.950.84−0.11

For efficacy, the contribution is \(0.50(0.72-0.60)=0.50(0.12)=0.0600\). For symptom improvement, it is \(0.25(0.65-0.70)=0.25(-0.05)=-0.0125\). For avoiding serious adverse events, it is \(0.25(0.84-0.95)=0.25(-0.11)=-0.0275\).

Adding the contributions gives \(\Delta U=0.0600-0.0125-0.0275=0.0200\). Equivalently, \(U(A)=0.50(0.60)+0.25(0.70)+0.25(0.95)=0.7125\), while \(U(B)=0.50(0.72)+0.25(0.65)+0.25(0.84)=0.7325\); their difference is 0.0200 utility units. Under these hypothetical scores and weights, B has the higher total utility. Its efficacy advantage more than offsets its lower scores on the other two attributes.

The two negative contributions are not “negative outcomes” in general; they are negative contributions to the comparison favoring B. In particular, the score for avoiding serious adverse events is lower for B, so that attribute counts against B.

Why the Contributions Sum to the Total Difference

The additive model gives a direct justification. Write the total utility for each treatment using the same weights and subtract the two expressions. Distributing the weights across the differences yields the attribute contributions:

$$ \begin{aligned} \Delta U &= U(B)-U(A) \\ &= \sum_{j=1}^{J}w_j u_j(B)-\sum_{j=1}^{J}w_j u_j(A) \\ &= \sum_{j=1}^{J}w_j\bigl[u_j(B)-u_j(A)\bigr] \\ &= \sum_{j=1}^{J}C_j . \end{aligned} $$

This identity also provides a practical check: the sum of the displayed contributions must equal the total utility difference, up to rounding. If it does not, check for a sign error, a different set of weights, inconsistent attribute scores, or premature rounding. The equality follows from additivity; it does not establish that additivity is an appropriate description of preferences.

Worked Example: Translating Outcome Changes Into Contributions

Consider a hypothetical comparison with two attributes: response and avoiding a severe adverse event. The model assigns weights 0.65 and 0.35, respectively. Define response utility as the response probability divided by 0.40, and define severe-event-avoidance utility as \(1-p/0.20\), where \(p\) is the severe adverse-event probability. Suppose the hypothetical probabilities are:

  • Treatment A: response probability 0.24 and severe adverse-event probability 0.04.
  • Treatment B: response probability 0.30 and severe adverse-event probability 0.08.

The stated transformations give response utilities \(0.24/0.40=0.60\) for A and \(0.30/0.40=0.75\) for B. They give severe-event-avoidance utilities \(1-0.04/0.20=0.80\) for A and \(1-0.08/0.20=0.60\) for B. The transformation denominators and weights are part of this hypothetical model; they are not universal clinical standards.

The response contribution favoring B is \(0.65(0.75-0.60)=0.65(0.15)=0.0975\). The contribution from avoiding severe events is \(0.35(0.60-0.80)=0.35(-0.20)=-0.0700\). Thus, the response gain contributes 0.0975 utility units, while the higher severe-event risk contributes −0.0700 utility units.

The net difference is \(0.0975-0.0700=0.0275\). Direct calculation confirms this: \(U(A)=0.65(0.60)+0.35(0.80)=0.6700\), and \(U(B)=0.65(0.75)+0.35(0.60)=0.6975\). In this specified model, B has 0.0275 higher utility units. This conclusion is conditional on the hypothetical probabilities, transformations, and weights; it is not a stand-alone claim that B is clinically preferable in every decision context.

Reading the Pattern of Contributions

A decomposition helps distinguish a large attribute change from a large contribution. The size of \(C_j\) depends on both the change in attribute utility and its weight. A substantial improvement on a low-weight attribute may contribute less to the total than a modest change on a high-weight attribute. For that reason, report the utility difference and the weighted contribution rather than ranking attributes only by their raw changes.

Positive and negative contributions can offset. When the net difference is small, it may reflect close agreement across attributes, or large benefits and harms that nearly cancel. Those situations have different implications for understanding the comparison, even if the total scores are similar. Avoid reporting the percentage of the net difference “explained” by an attribute when positive and negative contributions nearly cancel: such percentages can exceed 100%, be negative, or become unstable when the net difference is near zero.

Worked Example: A Small Net Difference With Large Offsetting Contributions

Suppose a hypothetical additive model uses weights 0.40 for efficacy, 0.35 for symptom improvement, and 0.25 for avoiding a serious adverse event. Treatment A has attribute utilities 0.50, 0.60, and 0.90; Treatment B has utilities 0.60, 0.75, and 0.60, respectively. Larger values are preferred on all three scales.

The efficacy contribution is \(0.40(0.60-0.50)=0.0400\). The symptom-improvement contribution is \(0.35(0.75-0.60)=0.0525\). The contribution from avoiding serious adverse events is \(0.25(0.60-0.90)=-0.0750\). Therefore, \(\Delta U=0.0400+0.0525-0.0750=0.0175\).

The total scores are \(U(A)=0.40(0.50)+0.35(0.60)+0.25(0.90)=0.6350\) and \(U(B)=0.40(0.60)+0.35(0.75)+0.25(0.60)=0.6525\), also giving a difference of 0.0175. The harm-related contribution has the largest absolute magnitude, but the two positive contributions together are larger, leaving B with the higher total in this model. The net difference alone would not reveal this offsetting pattern.

Model Structure, Diagnostics, and Limitations

Before interpreting a decomposition, verify the inputs and the arithmetic. Confirm that the attribute utilities share their intended directions, that the same weights are applied to both treatments, and that the weights match the utility scales. Recalculate the total difference as the sum of contributions. If scores are estimated rather than fixed, distinguish the point-estimate decomposition from uncertainty in the scores or weights: a list of point contributions alone does not show how uncertain their signs or magnitudes are.

The main alternative to an additive model is a utility function with interactions among attributes, such as a multiplicative model. In that setting, the simple expression \(w_j[u_j(B)-u_j(A)]\) may not account for the total difference because the value of changing one attribute can depend on the levels of others. One exact option is to replace attributes sequentially: change one attribute from its value under A to its value under B while holding the other attributes at their current values. The resulting stepwise changes sum to the total difference, but the individual amounts can depend on the order of replacement. Averaging contributions over all replacement orders is another attribution convention; it is not the same as the unique, direct decomposition of an additive model.

A further alternative is to report the outcome differences themselves without converting them to utility contributions. That can make clinical quantities, such as response probability or adverse-event risk, easier to recognize, but it does not express how the specified preference weights combine those outcomes. A useful report can present both the attribute-level clinical changes and their utility contributions, while keeping those two scales distinct.

No decomposition validates the utility model, weights, or underlying outcome estimates. A precise sum can still be misleading if the attribute definitions omit relevant outcomes, the scales are poorly chosen, or the additive structure fails to represent preferences. Keep conclusions conditional: the decomposition identifies what drives the score under the specified model, not what must drive every patient’s or decision-maker’s preferences.

Common pitfall: A reader sees a positive total difference for B and concludes that every attribute favors B, or treats the largest raw utility change as the strongest driver. The correction is to calculate each \(C_j=w_j[u_j(B)-u_j(A)]\), preserve its sign, and compare weighted contributions. A positive total can include important negative contributions, and a large raw change can have a modest contribution if its weight is small.

Check Your Understanding

For each question, calculate or interpret the requested contribution. Solutions are included below.

  1. An additive model gives an efficacy weight of 0.60 and a safety weight of 0.40. The utility differences \(u(B)-u(A)\) are 0.08 for efficacy and −0.10 for safety. What are the contributions and total difference?
  2. A model has three weights, 0.50, 0.30, and 0.20. The utility differences for B versus A are 0.06, −0.10, and −0.05. Calculate the total utility difference and state which treatment has the higher score.
  3. A safety attribute is defined as avoiding an adverse event, with utility \(u=1-p/0.10\). The adverse-event probability increases from 0.02 under A to 0.04 under B, and the attribute weight is 0.25. What is the safety contribution favoring B?
  4. Two attributes contribute +0.12 and −0.10 to the difference favoring B. What is the total difference, and what does the negative contribution mean?

Solutions. (1) Efficacy contributes \(0.60(0.08)=0.048\), and safety contributes \(0.40(-0.10)=-0.040\). The total is \(0.048-0.040=0.008\), so B has the higher score by 0.008 utility units. (2) The contributions are \(0.50(0.06)=0.030\), \(0.30(-0.10)=-0.030\), and \(0.20(-0.05)=-0.010\). The total is \(-0.010\), so A has the higher score by 0.010 utility units. (3) The safety utilities are \(1-0.02/0.10=0.80\) for A and \(1-0.04/0.10=0.60\) for B. The contribution is \(0.25(0.60-0.80)=-0.050\) utility units; the increased event probability counts against B. (4) The total is \(0.12-0.10=0.02\) utility units favoring B. The negative contribution means that this attribute favors A and offsets part of the positive contribution.