Why subgroup differences can change a benefit-risk decision
A treatment can have an attractive average benefit-risk profile while offering little benefit, or even net harm, to some patients. Conversely, a modest average effect may conceal a subgroup with a more favorable profile. The decision therefore depends on more than whether subgroup estimates differ: it depends on the size and uncertainty of each subgroup's efficacy and safety effects, the population proportions, and the benefit-risk weights used to compare outcomes.
This tutorial uses a hypothetical two-subgroup case. Efficacy is expressed as the treatment-related increase in response rate, in percentage points. Safety is expressed as the treatment-related increase in adverse-event rate, also in percentage points. The weights are treated as already specified for the decision; they are not estimated from the trial.
The weights encode how efficacy and safety outcomes are valued relative to one another. In the case below, \(w_B=1\) and \(w_H=2\): a one-percentage-point increase in adverse events counts as two utility units of harm, while a one-percentage-point increase in response counts as one utility unit of benefit. Thus utility is in decision units, not in percentage points of response or adverse events. A positive utility is favorable under these weights; it is not by itself proof that treatment is clinically worthwhile under every relevant criterion.
Combining effects and uncertainty
Subgroup effects should be defined on the same scale and for a clinically interpretable estimand. For example, the differences in response and adverse-event rates here are treatment minus control absolute risk differences. The population proportions should refer to the population in which the policy would be used. A randomized comparison within each subgroup supports a treatment-effect interpretation under the trial's assumptions, but a subgroup association is not automatically evidence that subgroup membership causes different treatment response.
Uncertainty must be propagated through the same weights and aggregation used for the decision. Suppose the posterior distribution of the efficacy and safety effects in subgroup \(g\) has variances \(V(B_g)\) and \(V(H_g)\), and covariance \(C(B_g,H_g)\). Then:
The covariance term matters because efficacy and safety estimates may move together or in opposite directions across posterior draws. For a weighted average of subgroup utilities, the general variance is:
If subgroup utilities are independent, this reduces to \(\sum_g\pi_g^2V(U_g)\). Independence is a simplifying assumption, not a default fact. In particular, estimates based on a shared control group can be correlated. A joint model or joint posterior draws preserve these dependencies and allow each policy utility to be calculated for every draw.
Worked Example: Net Utility Within Each Subgroup
In a hypothetical randomized trial, subgroup A has posterior mean efficacy benefit \(B_A=12\) percentage points with posterior standard deviation 3 points, and posterior mean safety harm \(H_A=4\) points with standard deviation 2 points. Subgroup B has \(B_B=5\) points with standard deviation 2.5 points, and \(H_B=3\) points with standard deviation 1.5 points. For this calculation, assume the posterior efficacy and safety effects are independent within each subgroup. Use \(w_B=1\) and \(w_H=2\).
For subgroup A, the posterior mean net utility is: $$ E(U_A)=1(12)-2(4)=4\text{ utility units}. $$ Under the stated independence assumption, its posterior variance and standard deviation are: $$ V(U_A)=1^2(3^2)+2^2(2^2)=9+16=25,\qquad SD(U_A)=\sqrt{25}=5. $$
For subgroup B: $$ E(U_B)=1(5)-2(3)=-1\text{ utility unit}, $$ $$ V(U_B)=1^2(2.5^2)+2^2(1.5^2)=6.25+9=15.25,\qquad SD(U_B)=\sqrt{15.25}=3.905. $$
The posterior means favor treatment in A and disfavor it in B under these weights. However, the standard deviations are large relative to the means. The signs of the posterior means alone do not show that either subgroup decision is certain.
Why the variance formula has this form
The variance calculation follows from the general identity \(V(aX+bY)=a^2V(X)+b^2V(Y)+2ab\,\mathrm{Cov}(X,Y)\). Substituting \(a=w_B\), \(b=-w_H\), \(X=B_g\), and \(Y=H_g\) gives the negative covariance term in \(V(U_g)\). If efficacy and harm estimates have positive covariance, high-benefit draws also tend to have high-harm draws, reducing the variance of their difference relative to the independent case. Negative covariance has the opposite effect. This is why separate standard errors alone are insufficient unless their covariance is also known or reasonably assumed to be zero.
Worked Example: From Subgroup Effects to a Treat-All Policy
Suppose subgroup A makes up 40% of the intended population and subgroup B makes up 60%. Use the posterior means from the preceding example. First calculate the population-average utility of treating everyone: $$ E(U_{\mathrm{all}})=0.40(4)+0.60(-1)=1.60-0.60=1.00\text{ utility unit per person}. $$
For a transparent uncertainty calculation, additionally assume the subgroup utility estimates are independent and that the subgroup proportions are known. Then: $$ V(U_{\mathrm{all}})=0.40^2(25)+0.60^2(15.25)=4.00+5.49=9.49, $$ $$ SD(U_{\mathrm{all}})=\sqrt{9.49}=3.081\text{ utility units}. $$
The treat-all policy has positive posterior mean utility, but its uncertainty is substantial. Treating only A gives: $$ E(U_{\mathrm{target\ A}})=0.40(4)=1.60\text{ utility units per person in the full population}. $$ This policy has a higher expected utility than treating everyone in this case because it avoids treating subgroup B, whose posterior mean utility is negative. It does not follow that targeting is certainly preferable: the estimates in both subgroups remain uncertain.
Worked Example: Quantifying Decision Uncertainty
Continue the hypothetical case and approximate the posterior distributions of subgroup utilities as normal, using the means and standard deviations already calculated. For a normal distribution with mean \(m\) and standard deviation \(s\), the probability that utility is positive is \(\Phi(m/s)\), where \(\Phi\) is the standard normal cumulative distribution function.
For subgroup A: $$ P(U_A>0\mid\text{data})\approx\Phi(4/5)=\Phi(0.800)=0.788. $$ For subgroup B: $$ P(U_B>0\mid\text{data})\approx\Phi(-1/3.905)=\Phi(-0.256)=0.399. $$ For the treat-all policy: $$ P(U_{\mathrm{all}}>0\mid\text{data})\approx \Phi(1/3.081)=\Phi(0.325)=0.627. $$
These are posterior probabilities under the specified model and assumptions. They are not probabilities that a null hypothesis is true, nor are they guarantees of a favorable outcome for an individual patient. They describe uncertainty about the decision utilities given the data and model.
The difference between targeting A and treating everyone is \(U_{\mathrm{target\ A}}-U_{\mathrm{all}}=-0.60U_B\): the contribution from A cancels, and the policy difference depends on whether treating B adds or subtracts utility. Its posterior mean is \(-0.60(-1)=0.60\), and its standard deviation is \(0.60(3.905)=2.343\). Thus: $$ P(U_{\mathrm{target\ A}}>U_{\mathrm{all}}\mid\text{data}) \approx\Phi(0.60/2.343)=\Phi(0.256)=0.601. $$ Targeting A has the largest expected utility among these three policies (treat all, target A, or treat nobody), but the probability that it outperforms treating everyone is only about 0.601 under this approximation. The policy ranking is therefore uncertain.
From estimates to an actionable decision
The calculations above use posterior means to rank policies by expected utility and posterior probabilities to show how uncertain that ranking is. A decision maker may set an additional evidentiary or clinical requirement—for example, requiring a high posterior probability of positive utility before recommending treatment. That requirement is distinct from choosing the benefit-risk weights. It should be specified in a way that matches the decision context rather than inferred after seeing the subgroup results.
A subgroup-targeted policy is meaningful only if subgroup membership can be determined reliably and the policy can be implemented. It also needs a clinically coherent definition established before the decision is made. A subgroup that is easy to identify but very small may have little population impact; a subgroup with a favorable estimated utility may be impractical to distinguish in routine care. These considerations affect the feasible policy set, even though they do not change the utility arithmetic.
Specify the efficacy and safety estimands, their directions, subgroup definitions, relevant population proportions, and benefit-risk weights.
Use the joint distribution of efficacy and safety effects, including covariance within and across subgroups, rather than combining marginal summaries as if they were automatically independent.
For each posterior draw, calculate utility for treating everyone, treating a specified subgroup, and any other available policy.
Compare posterior mean utilities and report quantities such as the probability of positive utility or the probability one policy has higher utility than another.
Diagnostics, alternatives, and limitations
Several checks can reveal when a subgroup-based conclusion is not reliable. Confirm that the efficacy and safety effects use compatible populations, comparators, follow-up periods, and definitions. Inspect posterior distributions, not only means and standard deviations: a normal approximation can be misleading when effects are skewed, bounded, or multimodal. Check the estimated covariance structure, particularly when a common control group or a joint model links subgroup estimates. Recalculate utilities from joint posterior draws when available; this also accommodates non-normal distributions without relying on the normal approximation used in the examples.
Large subgroup uncertainty is itself a decision-relevant result. Small subgroups often have imprecise estimates, and selecting a subgroup because its observed estimate looks unusually favorable can exaggerate the apparent contrast. A formal treatment-by-subgroup interaction may help assess whether effects differ, but a difference between subgroup estimates is not established merely because one subgroup's estimate is statistically significant and the other's is not. Conversely, a failure to establish interaction does not prove that effects are identical. The decision should reflect the uncertainty in the subgroup effects and the consequences of the available policies.
| Approach | What it provides | Main caution |
|---|---|---|
| Population-average benefit-risk | A single utility for treating the intended population as a whole | Can hide important subgroup differences |
| Separate subgroup estimates | Utility and uncertainty for each defined subgroup | May be unstable when subgroup data are sparse; estimates can be dependent |
| Joint posterior policy comparison | Expected utility and probabilities comparing feasible policies while retaining estimated dependence | Results depend on the model, priors, estimands, and specified weights |
No calculation repairs poorly defined endpoints, biased subgroup comparisons, missing safety follow-up, or an unrepresentative trial population. If subgroup proportions are uncertain or differ between the trial and intended-use population, their uncertainty should also be incorporated. The worked case treats those proportions as known, uses normal posterior approximations, and omits implementation costs and other outcomes. Those simplifications make the arithmetic clear but constrain how strongly the numerical result can support a real decision.
Common mistake
Wrong reasoning: “Treatment is beneficial in subgroup A because its estimated utility is positive, and not beneficial in B because its estimate is negative; therefore targeting A is clearly better.” This treats uncertain estimates as known values and ignores how much of the population is in each subgroup.
Correction: Calculate utilities using the stated weights and population proportions, then propagate uncertainty through the same calculations. In the case above, targeting A has higher expected utility than treating everyone, but the posterior probability that it is better is only about 0.601 under the stated approximation. The conclusion is a preference in expected utility, not certainty about the best policy.
Check Your Understanding
Try each question, then compare with the solutions below.
- Using \(w_B=1\), \(w_H=2\), \(B=8\) percentage points, and \(H=2\) percentage points, calculate subgroup utility.
- If subgroup proportions are 25% and 75%, and their utilities are 6 and \(-2\), calculate the treat-all utility and the utility of targeting only the first subgroup.
- Why is it generally incorrect to calculate population utility uncertainty using only the separate subgroup standard deviations while assuming the subgroup estimates are independent?
- For a posterior normal utility with mean 2 and standard deviation 4, calculate the approximate posterior probability that utility is positive.
- Explain why “significant in one subgroup but not significant in another” does not, by itself, establish that treatment effects differ between subgroups.
Solutions. (1) \(U=1(8)-2(2)=4\) utility units. (2) Treat-all utility is \(0.25(6)+0.75(-2)=1.5-1.5=0\); targeting the first subgroup gives \(0.25(6)=1.5\) utility units per person in the full population, with no treatment contribution from the second subgroup. (3) Subgroup estimates may be correlated, for example through a shared control group; the variance of a weighted sum includes covariance terms. (4) \(P(U>0\mid\text{data})=\Phi(2/4)=\Phi(0.500)\approx0.691\). (5) The two significance decisions depend on each estimate's precision as well as its size. A direct comparison of subgroup effects, with its uncertainty, is needed to assess evidence of a difference.