Joint Uncertainty in Benefits and Harms

Why Joint Uncertainty Matters

A benefit-risk assessment can report an average efficacy outcome and an average safety outcome, yet still miss how uncertain their combination is. If model parameters governing efficacy and harm are associated, or if the outcomes themselves are associated, replacing their joint distribution with two separate marginal distributions loses information. That information can materially change the uncertainty in net utility, even when the marginal means are unchanged.

The aim here is to construct a predictive distribution for efficacy and safety outcomes together, and propagate it through a specified utility function. As in the earlier material on separating parameter and preference uncertainty, preferences can be treated as fixed for this calculation. Let $w_B$ and $w_H$ denote fixed weights for benefit and harm; the focus is on preserving dependence in the outcome and parameter models, not on introducing additional preference uncertainty.

Definition: For treatment $a$, let $Y_B$ be an efficacy or benefit outcome, $Y_H$ a harm outcome, and $\theta$ the parameters governing their distributions. Given observed data $D$, the joint predictive distribution is $$ p(y_B,y_H\mid D,a)=\int p(y_B,y_H\mid \theta,a)\,p(\theta\mid D)\,d\theta. $$ It averages the conditional joint outcome distribution over uncertainty in the parameters. A net-utility outcome is $U=g(Y_B,Y_H;w_B,w_H)$. Its predictive distribution is obtained by applying this utility function to draws from the joint predictive distribution.

The conditional distribution $p(y_B,y_H\mid\theta,a)$ must represent any dependence between outcomes that is part of the model. Even if outcomes are conditionally independent given $\theta$, they can be dependent in the predictive distribution because the posterior distribution of $\theta$ can associate efficacy and safety parameters. Conversely, posterior parameter dependence and within-outcome dependence are distinct sources of association.

Linear Net Utility and Dependence

For a simple additive assessment, suppose larger $B$ means more benefit, larger $H$ means more harm, and net utility is $U=w_BB-w_HH$, with nonnegative weights. The weights put the two outcomes on a common utility scale. The variance of $U$ depends not only on the two marginal variances but also on the covariance between benefit and harm.

Result: If $B$ and $H$ have finite variances, then $$ \operatorname{Var}(U)=w_B^2\operatorname{Var}(B)+w_H^2\operatorname{Var}(H)-2w_Bw_H\operatorname{Cov}(B,H). $$ The covariance is calculated under the joint distribution relevant to the uncertainty being summarized. Positive benefit-harm covariance reduces the variance of a benefit-minus-harm utility; negative covariance increases it.

The result follows by expanding the variance of a linear combination. Since $\operatorname{Var}(cX)=c^2\operatorname{Var}(X)$, and $\operatorname{Var}(X+Y)=\operatorname{Var}(X)+\operatorname{Var}(Y)+2\operatorname{Cov}(X,Y)$, take $X=w_BB$ and $Y=-w_HH$. Then $\operatorname{Cov}(X,Y)=-w_Bw_H\operatorname{Cov}(B,H)$, giving the stated expression. The mean is $E(U)=w_BE(B)-w_HE(H)$; covariance does not alter that mean, but it does alter the spread and shape of the utility distribution.

Worked Example: Analytic Utility Distribution With Correlated Outcomes

Suppose a hypothetical model gives a joint bivariate normal predictive distribution for a treatment's benefit $B$ and harm burden $H$, measured in utility-compatible points. Let $E(B)=12$, $E(H)=3$, $SD(B)=4$, $SD(H)=2$, and correlation $\rho_{BH}=0.60$. Define $U=B-2H$. Under these assumptions, calculate the mean and standard deviation of net utility.

The covariance is $\operatorname{Cov}(B,H)=\rho_{BH}SD(B)SD(H)=0.60(4)(2)=4.8$ points squared. Therefore,

E(U)=12-2(3)=6\text{ points}

Using the variance formula with $w_B=1$ and $w_H=2$,

\operatorname{Var}(U)=1^2(4^2)+2^2(2^2)-2(1)(2)(4.8) =16+16-19.2=12.8\text{ points}^2.

Thus $SD(U)=\sqrt{12.8}=3.578$ points. Because a linear combination of jointly normal variables is normal, $U\sim N(6,12.8)$, where the second parameter is the variance. If the same marginal means and standard deviations were incorrectly combined under independence, the variance would be $16+16=32$ points squared, and the standard deviation would be $5.657$ points. The positive association means that higher benefit tends to occur with higher harm; in the benefit-minus-harm calculation, those movements partly offset. The normality conclusion depends on the stated joint normal model and should not be assumed for arbitrary predictive distributions.

Preserving Parameter Dependence in Prediction

In Bayesian calculations, posterior draws provide a direct way to carry parameter dependence through to utility. For each draw $m$, retain the efficacy and safety parameters from that same draw. Calculate the associated predictive outcome distribution, simulate outcomes if the target is a future individual or group, and then calculate utility. Pairing is essential: separately shuffling the efficacy and safety draws replaces the joint posterior with an artificial distribution that has the same marginals but a different dependence structure.

Draw the parameter vector.
Draw $\theta^{(m)}$ from $p(\theta\mid D)$, keeping efficacy and safety components together.

Generate joint outcomes.
Draw $(Y_B^{(m)},Y_H^{(m)})$ from the model's conditional joint distribution given $\theta^{(m)}$ and treatment $a$. If the model asserts conditional independence, use it here; do not impose it merely for convenience.

Calculate utility draw by draw.
Set $U^{(m)}=g(Y_B^{(m)},Y_H^{(m)};w_B,w_H)$. Summaries or plots of the $U^{(m)}$ values describe the resulting predictive uncertainty, subject to adequate simulation and model fit.

A different target is the distribution of expected utility conditional on parameters. In that case, calculate $E(U\mid\theta^{(m)},a)$ for each posterior draw rather than simulating an individual outcome. This retains parameter uncertainty but excludes individual outcome variability conditional on the parameters. The two distributions answer different questions and should be labelled accordingly.

Worked Example: Paired Posterior Draws for Binary Outcomes

In a hypothetical analysis, a response earns 10 utility points and an adverse event incurs a 4-point penalty. For each posterior draw, the conditional expected utility is $10p_R-4p_A$, where $p_R$ and $p_A$ are the response and adverse-event probabilities. Four equally weighted posterior draws are:

Draw	$p_R$	$p_A$
1	0.60	0.10
2	0.70	0.20
3	0.50	0.05
4	0.80	0.25

Using each paired row, the expected utilities are $10(0.60)-4(0.10)=5.6$, $10(0.70)-4(0.20)=6.2$, $10(0.50)-4(0.05)=4.8$, and $10(0.80)-4(0.25)=7.0$ points. Their equally weighted mean is $(5.6+6.2+4.8+7.0)/4=5.9$ points. Their variance, using the four-draw distribution with divisor 4, is

\frac{(5.6-5.9)^2+(6.2-5.9)^2+(4.8-5.9)^2+(7.0-5.9)^2}{4} =\frac{2.60}{4}=0.6500\text{ points}^2.

This is a distribution of parameter-conditional expected utility across the four posterior draws, not a full distribution of future individual utilities. The distinction matters: outcome simulation would add conditional outcome variability.

The parameter-draw covariance shows why pairing matters. The means of $p_R$ and $p_A$ are 0.65 and 0.15. Their population variances across these draws are 0.0125 and 0.00625, and their covariance is 0.00875. Thus the utility variance is $100(0.0125)+16(0.00625)-80(0.00875)=0.6500$ points squared. If efficacy and safety probabilities were independently recombined, the covariance term would be omitted and the variance would instead be $1.25+0.10=1.35$ points squared. The marginals and mean stay the same, but the uncertainty changes. These four draws are illustrative, not sufficient for a precise posterior summary in an actual analysis.

Predictive Dependence Between Outcomes

Dependence can also arise between outcomes for the same individual, even when parameters are held fixed. For binary outcomes, it is often clearest to specify the joint probabilities for the four possible combinations. A model using only the marginal response and harm probabilities cannot determine the probability of each combination unless it also makes an assumption about their association.

Worked Example: A Joint Binary Outcome Distribution

Consider a hypothetical future patient, with response indicator $R$ and adverse-event indicator $A$. Suppose the predictive probabilities are $P(R=1,A=1)=0.08$, $P(R=1,A=0)=0.52$, $P(R=0,A=1)=0.12$, and $P(R=0,A=0)=0.28$. These probabilities are nonnegative and sum to 1. Assign utility $U=10R-4A$ points. The four states have utilities 6, 10, $-4$, and 0 points, respectively. Hence,

E(U)=0.08(6)+0.52(10)+0.12(-4)+0.28(0)=5.20\text{ points}.

The response probability is $0.08+0.52=0.60$, and the adverse-event probability is $0.08+0.12=0.20$. The second moment and variance are

E(U^2)=0.08(36)+0.52(100)+0.12(16)+0.28(0)=56.8, \qquad \operatorname{Var}(U)=56.8-(5.20)^2=29.76\text{ points}^2.

As a check, $\operatorname{Cov}(R,A)=0.08-(0.60)(0.20)=-0.04$, so the variance formula gives $100(0.60)(0.40)+16(0.20)(0.80)-80(-0.04)=29.76$ points squared. If response and adverse event were assumed independent with those same marginals, the covariance would be zero and the variance would be 26.56 points squared. The mean would remain 5.20 points, but the distribution would differ. This calculation describes a single predictive joint distribution; integrating over uncertain parameters can produce an additional mixture and need not preserve any particular distributional shape.

Diagnostics, Alternatives, and Limitations

A joint calculation is only as credible as its dependence assumptions and predictive model. Check that joint probabilities are coherent, simulated probabilities remain within their valid range, and any covariance matrix is positive semidefinite. For simulation, inspect trace plots or other convergence summaries for the parameter draws, verify that paired efficacy and safety values come from the same draw, and assess Monte Carlo error by increasing the number of draws. Compare simulated marginal summaries with the fitted model's intended marginal predictions, then inspect joint summaries as well. Matching marginal means alone does not validate the dependence structure.

Three common approaches have different consequences:

Approach	What it retains	Main limitation
Joint posterior predictive simulation	Parameter uncertainty and modeled outcome dependence	Requires a defensible joint model and adequate simulation
Analytic calculation from means, variances, and covariance	Exact moments for a linear utility when those moments are known	Moments alone do not determine the full utility distribution
Separate marginal predictions combined under independence	Marginal outcome distributions	Erases dependence and can misstate utility uncertainty

A plug-in calculation at posterior mean parameters is also not generally equivalent to averaging over posterior draws: nonlinear outcome models and nonlinear utility functions make those operations non-interchangeable. Even a correct joint simulation can be limited by sparse data on outcome combinations, misspecified dependence, transportability to the intended population, or weights that do not represent the decision context. A linear utility may be convenient, but when the specified utility is nonlinear, calculate that function for every joint draw rather than applying it only to separate means. A point estimate of mean utility is not a substitute for its distribution when the spread and shape are relevant to the assessment.

Common Pitfall

Wrong reasoning: “The efficacy and harm models give reliable marginal estimates, so I can simulate each outcome independently and subtract the weighted results.” This silently imposes independence. It can distort utility variance and, for nonlinear utility or nonnormal outcomes, other features of the distribution as well.

Correction: Specify or estimate the joint distribution that is appropriate to the prediction target. Preserve efficacy and safety parameter values as paired draws, and preserve conditional outcome dependence where the model supports it. If independence is a deliberate assumption, state it and assess how conclusions change under plausible alternatives. Do not infer independence merely because two separate marginal models were fitted.

Key takeaway: Net utility inherits the dependence structure of efficacy and harm. Calculate it from joint predictive draws or use an analytic result that includes covariance; separate marginal predictions alone are not enough to determine its uncertainty.

Check Your Understanding

Answer each question, then compare with the solutions below.

If benefit and harm have positive covariance, what happens to the variance of $U=B-H$, compared with an independence calculation using the same marginal variances?
For $U=2B-H$, suppose $\operatorname{Var}(B)=9$, $\operatorname{Var}(H)=4$, and $\operatorname{Cov}(B,H)=1.5$. Calculate $\operatorname{Var}(U)$.
Why should efficacy and safety parameters be taken from the same posterior draw when generating joint predictions?
Does the distribution of expected utility across posterior parameter draws include individual outcome variability conditional on the parameters?

Solutions. 1. The covariance term is $-2\operatorname{Cov}(B,H)$, so positive covariance makes the variance lower than under independence. 2. $\operatorname{Var}(2B-H)=4(9)+4-2(2)(1)(1.5)=36+4-6=34$ utility units squared. 3. Pairing retains posterior dependence between the parameters; independent recombination changes the joint distribution while leaving marginal distributions intact. 4. No. It represents parameter uncertainty in conditional expected utility; simulating outcomes conditional on each draw is needed to add individual outcome variability.