Joint Efficacy and Safety Under Parameter Uncertainty

Why Joint Attainment Is a Separate Question

A candidate dose can look promising on efficacy and safety considered separately, yet still have a low probability of meeting both requirements at once. For dose $d$, uncertainty in model parameters may make the predicted efficacy vary across plausible parameter values. The predicted toxicity may vary too. A dose decision that requires both an efficacy minimum and a toxicity maximum therefore concerns the overlap of two events, not either event alone.

This tutorial treats the thresholds as specified and the uncertainty as represented by a distribution over model parameters. That distribution might be a posterior distribution or an uncertainty distribution obtained by resampling or another method. The interpretation depends on how it was constructed: posterior probabilities are conditional on the observed data and model, while resampling distributions describe uncertainty under their sampling procedure. In either case, the calculation must preserve the joint parameter draws used to evaluate efficacy and toxicity.

Definition: Let $E(d;\theta)$ be the model-based efficacy quantity at dose $d$ under parameter vector $\theta$, and let $T(d;\theta)$ be the corresponding toxicity quantity. For efficacy threshold $E_{\min}$ and maximum acceptable toxicity threshold $T_{\max}$, define the events $A=\{E(d;\theta)\geq E_{\min}\}$ and $B=\{T(d;\theta)\leq T_{\max}\}$. Joint attainment is $A\cap B$. Under an uncertainty distribution $G$ for $\theta$, its probability is $P_G(A\cap B)$.

The meaning and units of $E$ and $T$ must be specified before calculating. For example, $E$ might be a predicted biomarker change in percentage points, and $T$ might be a model-predicted probability of a dose-limiting toxicity over a stated follow-up period. The thresholds must use those same units and time frame. This probability concerns whether the model quantities meet their thresholds across parameter uncertainty. It is not automatically the probability that a particular patient will experience an efficacy response without toxicity; that would also involve individual outcome variability and a clearly defined patient or population.

Calculating Joint Attainment From Paired Draws

Suppose $M$ draws $\theta^{(1)},\ldots,\theta^{(M)}$ represent uncertainty in the model parameters. For every draw, evaluate both model quantities at the same dose using that same draw. Record an indicator equal to 1 when both thresholds are met and 0 otherwise. The average of these indicators estimates the joint probability.

\widehat{P}_G(A\cap B) = \frac{1}{M}\sum_{m=1}^{M} I\left\{ E(d;\theta^{(m)})\geq E_{\min}, \ T(d;\theta^{(m)})\leq T_{\max} \right\}

The word paired matters. Each efficacy value and toxicity value in a row must be calculated from the same parameter draw (and, where relevant, the same simulated scenario). If efficacy and toxicity share parameters, their values will generally be statistically related across the uncertainty distribution. Shuffling one column destroys that relationship and estimates a different quantity.

The marginal probabilities are $P_G(A)$, the probability that the efficacy threshold is met, and $P_G(B)$, the probability that the toxicity threshold is met. Knowing these two values does not in general determine $P_G(A\cap B)$. The product $P_G(A)P_G(B)$ equals the joint probability only if the events $A$ and $B$ are independent under $G$. Independence should not be assumed simply because efficacy and toxicity are reported separately.

Useful bound: For any two events with marginal probabilities $p_E=P_G(A)$ and $p_S=P_G(B)$, the joint probability obeys $$ \max(0,p_E+p_S-1)\leq P_G(A\cap B)\leq \min(p_E,p_S). $$ These bounds follow because $A\cap B$ is a subset of each event, and $P_G(A\cup B)\leq 1$ while $P_G(A\cup B)=p_E+p_S-P_G(A\cap B)$.

Worked Example: Estimating Joint Attainment From 1,000 Draws

Consider a hypothetical dose with efficacy threshold $E_{\min}=50$ percentage points of predicted biomarker reduction and toxicity threshold $T_{\max}=0.25$, a 25% predicted probability of a specified toxicity within 28 days. Across 1,000 paired parameter draws, efficacy meets its threshold in 700 draws, toxicity stays at or below its threshold in 800 draws, and both conditions hold in 620 draws.

The joint estimate uses the count satisfying both criteria:

\widehat{P}_G(A\cap B)=\frac{620}{1000}=0.620

Thus, under the uncertainty distribution represented by these draws, the estimated probability of meeting both model-based criteria is 62.0%. The marginal estimates are $700/1000=0.700$ for efficacy and $800/1000=0.800$ for safety. Multiplying them gives $0.700\times0.800=0.560$, which is not the observed joint estimate. The paired draws show the events overlap more often than that independence calculation would imply.

As a rough Monte Carlo standard error for 1,000 independent draws, use $\sqrt{\hat p(1-\hat p)/M}$:

SE_{\mathrm{MC}} \approx \sqrt{\frac{0.620(1-0.620)}{1000}} = 0.0153

This is about 1.53 percentage points. It describes simulation precision under the independent-draw approximation; it does not measure uncertainty from model misspecification, uncertain thresholds, or inadequate data. If the draws are autocorrelated, this simple formula can understate Monte Carlo error.

Why the Margins Do Not Determine the Overlap

The bound in the information box gives a quick check on any joint estimate. Its upper bound follows because an event that satisfies both conditions must be contained in each condition separately. For the lower bound, rearrange the union identity:

P_G(A\cap B)=P_G(A)+P_G(B)-P_G(A\cup B) \geq p_E+p_S-1

Probabilities are also nonnegative, so the lower bound is the larger of zero and $p_E+p_S-1$. These bounds can be wide. They do not estimate the joint probability; they show what the marginal probabilities alone permit.

Worked Example: Same Margins, Different Joint Probabilities

Suppose two hypothetical sets of 1,000 paired draws each have the same margins: efficacy is adequate in 800 draws and toxicity is within the limit in 700. The joint count, however, differs between the sets.

In set A, both conditions hold in 650 draws. The estimated joint probability is $650/1000=0.650$. The four mutually exclusive outcomes are: both, 650; efficacy only, $800-650=150$; safety only, $700-650=50$; neither, $1000-(650+150+50)=150$.

In set B, both conditions hold in 550 draws. The estimated joint probability is $550/1000=0.550$. The corresponding four counts are: both, 550; efficacy only, $800-550=250$; safety only, $700-550=150$; neither, $1000-(550+250+150)=50$. Both sets have the same efficacy and safety margins, but their joint estimates differ by 0.100.

The probability bounds here are

\max(0,0.800+0.700-1)=0.500 \leq P_G(A\cap B)\leq \min(0.800,0.700)=0.700.

Both estimates, 0.650 and 0.550, satisfy the bounds. The independence product would be $0.800\times0.700=0.560$; it is neither implied by the margins nor a general substitute for the joint calculation.

Worked Calculation With Explicit Paired Values

The following small hypothetical example makes the event definition visible row by row. Suppose predicted efficacy is measured in percentage points and the toxicity quantity is a probability over 28 days. The thresholds are $E_{\min}=50$ percentage points and $T_{\max}=0.25$. Each row represents one paired draw from the parameter-uncertainty distribution.

Worked Example: Applying Both Thresholds to Each Draw

Draw	Predicted efficacy	Predicted toxicity probability	Both criteria met?
1	54	0.18	Yes
2	52	0.22	Yes
3	49	0.16	No
4	57	0.31	No
5	51	0.24	Yes
6	48	0.28	No
7	60	0.20	Yes
8	53	0.26	No
9	50	0.23	Yes
10	46	0.19	No

Draw 9 meets the efficacy condition because the rule is $E\geq50$, including equality. Draws 4 and 8 meet the efficacy condition but fail the toxicity condition. Draws 3 and 10 meet the toxicity condition but fail the efficacy condition. Five of the ten paired draws meet both, so the estimate is $5/10=0.500$. The efficacy margin is $7/10=0.700$, and the safety margin is $7/10=0.700$; their product, 0.490, happens to be close to the joint estimate but is not the calculation that establishes it.

With only ten draws, this estimate is very imprecise. The independent-draw Monte Carlo standard error approximation is $\sqrt{0.500(1-0.500)/10}=0.158$, or 15.8 percentage points. The example is useful for illustrating thresholding, not for supporting a stable dose decision.

Assumptions, Diagnostics, and Common Failure Modes

A joint probability is meaningful only relative to its specified uncertainty distribution, model, and event definition. Before interpreting the result, check that the efficacy and toxicity functions correspond to the same dose, relevant population, and time horizon; that the thresholds are clinically defined and expressed in matching units; and that the draws represent uncertainty in the intended model parameters. Any transformation, such as converting a log-odds toxicity prediction to a probability, must be applied consistently before comparing it with a probability threshold.

A practical diagnostic is to retain, for each draw, the efficacy value, toxicity value, and two event indicators. Then confirm that the joint count cannot exceed either marginal count and that the estimated probability satisfies the bounds. Inspect draws near each threshold, since a small number of borderline values can materially affect the joint estimate when the draw count is limited. For posterior Markov chain draws or other dependent simulations, assess simulation precision using methods appropriate to correlated draws rather than treating every stored draw as independent.

A frequent mistake is to multiply the marginal probabilities without checking independence. For example, seeing $P_G(A)=0.80$ and $P_G(B)=0.70$, a reader may report $0.56$ as the chance of meeting both. That conclusion requires $A$ and $B$ to be independent under the uncertainty distribution. Shared model parameters can make their threshold events dependent, so the correction is to evaluate both criteria on every same parameter draw and count the overlap.

Another mistake is to pair draws arbitrarily—for example, sorting efficacy and toxicity values independently and matching values by rank—or to resample each endpoint separately. Such operations alter the dependence structure and may change the joint probability while leaving the marginal distributions unchanged. The paired rows must be retained from the model fitting or simulation that generated them.

How This Calculation Relates to Alternatives

The direct paired-draw estimate is the general computational approach when a joint uncertainty distribution is available. Marginal probabilities are useful summaries, and the probability bounds can show whether a reported joint value is possible, but neither identifies the overlap. A product of marginal probabilities is a special-case shortcut under independence; it should only be used when that assumption is justified for the two events, not merely because separate endpoint analyses were conducted.

An analytic calculation may be possible when the joint distribution and threshold events have a tractable form. It can provide a useful check on a simulation result. In many nonlinear dose-response models, however, the threshold events are defined by complicated functions of shared parameters, making direct evaluation over paired draws simpler and less restrictive. Regardless of method, the estimand remains the same: probability mass assigned to parameter values for which both conditions hold.

The calculation also has limitations. A high joint probability does not establish that the efficacy and toxicity models are well calibrated, that their structures are correct, or that the thresholds capture all relevant clinical trade-offs. It does not account for uncertainty in the thresholds unless that uncertainty is explicitly represented. Nor does it automatically include variability in observed patient outcomes. Those are different sources of uncertainty and require an appropriately defined model and estimand rather than an unlabelled adjustment to the joint parameter probability.

Key takeaway: Joint attainment is the probability of the intersection of efficacy and safety events under a stated uncertainty distribution. Estimate it from paired draws, report the thresholds and units, and do not infer it from the two marginal probabilities unless independence is justified.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions below.

If efficacy attainment is 0.75 and safety attainment is 0.80, what are the allowable bounds for their joint probability?
In 2,000 paired draws, both thresholds are met in 1,180 draws. What is the estimated joint probability?
Why is multiplying two marginal probabilities not generally a valid joint estimate?
Across 500 draws, efficacy passes in 350, safety passes in 300, and both pass in 240. Is the joint count consistent with the probability bounds?

Solutions. (1) The lower bound is $\max(0,0.75+0.80-1)=0.55$; the upper bound is $\min(0.75,0.80)=0.75$. (2) The estimate is $1180/2000=0.590$, or 59.0%. (3) Multiplication is valid for the joint probability only when the two threshold events are independent under the uncertainty distribution; shared parameters can induce dependence. (4) The margins are $350/500=0.700$ and $300/500=0.600$, giving bounds $\max(0,0.700+0.600-1)=0.300$ and $\min(0.700,0.600)=0.600$. The joint estimate is $240/500=0.480$, which is within those bounds and is consistent.