Tutorials › Model-Informed Drug Development (MIDD) › Dose Modification and Selection Bias in Safety Models

Exposure-Response for Efficacy and Safety · Tutorial 211 of 596

Dose Modification and Selection Bias in Safety Models

Learn to define event-free risk sets after dose changes, recognize selection bias in observed safety models, and assess its impact with adjustment and sensitivity analyses.

Advanced 10 min read

What You'll Learn

  • Define the risk set for a time-to-first adverse-event analysis when dose changes over time.
  • Explain how safety signals can influence both subsequent exposure and event risk.
  • Distinguish a time-varying exposure association from a causal effect of dose modification.
  • Standardize event risks across a common distribution of measured safety signals.
  • Calculate stabilized inverse-probability weights and assess the assumptions they require.
  • Diagnose sparse support, model misspecification, and incorrect event denominators.

Why Dose Changes Complicate Safety Exposure-Response

In clinical development, a safety signal can lead to a dose reduction, interruption, or discontinuation. The change may lower a patient's subsequent exposure, but the signal may also predict a later adverse event. Consequently, patients observed at lower exposure can have a higher underlying risk than patients who remain at the original dose. Conversely, patients who remain at a high dose may be a selected group who have so far tolerated treatment. Comparing outcomes by observed dose alone can therefore mix the effect of exposure with the reasons that exposure changed.

A key safeguard is to define exactly who is eligible to experience the endpoint at each time. This is especially important for a time-to-first-event analysis: a patient who has already experienced the event is no longer at risk for a first event. The dose change, the event definition, and the denominator must all refer to the same time interval and estimand.

Definition: For interval \(k\), the risk set \(R_k\) for a time-to-first-event endpoint consists of patients who are under observation and have not experienced that endpoint before the start of interval \(k\). The interval-specific risk is the number of first events during the interval divided by the number of eligible patients in \(R_k\), subject to the stated handling of censoring and competing events. A patient who has already had the endpoint cannot contribute another first event.

Exposure must also be time-indexed. Let \(A_k\) denote the dose or exposure category that applies during interval \(k\), assigned using information available before that interval begins. Let \(Y_k\) indicate a first event during interval \(k\). A signal measured before the dose decision can help explain both \(A_k\) and \(Y_k\). A signal measured after the event, by contrast, cannot be treated as a pre-event reason for the dose decision. Temporal ordering is part of the analysis, not a detail to resolve after fitting a model.

How Selection Enters the Comparison

Suppose a measured safety signal \(L\) influences the dose decision and predicts the endpoint. The event risk among patients observed at dose category \(a\) is a mixture over the signals in that group:

$$ P(Y_k=1\mid A_k=a,R_k) = \sum_l P(Y_k=1\mid A_k=a,L=l,R_k)\,P(L=l\mid A_k=a,R_k). $$

The signal distribution \(P(L=l\mid A_k=a,R_k)\) can differ across dose groups because the signal helped determine the dose. Thus, even if risks are compared correctly within signal strata, the crude comparison can differ simply because the groups contain different proportions of higher-risk patients. This is selection through a dose decision based on evolving patient information; it is often described as confounding by indication.

A useful causal interpretation requires stronger conditions than a correctly calculated crude rate. In particular, relevant predictors of both the dose decision and the endpoint must be measured and adjusted for; each dose option being compared must occur with positive probability for the relevant patient histories; and the exposure and outcome models must be adequately specified. Under these conditions, consistency (the observed outcome under a patient's received dose corresponds to the outcome under that dose) and sequential exchangeability (no unmeasured common causes of dose and outcome at each decision time, conditional on measured history) support causal comparisons. These assumptions are not guaranteed by randomization at baseline: later dose changes are not necessarily randomized.

Worked Example: Keeping the First-Event Risk Set Correct

Consider a hypothetical study with 100 patients who begin interval 1 on the same dose. Twenty experience the first adverse event in interval 1. The remaining 80 are event-free at the start of interval 2. Before interval 2, 20 of those 80 have a separate safety signal that prompts dose reduction; the other 60 continue the dose. During interval 2, one of the 20 reduced-dose patients and six of the 60 continuing-dose patients experience the endpoint for the first time.

The interval-2 risk set contains 80 patients, not all 100 who began interval 1. The interval-specific risks by dose are:

$$ \widehat{P}(Y_2=1\mid A_2=\text{reduced},R_2)=\frac{1}{20}=0.0500, \qquad \widehat{P}(Y_2=1\mid A_2=\text{continued},R_2)=\frac{6}{60}=0.100. $$

The observed reduced-minus-continued risk difference is \(0.0500-0.100=-0.0500\), or −5.00 percentage points during interval 2. This is a descriptive contrast, not by itself evidence that dose reduction caused a lower risk. The safety signal prompting reduction may predict the endpoint, and other differences between the groups may remain. The 20 patients who had an interval-1 endpoint are not included in either interval-2 denominator and cannot contribute interval-2 first events.

Adjustment for Measured Safety Signals

One transparent sensitivity analysis is direct standardization. First calculate an event risk within each dose group and each level of a pre-decision signal. Then average those risks using the same signal distribution for both dose groups. This removes differences in the measured signal mix from the comparison, while leaving other potential sources of bias unresolved.

Worked Example: Crude and Standardized Risks Can Disagree

In a separate hypothetical interval, 200 event-free patients are classified by a pre-decision safety signal and subsequent dose. Among 100 patients whose dose is reduced, 80 have the signal and 20 do not. There are 16 first events among the 80 with the signal and none among the 20 without it. Among 100 patients who continue their dose, 20 have the signal and 80 do not; there are six events in the signal group and two in the no-signal group.

The crude risks are \(16/100=0.160\) for dose reduction and \(8/100=0.0800\) for continued dose. The crude risk difference is \(0.160-0.0800=0.0800\), or 8.00 percentage points higher in the reduced-dose group. But the dose groups have very different signal distributions: 80% of the reduced-dose group and 20% of the continued-dose group have the signal.

For standardization, use a target population with 50% signal-positive and 50% signal-negative patients. The within-stratum risks are \(16/80=0.200\) and \(0/20=0\) for reduced dose, and \(6/20=0.300\) and \(2/80=0.0250\) for continued dose. The standardized risks are:

$$ R_{\text{reduced}}^{*}=0.50(0.200)+0.50(0)=0.100, \qquad R_{\text{continued}}^{*}=0.50(0.300)+0.50(0.0250)=0.1625. $$

The standardized risk difference is \(0.100-0.1625=-0.0625\), or −6.25 percentage points. The direction differs from the crude contrast because the groups have different signal distributions. This example does not establish that dose reduction is protective: the zero observed events in one small stratum is imprecise, and standardization only adjusts for the signal included in the calculation. Unmeasured predictors, sparse data, and other assumptions still matter.

Weighting as a Sensitivity Analysis

Inverse-probability weighting is another way to create a dose comparison less dependent on measured differences in patient history. At each decision time, estimate the probability of the dose category actually received, conditional on measured pre-decision history. A stabilized weight for a patient receiving category \(a\) can be written as:

$$ w_i= \frac{P(A_{ik}=a\mid R_{ik})} {P(A_{ik}=a\mid L_{ik},R_{ik})}, $$

where \(L_{ik}\) denotes measured history available before the decision. The numerator is the marginal probability of that dose category in the risk set; the denominator is the probability conditional on history. In practice, weights may be estimated from a model, and an analysis may also need to address informative censoring. The weighted comparison is meaningful as a causal sensitivity analysis only if the histories capture important dose-outcome predictors, there is adequate overlap in treatment probabilities, and the probability models are suitable.

Worked Example: Calculating Stabilized Weights

In a hypothetical risk set of 100 event-free patients, 40 have a pre-decision safety signal and 60 do not. Dose is reduced for 20 of the signal-positive patients and 12 of the signal-negative patients. There are four first events among the 20 signal-positive patients with reduced dose and one among the 12 signal-negative patients with reduced dose. Among patients continuing dose, there are eight events among 20 signal-positive patients and four among 48 signal-negative patients.

The marginal probabilities are \(P(A=\text{reduced})=32/100=0.320\) and \(P(A=\text{continued})=68/100=0.680\). Conditional probabilities of dose reduction are \(20/40=0.500\) with the signal and \(12/60=0.200\) without it. The stabilized weights for reduced-dose patients are therefore \(0.320/0.500=0.640\) with the signal and \(0.320/0.200=1.60\) without it. For continuing-dose patients, the conditional probabilities of continuing are \(20/40=0.500\) and \(48/60=0.800\), giving weights \(0.680/0.500=1.36\) and \(0.680/0.800=0.850\), respectively.

The weighted reduced-dose risk is:

$$ \frac{4(0.640)+1(1.60)}{20(0.640)+12(1.60)} =\frac{4.16}{32} =0.130. $$

The weighted continued-dose risk is:

$$ \frac{8(1.36)+4(0.850)}{20(1.36)+48(0.850)} =\frac{14.28}{68} =0.210. $$

The weighted risk difference is \(0.130-0.210=-0.0800\), or −8.00 percentage points. The corresponding crude risks are \(5/32=0.156\) and \(12/68=0.176\), a difference of about −2.02 percentage points. The change in contrast shows the effect of rebalancing this measured signal in these data; it does not show that all selection bias has been removed. A causal interpretation additionally depends on the assumptions above, on correct event and exposure timing, and on reliable estimation of the dose probabilities.

Diagnostics, Alternatives, and Limits

Before interpreting an exposure-response model with dose changes, check the data construction and the model behavior. Verify that each dose category applies before the events counted in its interval; that the risk-set denominator includes only patients eligible for the endpoint; and that pre-decision signals are distinguished from measurements taken after an event. For a first-event endpoint, any event after a patient's first event must not be counted as another first event. If the scientific question concerns subsequent events, it requires a recurrent-event definition and corresponding risk-set and estimand choices rather than relabeling those events as first events.

Useful diagnostics include tabulating dose decisions by signal history, checking how many patients in each history stratum receive each dose option, inspecting estimated probabilities and weight distributions, and comparing crude with adjusted results. Extreme weights indicate that some histories have little support for one of the dose options; a fitted model cannot create information where there are few or no comparable patients. Weight truncation can reduce instability but changes the analysis and does not repair a lack of overlap. A zero-event cell can also make a stratum-specific estimate unstable, as in the standardization example.

The main alternatives answer different questions. A treatment-policy analysis based on original randomized assignment preserves the randomized comparison but estimates the effect of assignment under the dose changes and discontinuations that occurred; it does not directly estimate the effect of maintaining a particular subsequent exposure. An observed-dose analysis describes outcomes by dose actually received, but is vulnerable to the time-varying selection described here. Standardization and weighting can adjust for measured, time-ordered predictors of dose and outcome, but cannot correct unmeasured confounding merely by being more elaborate. A lagged-exposure analysis can help prevent an event from being assigned to exposure measured after its onset, but lagging alone does not remove selection by prior safety history.

Competing events and censoring also need explicit handling because they determine who remains eligible or observable. The chosen approach should match the endpoint and estimand; changing the denominator without stating what happened to patients who died, discontinued, or were lost to follow-up changes the question. Finally, a signal-triggered dose reduction may be followed by lower exposure and altered risk, but distinguishing these effects requires adequately measured histories and sufficient data. These analyses are sensitivity tools for understanding the observed association, not a substitute for design or a guarantee of causal identification.

Key takeaway: For a time-to-first adverse event, build every interval's analysis from the patients still event-free and under observation at its start. Treat dose changes as time-varying decisions influenced by prior safety history, and interpret crude, standardized, or weighted dose comparisons in light of their assumptions.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions below.

  1. At the start of an interval, 50 patients remain event-free. Three first events occur during the interval among 10 patients whose dose was reduced, and four occur among 40 who continued dose. What are the two observed risks, and which patients are in the denominator?
  2. Why can the crude event risk be higher after dose reduction even if the reduction itself does not increase risk?
  3. In the stabilized-weight example, what is the reduced-dose weight for a signal-negative patient, and what information determines its numerator and denominator?
  4. What does it imply if no signal-positive patients continue dose in a risk set where signal-positive patients sometimes receive dose reduction?

Solutions. (1) The reduced-dose risk is \(3/10=0.300\), and the continued-dose risk is \(4/40=0.100\). The denominators are the patients in each dose group who were event-free and under observation at the interval start; anyone with a prior first event is excluded. (2) A signal that prompts reduction may itself predict the endpoint. The reduced-dose group can therefore begin the interval with a higher-risk mix, producing a higher crude risk even if the dose change is not harmful. (3) It is \(0.320/0.200=1.60\). The numerator is the marginal probability of dose reduction in the risk set; the denominator is the probability of reduction conditional on the patient's pre-decision signal history. (4) There is no observed support for comparing continued versus reduced dose among signal-positive patients in that risk set. Positivity fails for that comparison, so adjustment cannot reliably estimate the missing contrast from those data alone.