Tutorials › Nonlinear Mixed-Effects Models (NLME) › Dose Selection Under Covariate Distribution Shift

Dose Selection by Simulation · Tutorial 725 of 797

Dose Selection Under Covariate Distribution Shift

Learn to transport dose-specific target-attainment estimates to a different covariate distribution and determine whether the recommended dose changes.

Advanced 12 min read

What You'll Learn

  • Define population probability of target attainment under a specified covariate distribution.
  • Derive the density-ratio reweighting estimator and identify its overlap requirement.
  • Calculate how a changed covariate mix can change the dose that meets a PTA threshold.
  • Compare reweighted estimates with estimates from newly sampled target-population covariates.
  • Diagnose unstable weights, poor overlap, Monte Carlo error, and failures of transportability.

Why the Covariate Distribution Matters

A dose can perform differently across covariate profiles because covariates such as body size, organ function, or disease severity influence exposure or response. Even if the relationship between covariates and target attainment is unchanged, the population probability of target attainment (PTA) can change when the intended treatment population has a different mix of profiles from the development sample. A dose recommendation based on the development sample may therefore not meet the same population target after transport to the intended population.

The issue is not resolved by comparing a few covariate subgroups alone. The goal is to average the model's covariate-conditional prediction over the covariate distribution of the population in which the dose will be used. This tutorial considers that change in covariate distribution while treating the fitted model and its conditional predictions as applicable to both populations.

Definition: Let \(X\) denote the vector of covariates, \(d\) a candidate dose, and \(g_d(x)=P(\text{target attained}\mid d,X=x)\) the model-based conditional PTA for profile \(x\). For a target-population covariate distribution \(F_T\), the target-population PTA is $$ \operatorname{PTA}_T(d)=\int g_d(x)\,dF_T(x)=E_T[g_d(X)]. $$ It is the probability of attaining the specified target for a randomly selected individual from the target population, under dose \(d\), when the model and its conditional predictions apply there.

The function \(g_d(x)\) can represent a probability that already integrates over individual random effects and relevant residual variability in the fitted model. Those distributions and the definition of target attainment must be appropriate for the intended population. Here the transport step changes \(F_T\), not the conditional model \(g_d(x)\). In particular, this approach does not by itself address a change in the covariate-response relationship, model parameters, adherence, or other features of care.

Two Ways to Average Over the Target Population

If covariate data are available from the intended population, the most direct approach is to simulate or predict for those covariate profiles and average their conditional PTAs. If only development-sample covariates are available, reweighting can estimate the target average, provided the target distribution is represented in the development sample.

Result: Suppose the development-sample covariate distribution has density \(f_S(x)\), the target distribution has density \(f_T(x)\), and \(f_T(x)>0\) only where \(f_S(x)>0\). Define the density ratio \(w(x)=f_T(x)/f_S(x)\). Then $$ \operatorname{PTA}_T(d)=E_S[w(X)g_d(X)]. $$ This identity requires the same conditional target probability \(g_d(x)\) in both populations and adequate support in the development distribution for the target profiles.

The identity follows by changing the measure in the expectation:

$$ E_S[w(X)g_d(X)] =\int \frac{f_T(x)}{f_S(x)}g_d(x)f_S(x)\,dx =\int g_d(x)f_T(x)\,dx =E_T[g_d(X)]. $$

For a development sample \(x_1,\ldots,x_n\), a common finite-sample estimate is the normalized weighted average

$$ \widehat{\operatorname{PTA}}_T(d) =\frac{\sum_{i=1}^{n}w_i g_d(x_i)}{\sum_{i=1}^{n}w_i}, \qquad w_i=w(x_i). $$

Normalization is useful when weights are estimated or do not sum exactly to \(n\). If covariates fall into strata \(k\), the same calculation can be written \(\sum_k \pi_{T,k}\hat g_{d,k}\), where \(\pi_{T,k}\) is the target-population proportion in stratum \(k\) and \(\hat g_{d,k}\) is the estimated conditional PTA there. This is direct standardization over strata; it does not require individual-level density estimates when stratum proportions and conditional PTAs are available.

Worked Dose-Selection Examples

Worked Example: A Shift Changes the Selected Dose

Suppose two covariate strata have conditional PTAs predicted by a fitted model. In a hypothetical development population, 75% are in stratum A and 25% in stratum B. The intended population is 40% A and 60% B. The conditional PTAs are assumed to apply in both populations.

Candidate dosePTA in APTA in B
400 mg0.980.83
600 mg0.990.92

For 400 mg, the development-population PTA is \(0.75(0.98)+0.25(0.83)=0.735+0.2075=0.9425\), or 94.25%. The target-population PTA is \(0.40(0.98)+0.60(0.83)=0.392+0.498=0.890\), or 89.0%.

For 600 mg, the development-population PTA is \(0.75(0.99)+0.25(0.92)=0.7425+0.230=0.9725\), or 97.25%. In the target population it is \(0.40(0.99)+0.60(0.92)=0.396+0.552=0.948\), or 94.8%.

If the decision rule is to choose the lowest candidate dose with target-population PTA at least 90%, 400 mg qualifies in the development population but not in the target population. The target-population recommendation is therefore 600 mg among these candidates. This conclusion concerns the stated PTA rule only; any safety constraints must also be met.

With individual development-sample records, estimated density ratios can perform the same transport. For discrete strata, the ratio for stratum \(k\) is \(\pi_{T,k}/\pi_{S,k}\). These weights make profiles that are underrepresented in the development sample count more heavily in the target average.

Worked Example: Reweighting Simulated Attainment Outcomes

Consider a hypothetical development sample of eight covariate profiles: six in stratum A and two in stratum B. The target proportions are 40% A and 60% B, whereas the sample proportions are 75% and 25%. At one candidate dose, simulated attainment indicators for the eight profiles are A: 1, 1, 1, 0, 1, 0 and B: 1, 0. The indicators are illustrative Monte Carlo outcomes from the conditional model, not clinical observations.

The stratum weights are \(w_A=0.40/0.75=0.5333\) and \(w_B=0.60/0.25=2.400\). The weighted sum of attainments is \(4(0.5333)+1(2.400)=2.133+2.400=4.533\). The sum of weights is \(6(0.5333)+2(2.400)=3.200+4.800=8.000\). Thus the normalized estimate is \(4.533/8.000=0.5666\), approximately 56.7%.

The unweighted attainment proportion is \(5/8=62.5\%\), but it averages over the development sample's 75%/25% mix rather than the target's 40%/60% mix. The reweighted estimate is lower because stratum B has the lower observed attainment proportion, \(1/2=50\%\), and receives greater representation in the target average. With only eight profiles, this estimate is imprecise; a single simulated set of indicators should not be mistaken for a precise population PTA.

A second route is to generate covariate profiles directly from the intended population distribution, then evaluate \(g_d(x)\) at each profile. This avoids estimating density ratios, but it requires a credible target-population covariate distribution. If the conditional model uses simulated random effects or other stochastic components, those should be generated according to the model assumptions for each profile.

Worked Example: Direct Simulation From the Target Distribution

Suppose five hypothetical covariate profiles are sampled from the intended population. The fitted model gives the following conditional PTAs for two doses. Assume these values already average over the model's relevant within-profile variability.

Target-population profilePTA at 400 mgPTA at 600 mg
10.880.96
20.820.92
30.760.89
40.910.97
50.680.85

The direct Monte Carlo estimates are the sample averages. For 400 mg, \((0.88+0.82+0.76+0.91+0.68)/5=4.05/5=0.810\), or 81.0%. For 600 mg, \((0.96+0.92+0.89+0.97+0.85)/5=4.59/5=0.918\), or 91.8%.

At a 90% PTA threshold, 600 mg meets the criterion and 400 mg does not in this simulation. Because only five target profiles were used, the estimates can vary appreciably with the sample; they illustrate the calculation rather than establish a final dose recommendation. More profiles, and adequate simulation of model variability, are needed for stable estimates.

Assumptions, Diagnostics, and Alternatives

Reweighting or direct target-population simulation is meaningful only when several conditions are credible. First, the covariates that determine differences in conditional target attainment must be represented: omitted effect modifiers can make \(g_d(x)\) wrong for transport. Second, conditional on the modeled covariates, the response model and relevant random-effect assumptions must apply to the target population. Third, reweighting requires overlap: target profiles cannot occur in regions wholly absent from the development sample. Finally, the target distribution itself must be estimated or specified well enough for the averaging step.

Useful diagnostics focus on whether the target average is supported by the available profiles and whether the numerical estimate is stable:

  • Compare covariate distributions in the development and target populations, including joint patterns and clinically important tails, not only separate marginal means.
  • Inspect the range and concentration of weights. A few extremely large weights mean the estimate depends heavily on a small number of profiles.
  • Calculate the effective sample size \(n_{\mathrm{eff}}=(\sum_i w_i)^2/\sum_i w_i^2\). It is at most \(n\) for nonnegative weights and can be far smaller when weights are concentrated.
  • Repeat the Monte Carlo calculation with more profiles or simulation draws and check whether estimated PTA and the dose decision are stable. Also inspect conditional predictions for implausible values or abrupt patterns across covariates.

When overlap is poor, trimming or capping extreme weights may stabilize an estimate, but it changes the effective target distribution and can introduce bias. It is not a substitute for support. Consider collecting target-population covariate data, restricting the target estimand to a defensible region of common support, or explicitly acknowledging that the model-based estimate extrapolates beyond the data.

Direct simulation from target covariates is often preferable when representative target profiles are available: it makes the target averaging explicit and avoids density-ratio estimation. Reweighting is useful when development profiles and a target distribution description are available but new individual profiles are not. Stratified standardization can be more transparent when a few meaningful strata capture the relevant differences. All three methods rely on appropriate conditional predictions; they do not repair a misspecified model. Monte Carlo error, uncertainty in the target covariate distribution, and uncertainty in fitted parameters are also distinct sources of uncertainty and should not be conflated.

Common Mistake and Practical Interpretation

A common mistake is to reweight a development-sample PTA by changing only the overall proportion of a covariate while ignoring how that covariate is associated with other modeled covariates. For example, changing the marginal proportion with impaired renal function while leaving body size fixed at its development-sample distribution may create a joint covariate mix that does not represent the target population. The correction is to define the target distribution of the full covariate vector, or use a justified stratification that preserves the relevant joint structure.

Another mistake is to interpret a shifted estimate as evidence that the conditional drug response changed. A difference between \(\operatorname{PTA}_S(d)\) and \(\operatorname{PTA}_T(d)\) can arise solely because the populations weight the same profile-specific predictions differently. Conversely, if the conditional relationship itself differs, covariate reweighting alone is insufficient. A dose recommendation should therefore name the target population and its covariate distribution, state the PTA threshold and other decision constraints, and describe the support and uncertainty behind the estimate.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions below.

  1. If the target and development populations have the same conditional PTA function but different covariate distributions, which part of the population PTA calculation changes?
  2. In a two-stratum setting, development proportions are 0.80 and 0.20, and target proportions are 0.50 and 0.50. What are the stratum density-ratio weights?
  3. Why can a very large effective-sample-size reduction indicate a problem with a reweighted PTA estimate?
  4. Suppose a target covariate profile has positive probability in the target population but is absent from the development population. Can ordinary density-ratio reweighting identify its contribution without additional assumptions?

Solutions. (1) The conditional function \(g_d(x)\) stays the same under the transportability assumption, but the distribution over which it is averaged changes from \(F_S\) to \(F_T\). (2) The weights are \(0.50/0.80=0.625\) for the first stratum and \(0.50/0.20=2.50\) for the second. (3) It indicates that a small number of profiles carry much of the total weight, so the estimate may be unstable and sensitive to those profiles or their simulated outcomes. (4) No. The density ratio is undefined where the development density is zero and the target density is positive; the contribution requires new data or an explicit extrapolation assumption.