Why Weight Convenient Draws?
An individual PK posterior can be difficult to sample from directly, especially when the likelihood is costly or the posterior is concentrated in a small region of parameter space. Importance sampling offers a different route: draw parameter values from a distribution that is easier to sample, then give each draw a weight reflecting how relevant it is to the posterior. The weighted draws can approximate both individual parameter summaries and predictions under a proposed dosing regimen.
The method is only useful when the proposal explores the parts of parameter space that matter to the posterior. A large number of draws from a poor proposal can still produce an unreliable forecast. The weight distribution therefore matters as much as the number of draws.
Target Posterior, Proposal, and Weights
Let \(\theta\) denote an individual's PK parameters, \(y\) the individual's observed concentration data, and \(p(\theta)\) the individual prior, which may be informed by a population PK model and covariates. The individual posterior is proportional to the prior times the likelihood. Write its unnormalized density as \(g(\theta)=p(y\mid\theta)p(\theta)\), and its normalized density as \(\pi(\theta\mid y)\). The normalizing constant is \(p(y)=\int g(\theta)\,d\theta\).
The ratio compares the target density with the density that generated the draw. If \(q\) gives a draw a high probability but the posterior gives that region relatively little density, its weight is small; if the posterior-to-proposal ratio is large, its weight is large. The proposal need not be a posterior distribution. It does need to be evaluable at each draw, and it must assign positive density wherever the posterior has positive density. For stable estimates of a particular quantity, the proposal must also cover the important tails that influence that quantity.
When the proposal draws are independent and the relevant expectations are finite, the self-normalized estimate converges to the posterior expectation as \(M\) increases. A finite-sample estimate is not guaranteed to be accurate merely because it is correctly weighted. In particular, very large weights can make a nominally large draw set behave like only a few useful draws.
Why the Weighting Formula Works
For any function \(h\) with a finite posterior expectation, the proposal can be used to rewrite the posterior integral. This identity motivates the estimator and also shows why the common normalizing constant of the posterior cancels.
The approximation replaces expectations under \(q\) with averages over proposal draws. The denominator is estimated as well as the numerator, which is why this is called a self-normalized importance-sampling estimator. Unlike the exact integral identity, a finite-sample ratio can have bias, although it is consistent under standard regularity conditions. This derivation applies to posterior means, variances, probabilities, and other integrable summaries by choosing \(h\) accordingly.
Worked Example: Estimating Posterior Mean Clearance
Suppose a hypothetical individual has observed PK data, and a discrete teaching example uses four possible clearance values. A convenient proposal assigns them probabilities \(q\) shown below. The corresponding unnormalized target masses \(g\) are proportional to the individual posterior masses. For each proposal draw, the importance weight is \(g/q\).
| Clearance (L/h) | Proposal probability \(q\) | Target mass \(g\) | Weight \(g/q\) |
|---|---|---|---|
| 4 | 0.20 | 0.20 | 1 |
| 6 | 0.30 | 1.20 | 4 |
| 8 | 0.30 | 0.90 | 3 |
| 10 | 0.20 | 0.40 | 2 |
The weights sum to \(1+4+3+2=10\), so the normalized weights are \(0.10, 0.40, 0.30,\) and \(0.20\). The estimated posterior mean clearance is
Thus this weighted approximation gives a posterior mean clearance of 7.2 L/h for this hypothetical individual. The calculation does not claim that clearance is known to be 7.2 L/h; it summarizes the weighted posterior distribution. In an actual continuous-parameter analysis, \(q\) and \(g\) are densities rather than probability masses, but the ratio and normalization work in the same way.
From Parameter Weights to a Predictive Distribution
A forecast under a planned regimen requires both uncertainty about \(\theta\) and the PK model's distribution for a future observation. Let \(Y_{\mathrm{new}}\) be a future concentration at a specified time under a specified dose history. Its posterior predictive distribution averages the conditional future-data distribution over the individual posterior:
With importance draws, an expectation of a future quantity \(a(Y_{\mathrm{new}})\) is estimated by first calculating its conditional expectation under each parameter draw, then taking the normalized weighted average. Alternatively, one can simulate a future outcome for each draw and weight those simulated outcomes. The former reduces simulation noise when the conditional expectation is available; the latter can represent the full predictive distribution when simulation is needed.
Worked Example: Predicting a Concentration and a Threshold Event
Use the four normalized weights from the preceding hypothetical calculation. Under a new dose schedule, suppose the PK model gives the following conditional mean concentration and conditional probability that a future concentration exceeds 6 mg/L at the assessment time. These are hypothetical model outputs, not observed study results.
| Clearance (L/h) | Normalized weight | Conditional mean (mg/L) | Conditional \(P(Y_{\mathrm{new}}>6\mid\theta,y)\) |
|---|---|---|---|
| 4 | 0.10 | 8 | 0.90 |
| 6 | 0.40 | 7 | 0.70 |
| 8 | 0.30 | 5 | 0.20 |
| 10 | 0.20 | 3 | 0.05 |
The posterior predictive mean is the weighted sum of conditional means:
The predictive probability of exceeding 6 mg/L is the weighted average of the conditional exceedance probabilities:
The approximate probability is 0.44, or 44%, under this hypothetical model and regimen. It is not the probability that the posterior mean concentration exceeds 6 mg/L: it averages uncertainty in both the parameters and the future observation. A predictive interval would require the corresponding weighted predictive distribution, not just this mean and threshold probability.
Proposal Choice and Practical Computation
A proposal should be convenient to sample and sufficiently similar to the posterior to avoid extreme ratios. Possible choices include a prior distribution, a multivariate distribution centered near a posterior mode, or a mixture designed to cover multiple regions. A prior proposal is simple and respects the prior support, but may waste draws in regions with little likelihood support. A proposal centered near a mode can be efficient for a concentrated posterior, but can miss tails or separated regions if it is too narrow or too simple.
When parameters are transformed for sampling—for example, using log clearance to ensure positive clearance—the density ratio must use densities expressed with respect to the same parameterization. If the target is evaluated on the original scale while the proposal is expressed on a transformed scale, the appropriate change-of-variables Jacobian must be included. Omitting it changes the target distribution being approximated.
For numerical stability, calculate log weights as \(\log w^{(m)}=\log g(\theta^{(m)})-\log q(\theta^{(m)})\). Subtract the largest log weight before exponentiating; this multiplies all weights by the same constant, so normalized weights are unchanged. This avoids overflow from exponentiating very large log weights and underflow from very small ones.
Diagnostics, Failure Modes, and Alternatives
A basic diagnostic is the importance-sampling effective sample size (ESS), which summarizes weight concentration:
The ESS is at most \(M\), with equality when all weights are equal. It approaches 1 when one draw dominates. It is a diagnostic rather than a guarantee: a high ESS cannot detect posterior regions that the proposal never visited, and a low ESS signals that estimates may be unstable. Inspect normalized weights, the largest weights, and whether summaries change materially across independent runs or a larger draw set. For forecasts, also inspect the weighted distribution of the forecast quantities themselves.
Worked Example: Recognizing Weight Degeneracy
Suppose ten proposal draws produce raw weights \(0.90\) for one draw and \(0.01\) for each of the other nine. Their sum is \(0.90+9(0.01)=0.99\), and the sum of squared weights is \(0.90^2+9(0.01^2)=0.8109\). Therefore,
Although ten values were generated, the weight concentration corresponds to only about 1.21 equally weighted draws. A posterior mean or forecast can therefore be driven almost entirely by the draw with raw weight 0.90. Generating more draws from exactly the same poorly matched proposal may help, but a better proposal that explores the posterior-relevant region is often needed.
The central failure mode is inadequate support or tail coverage. If \(q(\theta)=0\) where the posterior has positive density, the ratio cannot correct for the missing region; no amount of weighting repairs a region that was never sampled. Even when support is technically adequate, a proposal with much lighter tails than the posterior can generate rare, enormous weights. An ESS check, weight plot, and repeat-run comparison can reveal instability, but none can prove that an unseen posterior region is irrelevant.
| Approach | Strength | Important limitation |
|---|---|---|
| Importance sampling | Uses independent proposal draws and directly estimates posterior expectations and forecasts. | Can be dominated by a few weights or fail when the proposal misses relevant regions. |
| MCMC | Can explore a posterior without requiring a proposal density tailored for importance ratios. | Draws are dependent; convergence and mixing must be assessed, and sampling can be inefficient. |
| Laplace approximation | Provides a compact local approximation around a posterior mode. | A local approximation can be inaccurate for skewed, heavy-tailed, or otherwise non-Gaussian posteriors. |
These methods answer related computational needs but are not interchangeable diagnostics. Importance sampling is attractive when a well-chosen proposal is available and weighted expectations are the goal. The earlier material on MCMC for individual PK posterior sampling provides another route to posterior draws; MCMC does not make the importance-weight calculation unnecessary if importance sampling is being used. A Laplace approximation can help construct a proposal, but its adequacy for that purpose must still be checked through the resulting weights and forecasts.
Common Mistake and Key Takeaway
Wrong reasoning: “I generated 10,000 proposal draws, so my posterior forecast must be precise.” The draw count alone does not establish precision. If one draw has nearly all the total weight, the effective information can be closer to one draw than to 10,000. If the proposal misses a posterior-relevant region altogether, the weighted result may be misleading even with apparently well-behaved weights among the draws obtained.
Correction: Check that the proposal covers the target support, calculate weights using the correct target and proposal densities, inspect weight concentration and ESS, and assess whether important posterior and predictive summaries are stable. Report the regimen and forecast quantity with units, and interpret a posterior predictive probability as conditional on the model, observed data, and dosing assumptions.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- What condition must the proposal satisfy in regions where the posterior density is positive?
- Suppose two proposal draws have weights 2 and 6, and the corresponding clearance values are 5 and 9 L/h. Calculate the self-normalized posterior mean clearance.
- Why is the predictive probability of exceeding a concentration threshold generally not obtained by checking whether the posterior mean concentration exceeds that threshold?
- For ten draws, one raw weight is 0.90 and each of the other nine is 0.01. What does the ESS tell you, and what can it not establish?
- Name one important difference between importance sampling and MCMC for individual posterior computation.
Solutions. (1) The proposal density must be positive wherever the posterior density is positive, so draws can in principle represent all posterior-relevant regions. (2) The normalized weights are \(2/(2+6)=0.25\) and \(6/(2+6)=0.75\). The mean is \(0.25(5)+0.75(9)=1.25+6.75=8.00\) L/h. (3) The predictive probability averages the conditional probability of the event over posterior parameter uncertainty and future observation variability; a mean does not determine that event probability. (4) The ESS is about 1.21, indicating severe weight concentration. It does not prove that no unvisited posterior region matters or guarantee accuracy. (5) Importance sampling uses proposal draws corrected by density-ratio weights, whereas MCMC generates dependent draws through a transition process targeting the posterior and requires assessment of mixing and convergence.