Worked Comparison of MAP and Posterior Predictive Forecasts

Why Compare Two Forecasts?

With only a few concentration measurements, an individual’s pharmacokinetic parameters may remain uncertain even after Bayesian updating. A forecast based on one selected parameter value is simple to compute, but it may conceal that uncertainty. A posterior predictive forecast instead averages over plausible parameter values and therefore represents both uncertainty about the individual and variability in a future observation.

The distinction matters when a forecast is used to assess an exposure target. A MAP-based forecast can be close to the posterior predictive mean in one example and still give a very different probability of crossing a clinically relevant threshold. The comparison below isolates this distinction in a deliberately simple hypothetical intravenous pharmacokinetic model.

Definition: Let $\theta$ denote an individual parameter with posterior distribution $p(\theta\mid y)$ after observed data $y$. A MAP-based forecast evaluates the future-outcome model at a maximum a posteriori estimate $\hat{\theta}_{\mathrm{MAP}}$. A posterior predictive forecast averages the future-outcome distribution over the posterior: $$ p(\tilde{y}\mid y)=\int p(\tilde{y}\mid\theta)\,p(\theta\mid y)\,d\theta. $$ Here $\tilde{y}$ is a future observation. The integral includes uncertainty in $\theta$, as well as future-observation variability represented by $p(\tilde{y}\mid\theta)$.

A Sparse-Data Example and Its Assumptions

Suppose a hypothetical patient receives a 400 mg intravenous bolus. Assume a one-compartment model with known volume $V=40$ L and an individual clearance $CL$ that remains constant over the forecast period. The concentration predicted by the structural model $t$ hours after the bolus is

C(t\mid CL)=\frac{400\ \mathrm{mg}}{40\ \mathrm{L}}\exp\left(-\frac{CL}{40\ \mathrm{L}}t\right).

For this comparison, suppose one concentration was observed 4 hours after dosing, and a Bayesian update produced the following discrete approximation to the posterior for clearance. The approximation can be understood as three weighted posterior particles. The weights are probabilities and sum to one.

Clearance $CL$ (L/h)	Posterior probability	Predicted concentration at 4 h (mg/L)
4	0.25	6.703
6	0.50	5.488
8	0.25	4.493

For a concrete illustration of these weights, take equal prior probabilities on the three clearance values and a measured concentration of approximately 5.598 mg/L, with normally distributed measurement error of about 0.934 mg/L. The observation lies approximately midway between the predictions at $CL=4$ and $CL=8$; under this illustrative likelihood, the likelihood at $CL=6$ is about twice that at either outer value. Normalizing the likelihood-weighted prior gives posterior weights approximately 0.25, 0.50, and 0.25. In practice, posterior weights would be obtained from the specified model and the actual observed data, not selected to make the comparison convenient.

The forecast below is for a future concentration 8 hours after the same bolus. To include future observation variability, assume the future measured concentration $Y$, conditional on clearance, follows

Y\mid CL,y\sim N\left(C(8\mid CL),\ (0.4\ \mathrm{mg/L})^2\right).

This normal additive-error model is a simplifying assumption for the example. It assumes the residual standard deviation is known, the same at each clearance value, and independent of $CL$. The dose, sampling times, volume, and model structure are treated as known, and the posterior particles are assumed to represent the relevant uncertainty in clearance. These assumptions make the comparison interpretable; they should not be silently carried over to a real analysis.

Worked Example: MAP and Posterior-Mean Concentration Forecasts

The largest posterior probability is 0.50 at $CL=6$ L/h, so the MAP value among these posterior particles is 6 L/h. Calculate the MAP forecast at 8 hours, and then calculate the posterior predictive mean.

At $CL=6$ L/h, the elimination rate is $6/40=0.15\ \mathrm{h}^{-1}$. Thus the model-predicted concentration, before adding future residual error, is

C(8\mid CL=6)=10\exp(-0.15\times 8) =10\exp(-1.2) =3.012\ \mathrm{mg/L}.

The posterior predictive mean averages the conditional means using the posterior probabilities. The predictions at $CL=4,6,8$ L/h are $4.493,\ 3.012,\ 2.019$ mg/L, respectively:

E(Y\mid y)=0.25(4.493)+0.50(3.012)+0.25(2.019) =3.134\ \mathrm{mg/L}.

The MAP-based structural forecast is 3.012 mg/L, whereas the posterior predictive mean is 3.134 mg/L. The posterior mean clearance is also 6 L/h, equal to the MAP clearance here, because the weights are symmetric. Yet plugging that mean clearance into the nonlinear concentration model still gives 3.012 mg/L, not 3.134 mg/L. Averaging predictions and predicting at an averaged parameter are different operations.

Where Predictive Uncertainty Comes From

A MAP-based forecast ordinarily needs a future-observation distribution as well as a parameter value. In this example, its predictive distribution is $Y\mid CL=6,y\sim N(3.012,\ 0.4^2)$, with concentration in mg/L. That distribution includes the assumed residual variability, but it conditions on a single clearance value. It does not represent uncertainty about which clearance value is appropriate for the patient.

The posterior predictive distribution is a mixture of three normal distributions, one for each posterior particle:

p(Y\mid y)=0.25\,p(Y\mid CL=4)+0.50\,p(Y\mid CL=6)+0.25\,p(Y\mid CL=8).

Its mean is the weighted mean of the component means. Its variance includes both the average within-component variance and the spread between component means. This follows from the law of total variance. Conditional on $CL$, the future observation has mean $C(8\mid CL)$ and variance $(0.4\ \mathrm{mg/L})^2$. Therefore,

\begin{aligned} \operatorname{Var}(Y\mid y) &=E\{\operatorname{Var}(Y\mid CL,y)\mid y\} +\operatorname{Var}\{E(Y\mid CL,y)\mid y\}\\ &=(0.4)^2+\sum_j w_j\{C(8\mid CL_j)-3.134\}^2\\ &=0.160+0.780\\ &=0.940\ (\mathrm{mg/L})^2. \end{aligned}

The first term, $0.160\ (\mathrm{mg/L})^2$, is the residual variance shared by the components. The second, approximately $0.780\ (\mathrm{mg/L})^2$, is the between-clearance contribution. Thus the posterior predictive standard deviation is $\sqrt{0.940}=0.9696$ mg/L, much larger than the MAP-based standard deviation of 0.4 mg/L.

Worked Example: Quantifying the Additional Uncertainty

Calculate the between-clearance variance directly from the three forecast means and compare it with the MAP-based predictive variance.

Using the posterior predictive mean 3.134 mg/L, the between-clearance variance is

\begin{aligned} &0.25(4.493-3.134)^2 +0.50(3.012-3.134)^2 +0.25(2.019-3.134)^2\\ &\qquad=0.25(1.359)^2+0.50(-0.122)^2+0.25(-1.115)^2\\ &\qquad\approx 0.780\ (\mathrm{mg/L})^2. \end{aligned}

Adding the residual variance gives $0.780+0.160=0.940\ (\mathrm{mg/L})^2$, or a posterior predictive standard deviation of 0.9696 mg/L. By contrast, the MAP-based predictive variance is $0.4^2=0.160\ (\mathrm{mg/L})^2$. The MAP-based distribution is narrower because it leaves out uncertainty across clearance values; it is not narrower because the assumed assay or residual error changed.

The posterior predictive mixture is not itself a single normal distribution. In particular, a symmetric interval formed as the mean plus or minus 1.96 predictive standard deviations is not guaranteed to be a 95% predictive interval. Predictive quantiles should be obtained from the mixture distribution, for example by evaluating or simulating from its components.

Target Probabilities Can Change Substantially

A point forecast does not answer every dosing question. If the relevant question is whether a future concentration exceeds 4 mg/L, calculate that probability under each forecast. Under the MAP-based model, the standardized threshold is

z_{\mathrm{MAP}}=\frac{4-3.012}{0.4}=2.470.

Consequently, the MAP-based probability of exceeding 4 mg/L is $1-\Phi(2.470)\approx0.0068$, where $\Phi$ is the standard normal cumulative distribution function. For the posterior predictive distribution, calculate the exceedance probability in each component and average those probabilities with the posterior weights. The component probabilities for $CL=4,6,8$ L/h are approximately 0.891, 0.0068, and 0.0000004:

P(Y>4\mid y) =0.25(0.891)+0.50(0.0068)+0.25(0.0000004) \approx0.226.

Worked Example: Comparing a Target-Exceedance Forecast

A hypothetical decision rule calls for attention if the probability that the 8-hour concentration exceeds 4 mg/L is greater than 0.10. Compare the two forecast approaches.

For the MAP-based forecast, $Y\mid CL=6,y$ has mean 3.012 mg/L and standard deviation 0.4 mg/L. Therefore,

P_{\mathrm{MAP}}(Y>4\mid y) =1-\Phi\left(\frac{4-3.012}{0.4}\right) =1-\Phi(2.470) \approx0.0068.

For the integrated forecast, the component means are 4.493, 3.012, and 2.019 mg/L, each with standard deviation 0.4 mg/L. The corresponding standardized thresholds are $-1.233,\ 2.470,$ and $4.953$. Thus,

\begin{aligned} P_{\mathrm{pred}}(Y>4\mid y) &=0.25\{1-\Phi(-1.233)\} +0.50\{1-\Phi(2.470)\} +0.25\{1-\Phi(4.953)\}\\ &\approx0.25(0.891)+0.50(0.0068)+0.25(0.0000004)\\ &\approx0.226. \end{aligned}

The MAP-based probability, approximately 0.0068, is below the hypothetical 0.10 rule; the posterior predictive probability, approximately 0.226, is above it. The integrated forecast therefore signals appreciable probability of exceeding the target, because one quarter of the posterior weight is on the faster-concentration prediction above 4 mg/L. These are probabilities under the stated model and posterior, not probabilities that a parameter value or model is true.

How to Choose, Check, and Interpret the Forecast

The approaches answer related but distinct questions. A MAP-based forecast describes the future outcome conditional on a selected parameter value. It is computationally economical and can be useful as a compact reference prediction, but it suppresses posterior uncertainty about the parameter. A posterior predictive forecast describes the distribution of a future observation given the data, integrating over parameter uncertainty. It is the appropriate object when the target is a predictive probability or predictive interval under the specified model. A posterior-mean parameter plug-in is a third possibility, but for nonlinear models its prediction need not equal the posterior predictive mean.

In this example, the posterior predictive mean exceeds the prediction at the posterior mean clearance because the concentration function is nonlinear in clearance. More generally, there is no universal direction for this difference: it depends on the model function and the posterior distribution. Neither the MAP value nor the posterior mean parameter should be treated as a substitute for integration when the quantity of interest is a probability of a future event.

A careful comparison also requires diagnostics. Check that posterior weights are normalized, that the posterior representation covers the plausible parameter range, and that the forecast is computed at the intended dose and time. With weighted particles, inspect whether a small number of particles carries most of the weight; a seemingly precise MAP value does not establish that the posterior is concentrated. Check sensitivity to plausible changes in the residual model and to the posterior approximation. If the posterior is multimodal or truncated, a single mode can be especially unrepresentative. A particle-based calculation should also be checked for adequate representation of the posterior tails that matter for the target probability.

The comparison has limits. Here, volume and the structural model are treated as known; there is no uncertainty about model form, dose history, timing, or residual variance. A different model could include uncertainty in these quantities, correlated errors, or a non-normal observation distribution. The discrete posterior is an approximation, and its quality depends on how well its support and weights represent the full posterior. The integrated forecast carries only the uncertainties included in the model; it does not automatically correct model misspecification or faulty input data.

Common pitfall: “The MAP forecast is 3.012 mg/L, so the future concentration is expected to be about 3.012 mg/L with standard deviation 0.4 mg/L.” This treats a forecast conditional on one clearance value as though it represented all posterior uncertainty. The correction is to state what is conditioned on: the MAP-based distribution has standard deviation 0.4 mg/L under the assumed residual model, while the posterior predictive distribution averages across clearance values and has standard deviation about 0.9696 mg/L in this example.

Check Your Understanding

Use the example’s posterior weights, forecast means, and residual standard deviation to answer the questions. Solutions are included for checking your calculations.

What is the MAP clearance, and what is the posterior predictive mean concentration at 8 hours?
Which two sources of variability contribute to the posterior predictive variance?
Why does plugging in the posterior mean clearance not produce the posterior predictive mean concentration in this example?
If the target is $Y>4$ mg/L, which approach gives a larger probability, and approximately what is that probability?

Solutions. (1) The MAP clearance is 6 L/h, the value with posterior probability 0.50. The integrated mean is $0.25(4.493)+0.50(3.012)+0.25(2.019)=3.134$ mg/L. (2) The predictive variance contains the conditional residual variance, 0.160 $(\mathrm{mg/L})^2$, plus the between-clearance variance, approximately 0.780 $(\mathrm{mg/L})^2$. (3) The concentration model is nonlinear in clearance, so the weighted average of concentrations is not generally the concentration evaluated at the weighted average clearance. Here, the posterior mean clearance is 6 L/h, whose plug-in prediction is 3.012 mg/L rather than 3.134 mg/L. (4) The posterior predictive approach gives the larger probability: approximately 0.226, compared with approximately 0.0068 under the MAP-based forecast.