Conditional And Unconditional Simulation Targets

Why the Simulation Target Must Be Explicit

A simulation result depends not only on the model and scenario but also on what is allowed to vary. A prediction calculated at one fitted parameter vector answers a different question from a prediction averaged over uncertainty in that vector. Both can be appropriate. The important requirement is to specify which question the simulation is designed to answer before interpreting its output.

The earlier material on simulation targets and scenario definitions establishes the need to hold the scenario clear. Here, assume a scenario \(s\) has been specified, including its dose, time points, and any fixed covariate values. Let \(\theta\) denote the model parameter vector, and let \(Y\) denote a simulated outcome under that scenario. The distinction in this tutorial concerns whether \(\theta\) is fixed at its fitted value or varied over a stated parameter-uncertainty distribution.

The term uncertainty-integrated is useful because “unconditional” can otherwise be misunderstood. It does not mean that every input is random or that the simulation represents every source of variability. It identifies one particular operation: averaging across parameter values under \(q(\theta)\). For frequentist analyses, such a distribution can represent uncertainty in an estimator; in a Bayesian analysis, it can represent a posterior distribution. The interpretation depends on how \(q(\theta)\) was defined.

Writing the Two Targets

Let \(m(s,\theta)=E(Y\mid s,\theta)\) be the model-implied mean outcome for scenario \(s\) at parameter vector \(\theta\). A conditional prediction of the mean is \(m(s,\hat{\theta})\). The uncertainty-integrated mean is the average of this model-implied mean across parameter values:

If \(Y\) has additional stochastic variation conditional on \(\theta\), write its conditional distribution as \(p(y\mid s,\theta)\). The conditional outcome distribution is \(p(y\mid s,\hat{\theta})\); the integrated outcome distribution averages those conditional distributions:

This distinction applies to event probabilities as well. If \(A\) is an event such as concentration exceeding a threshold, the conditional probability is \(P(A\mid s,\hat{\theta})\). The integrated probability is the average conditional event probability under \(q(\theta)\). In general, these targets differ, especially when the model prediction or event probability changes nonlinearly with \(\theta\).

A Short Justification: Integrated Means and Variances

For the integrated target, averaging conditional means gives the overall mean. This follows by taking the expectation first conditional on \(\theta\), then over \(q(\theta)\):

The same separation clarifies what variation is included in an integrated outcome distribution. Let \(v(s,\theta)=\operatorname{Var}(Y\mid s,\theta)\), and let \(\mu_{\mathrm{int}}=E(Y\mid s)\). Add and subtract the conditional mean \(m(s,\theta)\) inside the squared deviation:

The first term is the average conditional outcome variance; the second is the variance of the conditional means across parameter values. A plug-in simulation at \(\hat{\theta}\) includes only the conditional outcome variation at that fitted vector. This decomposition concerns the defined simulation distribution; it does not, by itself, say which uncertainty sources are scientifically relevant.

Worked Examples

Designing and Checking a Simulation

The simulation design should match the target in its estimand, not merely in its software implementation. For a conditional target, hold the parameter vector at \(\hat{\theta}\) and simulate any outcome randomness that the target includes. For an integrated target, generate parameter values according to the specified \(q(\theta)\), evaluate the scenario at those values, and average the resulting predictions or outcomes with the appropriate weights.

When the integration is approximated with \(M\) equally weighted parameter draws \(\theta^{(1)},\ldots,\theta^{(M)}\), an estimate of the integrated mean is

For unequal weights \(w_j\) that sum to one, use \(\sum_{j=1}^{M} w_j m(s,\theta^{(j)})\). This numerical average estimates the defined integral; its precision depends on the parameter draws and the simulation procedure. A finite simulation therefore has Monte Carlo error even when the target itself is clearly specified.

Diagnostics, Alternatives, and Limitations

A practical diagnostic is to compare the documented target with the actual inputs to the simulation. If the report calls an output uncertainty-integrated but every replicate uses the same \(\hat{\theta}\), the design has not integrated parameter uncertainty. Conversely, if parameter values vary, identify the distribution and weights that determine what average is being estimated. For a simulated mean, compare estimates from increasing numbers of draws or independent runs to assess whether Monte Carlo variation is material.

A scenario grid is a useful alternative when the goal is sensitivity analysis: calculate outputs at several deliberately chosen parameter values and show how the result changes. But a grid alone is not an uncertainty-integrated estimate. It becomes an approximation to one only when its points and weights represent a specified \(q(\theta)\). Likewise, a plug-in prediction is often simpler and directly answers the question at the fitted estimate, but it does not describe variation across parameter uncertainty.

Neither target is automatically superior. A plug-in target can be appropriate when the question explicitly conditions on the fitted model. An integrated target is appropriate when the intended quantity averages over parameter uncertainty as represented by \(q(\theta)\). The integrated result is only as meaningful as that distribution and the model over the parameter values it covers. It should not be described as accounting for all uncertainty or variability unless the simulation actually includes those sources and the target defines them.

Common Pitfall

Wrong reasoning: “I simulated many subjects with residual error, so my output includes parameter uncertainty.” More subjects or more residual draws at one fixed parameter vector improve the representation of the conditional outcome distribution, but do not vary the parameter vector. Correction: name each source of randomness that is included and check whether the simulation varies \(\theta\) according to \(q(\theta)\). The number of outcome replicates cannot substitute for parameter draws.