Simulation-Based Design For A Phase II Trial

From a PK/PD Model to a Trial Design

A phase II design should do more than produce a large number of observations. It should have a reasonable chance of answering the clinical question that motivates the trial. A PK/PD model can help translate dose and exposure into plausible treatment effects, while trial simulation can assess how often the proposed design and analysis would reach a prespecified decision under those effects.

The key is to simulate the whole planned experiment, not just the model-predicted mean response. A trial replicate includes participant allocation, variability in outcomes, and the planned statistical analysis. Repeating that process estimates how often the design would satisfy its success rule. Comparing candidate designs across several plausible effects shows whether conclusions depend on optimistic assumptions.

Definition: Simulation-based design evaluation repeatedly generates hypothetical trial data under a specified design and model, applies the prespecified analysis and decision rule to each replicate, and summarizes performance. For a success rule, estimated power is the proportion of replicates that meet the rule when a particular treatment effect is assumed.

Specify the Design and Decision Before Simulating

A simulation result is meaningful only relative to a clearly stated design and decision. Define the target population, treatment arms, allocation, sample size, treatment duration, endpoint, estimand, analysis, and success rule. For example, a design might randomize participants equally to placebo or a target dose, estimate the between-arm difference in change from baseline at week 12, and declare phase II success if the two-sided test has \(p<0.05\) in the direction of benefit.

The PK/PD model supplies plausible response distributions or treatment-effect scenarios. It may be used to simulate individual exposure and response directly, or to derive a set of treatment-effect assumptions for a simpler trial-outcome model. The assumptions should reflect relevant sources of variability, such as differences among participants, residual error, dropout, and adherence, when these are important to the design question. A simplified model is useful only if its omitted features are unlikely to change the design comparison.

For a two-arm continuous endpoint with equal group sizes, a simple starting model is that each participant’s outcome has a normal distribution with a common standard deviation \(\sigma\), and the difference in population means is \(\delta\). The observations are independent across participants, allocation is randomized, and the outcome distribution is adequately represented by the model. If an analysis assumes a known \(\sigma\), a two-sided z-test at level \(\alpha\) rejects when the absolute standardized difference exceeds \(z_{1-\alpha/2}\). A real trial will usually estimate nuisance parameters, so a simulation should use the actual planned analysis rather than treating this approximation as definitive.

Planning principle: Prespecify a clinically relevant effect, a range of plausible effects, and a decision criterion before examining simulation results. A design that performs well only at the most optimistic effect may not be a reliable phase II choice.

Worked Example: Translating Exposure Into Effect Scenarios

Suppose a hypothetical PK/PD analysis uses an Emax relationship for improvement from placebo:

E(C)=E_{\max}\frac{C}{EC_{50}+C}

Assume \(E_{\max}=1.2\) endpoint units and \(EC_{50}=3\) concentration units. For a simplified illustration, take typical steady-state concentrations of 2 concentration units at a lower dose and 6 concentration units at a higher dose.

At the lower dose, the model-predicted improvement is \(1.2\times 2/(3+2)=0.480\) endpoint units. At the higher dose, it is \(1.2\times 6/(3+6)=0.800\) endpoint units. Thus the model suggests a larger improvement at the higher exposure, but the predicted increase from the lower to the higher dose is only \(0.800-0.480=0.320\) endpoint units.

These are model predictions, not observed trial effects. They can inform a set of planning scenarios—for example, a target-dose effect of 0.4, 0.6, or 0.8 endpoint units versus placebo—if those values are considered plausible after accounting for uncertainty and the intended population. A full simulation could draw participant exposures from a PK model and pass each exposure through the Emax relationship; the simplified calculations here show how a model can anchor, but should not dictate, the effect scenarios.

Why Power Changes With Sample Size and Effect

For the simple two-arm normal model with known common \(\sigma\) and \(n\) participants per arm, the standard error of the estimated difference in means is \(\sigma\sqrt{2/n}\). Under a true difference \(\delta\), the standardized statistic has a normal distribution with mean \(\delta/SE\) and variance 1. The test rejects when the statistic is greater than \(z_{1-\alpha/2}\) or less than \(-z_{1-\alpha/2}\). Therefore, its power is

P(\text{reject }H_0\mid\delta) = \Phi\left(\frac{\delta}{SE}-z_{1-\alpha/2}\right) + \Phi\left(-\frac{\delta}{SE}-z_{1-\alpha/2}\right), \qquad SE=\sigma\sqrt{\frac{2}{n}}.

This expression follows by adding the two probabilities of falling in the rejection regions under the assumed alternative. The first term is the probability of exceeding the upper critical value; the second is the probability of falling below the lower critical value. For a positive effect that is several standard errors from zero, the second term is usually small, but it remains part of a two-sided test.

Worked Example: Approximate Power at Two Sample Sizes

Consider a hypothetical placebo-controlled trial with a continuous endpoint, a clinically relevant mean difference of \(\delta=0.50\) endpoint units, a known residual standard deviation \(\sigma=1.00\) endpoint unit, and a two-sided \(\alpha=0.05\) z-test. Assume independent outcomes and equal allocation. This calculation is a planning approximation, not a substitute for simulating an analysis that estimates the variance.

With \(n=30\) participants per arm, the standard error is \(SE=\sqrt{2/30}=0.2582\) endpoint units. The standardized effect is \(0.50/0.2581989=1.93649\). Using \(z_{0.975}=1.96\),

\text{Power} = \Phi(1.93649-1.96)+\Phi(-1.93649-1.96) = \Phi(-0.02351)+\Phi(-3.89649) \approx 0.49067.

The approximate power is therefore 0.491, or 49.1%. In this setting, a trial with 30 participants per arm would have only about a 49.1% chance of rejecting the null hypothesis at the specified effect.

For \(n=60\) per arm, \(SE=\sqrt{2/60}=0.18257\) endpoint units and \(\delta/SE=2.7386\). The power is \(\Phi(2.7386-1.96)+\Phi(-2.7386-1.96)=\Phi(0.7786)+\Phi(-4.6986)\approx0.782\). Thus doubling the sample size raises approximate power to about 78.2%, still just below an 80% criterion. This calculation also shows why a design should be evaluated at the effect considered clinically relevant, rather than at the largest effect the model can produce.

Estimate Performance by Simulating Complete Trials

In a simulation-based evaluation, each replicate represents one hypothetical trial under stated assumptions. Generate treatment assignments and outcomes, apply the planned analysis, and record whether the decision rule is met. If \(B\) replicates are simulated and \(S\) meet the rule, estimated power is \(\hat{\pi}=S/B\). Repeat this separately for each candidate design and each assumed effect scenario.

Define scenarios.
Use the PK/PD model to specify plausible effects and variability assumptions, including scenarios that are less favorable than the central prediction.

Generate each trial.
Apply the candidate allocation and sample size, then simulate the endpoint data under the selected scenario. Include important features of the planned trial where they affect the result.

Apply the planned analysis.
Fit the specified analysis to the simulated data and evaluate the prespecified success rule, including the direction of effect if required.

Summarize and compare.
Calculate success proportions, their Monte Carlo error, and any precision or operational metrics for each design and scenario.

Worked Example: Selecting a Candidate Sample Size

Suppose a hypothetical phase II design compares one target dose with placebo on a continuous endpoint. The PK/PD model informs three plausible treatment-effect scenarios: 0.4, 0.6, and 0.8 endpoint units versus placebo. Assume a residual standard deviation of 1.0 endpoint unit, independent outcomes, equal allocation, and a two-sided 0.05 test. For this illustration, each simulated outcome follows a normal distribution with the scenario-specific mean and stated standard deviation, and the planned success rule is rejection of the two-sided null in the beneficial direction.

Three designs are compared: 40, 60, or 80 participants per arm. In a hypothetical run of 10,000 replicates per design and scenario, the success counts are as follows.

Participants per arm	Effect 0.4 units	Effect 0.6 units	Effect 0.8 units
40	4,350 (43.5%)	7,660 (76.6%)	9,470 (94.7%)
60	5,920 (59.2%)	9,080 (90.8%)	9,920 (99.2%)
80	7,140 (71.4%)	9,670 (96.7%)	9,990 (99.9%)

Suppose the prespecified criteria are at least 80% power at the clinically relevant effect of 0.6 endpoint units, and an approximate 95% confidence-interval half-width no greater than 0.40 endpoint units for the treatment difference. The 40-per-arm design misses the power criterion at 0.6 units; the 60-per-arm design reaches 90.8%.

For the precision check, the approximate standard error at 60 participants per arm is \(\sqrt{2/60}=0.1826\) endpoint units. The approximate half-width is \(1.96\times0.1826=0.358\) endpoint units, below the 0.40-unit limit. The 60-per-arm design meets both criteria under the stated assumptions. Its power at an effect of 0.4 units is only 59.2%, however, so the design is not robust to that smaller effect if success at 0.4 is also required. The choice therefore depends on whether 0.6 is genuinely the prespecified planning target and whether the smaller effect is acceptable as a possible non-success scenario.

Simulation estimates have their own Monte Carlo error. For estimated power 0.908 from \(B=10{,}000\) replicates, its Monte Carlo standard error is \(\sqrt{0.908(1-0.908)/10{,}000}=0.00289\), or about 0.29 percentage points. A rough 95% Monte Carlo interval is \(0.908\pm1.96(0.00289)\), approximately 0.902 to 0.914. This interval quantifies simulation noise, not uncertainty about the true clinical effect or the adequacy of the model.

Diagnostics, Alternatives, and Limitations

Check implementation before relying on a design ranking. At minimum, verify that treatment allocation and sample sizes are correct, simulated outcomes have the intended means and variability, the analysis converges when relevant, and the success rule is coded exactly as specified. Under a null scenario, a test conducted at nominal level 0.05 should reject at a rate near 0.05 when its assumptions hold. Large deviations may indicate a coding problem, a mismatch between the simulated data and analysis, or an invalid test approximation. Repeat runs or increase the number of replicates if Monte Carlo error is large relative to the decision threshold.

Simulation is not automatically more reliable than an analytic calculation. A normal-theory formula is quick, transparent, and useful as a benchmark when its assumptions fit the design. Simulation is more flexible: it can represent nonlinear PK/PD relationships, multiple arms, non-normal outcomes, missing data, and complex analyses. A precision calculation answers a related but distinct question—how variable an estimate or interval is expected to be—and does not by itself describe the chance of meeting a success rule. These approaches complement one another; comparing a simple simulation with a known analytical result is a useful validation check.

A successful simulation does not establish that the model is correct or that the treatment will work. Results depend on the assumed effects, variability, endpoint, analysis, and trial conduct. Explore adverse but credible scenarios, examine whether conclusions change with reasonable assumptions, and report the model and decision rule clearly. Operational feasibility also matters: a statistically attractive sample size may be impractical because of recruitment, treatment duration, or measurement burden. Those constraints should be considered alongside power and precision, not hidden inside a single summary number.

Common pitfall: “The model predicts a large effect, so the trial has high power.” A predicted effect alone does not determine power; sample size, variability, analysis, and the decision rule matter as well. Evaluate the design across prespecified plausible effects and report which assumptions produce each performance estimate.

Check Your Understanding

Use the stated assumptions in each question. Brief solutions follow the questions.

Why is a single model-predicted effect often insufficient for phase II design evaluation?
With \(n=50\) per arm and \(\sigma=1.0\) endpoint unit, what is the approximate standard error of the difference in means?
A simulation has 8,000 replicates, of which 6,000 meet the success rule. What is the estimated power?
What does the Monte Carlo standard error measure, and what does it not measure?

Solutions. (1) The prediction may be uncertain and may not represent the full range of clinically plausible effects; power should be assessed at prespecified scenarios. (2) \(SE=\sqrt{2/50}=0.200\) endpoint units. (3) Estimated power is \(6{,}000/8{,}000=0.750\), or 75.0%. (4) It measures simulation-to-simulation noise in the estimated success proportion; it does not measure uncertainty in the true treatment effect or prove that the model assumptions are correct.