Generating Continuous Outcomes Across Arms and Stages

Why Generate Outcomes at the Participant Level?

A MAMS simulation needs data that match the trial scenario being studied. For a continuous outcome, this means specifying what an individual participant's outcome would be in each arm and stage, then generating an observation for each participant. A generator that encodes the wrong treatment effect, variance, or stage pattern can produce convincingly formatted data while answering the wrong simulation question.

This tutorial focuses on constructing those participant-level outcomes. It does not determine which arms continue or how interim decisions are made. The generator's job is to create outcomes under a clearly stated model, with each record assigned to the correct arm and stage.

The usual model assumes that outcomes are independent across participants and cells, conditional on the specified arm-stage distributions. It also assumes that the normal distribution and its parameters are an adequate representation of the scenario. These are modelling choices, not properties guaranteed by the act of generating random numbers. If participants contribute repeated measurements, are clustered, or are otherwise dependent, the independent model here is not sufficient.

Define the treatment effect at stage \(k\) as \(\Delta_{jk}=\mu_{jk}-\mu_{0k}\). This convention makes a positive effect mean a higher average outcome in treatment than control. If lower outcomes are beneficial, the effect can still be defined this way, but the direction of benefit must be stated separately. A common scenario has a shared control mean across stages and a treatment effect that does not change by stage; more general scenarios can specify separate means for each cell.

From Scenario Parameters to Participant Outcomes

For a common control mean \(\mu_0\) and a treatment effect \(\Delta_j\) that is constant over stages, set \(\mu_{0k}=\mu_0\) and \(\mu_{jk}=\mu_0+\Delta_j\). If the effect varies by stage, use \(\mu_{jk}=\mu_{0k}+\Delta_{jk}\). Variances can likewise be common, arm-specific, stage-specific, or fully arm-stage-specific, provided each value is positive.

This representation is a direct way to construct the outcomes: generate an independent standard normal draw \(Z_{ijk}\) for every participant and transform it using that cell's mean and standard deviation. The standard deviation is \(\sigma_{jk}\), not the variance \(\sigma_{jk}^2\). For example, a cell with variance \(100\) uses a multiplier of \(10\).

Worked Examples

Why the Generator Has the Intended Mean and Variance

The transformation \(Y_{ijk}=\mu_{jk}+\sigma_{jk}Z_{ijk}\) preserves the specified distribution because a standard normal variable has mean \(0\) and variance \(1\). Linearity of expectation and the variance rule for a constant multiple give:

Thus the generator targets the intended cell mean and variance. For \(n_{jk}\) independent participants in one cell, the sample mean \(\bar{x}\) has expectation \(\mu_{jk}\) and variance \(\sigma_{jk}^2/n_{jk}\). This follows because the expectation of an average is the average of the expectations, and the variance of a sum of independent outcomes is the sum of their variances. Therefore, small generated samples can have cell means far from their targets without indicating a coding error.

For independent treatment and control samples at one stage, the difference in sample means has expected value \(\Delta_{jk}\) and variance \(\sigma_{jk}^2/n_{jk}+\sigma_{0k}^2/n_{0k}\). The first statement follows by subtracting the two expected sample means. The second follows by adding their variances, since the two samples are independent. This provides a useful check on the scale of variation in simulated differences; it does not require each individual replicate's difference to equal the target effect.

Implementation Checks and Common Failure Modes

Before using generated data, inspect both the construction and its repeated-simulation behaviour. A few checks are exact: the number of records in each cell must equal \(n_{jk}\), all outcomes must use the correct arm-stage parameters, and participant identifiers must not accidentally duplicate records. Other checks are statistical: over many independent data sets, the average cell mean should approach \(\mu_{jk}\), and the variation within a cell should be compatible with \(\sigma_{jk}^2\). These empirical checks are noisy, especially for small samples or few replicates.

• Using a variance as a standard deviation. If the intended variance is \(100\), multiplying standard normal draws by \(100\) creates a variance of \(10{,}000\). Use \(\sigma=10\) in the transformation.
• Applying the treatment effect with the wrong sign. Under the convention here, \(\Delta_{jk}=\mu_{jk}-\mu_{0k}\). A positive effect raises the mean. Confirm that this matches the chosen beneficial direction.
• Applying one stage's parameters to every stage unintentionally. Explicitly construct and check each arm-stage mean and standard deviation, even when some are equal.
• Copying control records into multiple treatment comparisons. A stage's control participants are one set of participants with unique outcomes. Do not create duplicate versions of them merely to represent comparisons with different treatment arms.
• Resampling participants when assembling accumulated data. A participant's generated outcome should remain the same record wherever that participant appears in a later data set. Add new stage participants as new records rather than regenerating earlier observations.

There are alternatives to generating individual outcomes. Under the independent normal model, some calculations can be made from simulated sample means and their known standard errors, which may be faster when only summary statistics are needed. But that approach does not provide participant records for checking data assembly or analyses that require individual observations. Direct normal generation is transparent and flexible for the model stated here; it is not suitable if the actual scenario requires skewness, censoring, repeated measures, or dependence that the model omits.