Residual Simulation Differences Across Tools

Why Matching Predictions Is Not Enough

Two tools can produce the same concentration prediction at every observation time and still produce different simulated observations. The distinction is that a prediction describes the model’s expected or conditional response, while a simulated observation also includes a draw from the residual-error model. Comparing those two quantities as if they were the same is a common source of false alarms in software translation.

Residual simulation is separate from the event-handling and structural-model questions considered in the earlier material on event equivalence and prediction equivalence. It asks what the software does after the model has produced a prediction: which residual distribution is applied, at which records, with what scale and dependence, and whether that error component is included in the output being compared.

Definition: Let $\mu_{ij}$ denote the model prediction for subject $i$ at observation record $j$, conditional on the specified parameter values and, when applicable, that subject’s random effects. A conditional prediction is $\mu_{ij}$ without a residual-error draw. A simulated observation is a value generated from the observation model, such as $Y_{ij}=\mu_{ij}+\epsilon_{ij}$. A prediction may be population-level or individual-level; neither is automatically a simulated observation.

The precise interpretation of a prediction column depends on the software and the output requested. For this comparison, the key test is operational: was an observation-level residual draw actually applied to the prediction? The name of a saved column alone may not answer that question. Also keep interindividual random effects distinct from residual error: random effects change a subject’s parameter values or trajectory, whereas residual error represents variation at the observation level under the specified model.

Write the Observation Model Before Comparing Tools

A useful comparison begins with equations rather than software labels. In an additive-error model, an observation is the prediction plus an error draw:

Y_{ij}=\mu_{ij}+\epsilon_{ij}, \qquad \epsilon_{ij}\sim N(0,\sigma_{\mathrm{add}}^2).

Here $\sigma_{\mathrm{add}}$ is the residual standard deviation in the units of $Y$. In a proportional-error model on the original response scale, one common specification is:

Y_{ij}=\mu_{ij}(1+\epsilon_{ij}), \qquad \epsilon_{ij}\sim N(0,\sigma_{\mathrm{prop}}^2).

In this form, the residual standard deviation of $Y_{ij}$, conditional on $\mu_{ij}$, is $|\mu_{ij}|\sigma_{\mathrm{prop}}$. Another common model combines additive and proportional components:

Y_{ij}=\mu_{ij}+\epsilon_{\mathrm{add},ij}+\mu_{ij}\epsilon_{\mathrm{prop},ij}.

If the two components are independent, have mean zero, and are normally distributed with standard deviations $\sigma_{\mathrm{add}}$ and $\sigma_{\mathrm{prop}}$, then:

\operatorname{Var}(Y_{ij}\mid\mu_{ij}) =\sigma_{\mathrm{add}}^2+\mu_{ij}^2\sigma_{\mathrm{prop}}^2.

The independence assumption matters for this variance formula. If the components are dependent, a covariance term is needed. A software implementation must also agree on the exact observation equation, not merely use a label such as “combined error.” For example, a normal proportional-error model on the original scale is not the same distribution as an error model defined on a transformed response scale. Establish the equation and scale being used before interpreting a mismatch.

Worked Example: Generate One Additive-Error Observation

Suppose a hypothetical subject has an individual prediction of $10.0\ \mathrm{mg/L}$ at a record, and the residual model is additive normal error with $\sigma_{\mathrm{add}}=1.2\ \mathrm{mg/L}$. For illustration, take a standard-normal draw $z=0.50$, so $\epsilon=\sigma_{\mathrm{add}}z$.

The residual draw is:

\epsilon=(1.2\ \mathrm{mg/L})(0.50)=0.60\ \mathrm{mg/L}.

The simulated observation is:

Y=10.0\ \mathrm{mg/L}+0.60\ \mathrm{mg/L}=10.6\ \mathrm{mg/L}.

The value $10.0\ \mathrm{mg/L}$ remains the conditional prediction; $10.6\ \mathrm{mg/L}$ is the simulated observation for this particular draw. Another valid draw would generally give a different simulated observation even though the prediction and residual model are unchanged.

How the Four Tools Represent Residual Simulation

The tools differ in where the observation model is specified and which operation creates a noisy response. Their prediction outputs should therefore be checked separately from their simulation outputs. Interfaces and output-column names can vary by version and workflow, so identify the model equation and the operation used rather than assuming a particular column is always noisy.

Tool	What to distinguish	Practical check
NONMEM	The prediction calculated in the model code, often including quantities such as $F$ or $IPRED$, versus the response variable generated by the observation model using residual error terms such as EPS.	Inspect the $ERROR logic and the simulation setup. Confirm that the simulated response includes the intended EPS draw; do not treat a prediction column as a simulated DV by default.
Monolix	Prediction outputs from a fitted model versus observations generated by a simulation workflow using its residual-error model.	Check the residual-error specification and the selected simulation output. A prediction export does not, by itself, establish that residual error was simulated.
nlmixr2	Predictions and fit-related outputs versus a simulation that evaluates the specified observation model and produces responses with residual variability.	Check how the residual model is specified and whether the simulation operation generates observation-level residual draws, rather than only returning structural predictions.
mrgsolve	Structural or captured prediction quantities versus a response explicitly generated and captured using the model’s residual-error implementation.	Inspect the model code and simulation setup, including how residual draws are generated and used at observation records. A simulated trajectory alone need not include residual error.

In NONMEM, residual variability is commonly expressed in the observation-model code with EPS terms, while the prediction is calculated separately. A simulation must execute the response calculation with residual variability active to generate noisy observations. In Monolix and nlmixr2, a fitted residual-error model defines how observed data relate to predictions, but a prediction or fit output should not be assumed to be a newly simulated response. The simulation workflow must generate that response. In mrgsolve, the model author commonly makes the distinction explicit by calculating and capturing a prediction separately from a response that uses residual draws. Across all four tools, the safe practice is to inspect the implemented equation and the requested output, not infer simulation semantics from a software name or column label.

Worked Example: Combined Additive and Proportional Error

Consider a hypothetical prediction of $8.0\ \mathrm{mg/L}$, with $\sigma_{\mathrm{add}}=0.5\ \mathrm{mg/L}$ and $\sigma_{\mathrm{prop}}=0.10$. Assume independent standard-normal draws $z_{\mathrm{add}}=0.40$ and $z_{\mathrm{prop}}=-1.00$, with the combined model stated above.

The additive contribution is:

\epsilon_{\mathrm{add}}=(0.5\ \mathrm{mg/L})(0.40)=0.20\ \mathrm{mg/L}.

The proportional contribution is:

\mu\epsilon_{\mathrm{prop}} =(8.0\ \mathrm{mg/L})(0.10)(-1.00) =-0.80\ \mathrm{mg/L}.

Thus the simulated observation is:

Y=8.0+0.20-0.80=7.40\ \mathrm{mg/L}.

Under the stated independence and zero-mean assumptions, the conditional variance is $0.5^2+8.0^2(0.10)^2=0.89\ (\mathrm{mg/L})^2$, giving a conditional standard deviation of $\sqrt{0.89}=0.9434\ \mathrm{mg/L}$. A single simulated residual need not equal this standard deviation; the standard deviation describes the distribution across repeated draws.

Justifying the Comparison and Diagnosing Differences

A simple derivation shows why matching random draws is not always the first requirement. Under the combined model, write $Y-\mu=\epsilon_{\mathrm{add}}+\mu\epsilon_{\mathrm{prop}}$. If both components have mean zero, then $E(Y\mid\mu)=\mu$. If they are independent, the variance of their sum is the sum of their variances:

\begin{aligned} E(Y\mid\mu) &=\mu+E(\epsilon_{\mathrm{add}})+\mu E(\epsilon_{\mathrm{prop}})=\mu,\\ \operatorname{Var}(Y\mid\mu) &=\operatorname{Var}(\epsilon_{\mathrm{add}}) +\mu^2\operatorname{Var}(\epsilon_{\mathrm{prop}})\\ &=\sigma_{\mathrm{add}}^2+\mu^2\sigma_{\mathrm{prop}}^2. \end{aligned}

This result provides checks that do not depend on matching the exact random-number sequence: repeated simulated observations should have a mean near $\mu$ and a variance near the stated conditional variance, subject to Monte Carlo variation. But a small simulation can deviate noticeably from those theoretical values. To test whether two implementations use the same equation, a stronger controlled check is to provide the same prediction and the same residual standard-normal draws to both implementations, when the software workflow permits it. Then compare the resulting observation values record by record.

Lock the prediction inputs.
Use the same parameters, subject effects, event history, observation times, and prediction scale. Confirm the predictions agree before adding residual error.

Write the residual equation explicitly.
Record whether the error is additive, proportional, combined, or another specified form, including the distribution, scale, and any dependence assumptions.

Check the records receiving draws.
Verify that residual error is generated at the intended observation records, rather than being omitted or applied at unintended events or output times.

Separate model differences from random-number differences.
Different seeds or random-number implementations can produce different observations under the same model. First compare equations and, if possible, shared draws; then compare distributions over repeated simulations.

When predictions agree but simulated observations do not, investigate the residual specification, error scale, record alignment, and whether one output is a prediction while another is a response. When predictions themselves differ, residual simulation is not yet an adequate explanation: first resolve the prediction mismatch. Diagnostics across repeated simulations should examine conditional means and variances at representative prediction values, not just a single overall residual standard deviation. Proportional error has a prediction-dependent variance, so pooling records with different $\mu_{ij}$ can obscure whether the implementation is correct.

Common Pitfalls and Limits

Comparing a prediction with a simulated response. The difference may be exactly the residual draw, not a translation error. Label prediction and response quantities separately.
Assuming a seed guarantees matching observations across tools. A seed controls a tool’s random stream; it does not guarantee identical draws across different algorithms, call order, or software implementations. Compare with shared prescribed draws when possible, or compare distributional properties.
Using a common error label as proof of equivalence. Additive, proportional, or combined labels do not fully specify an equation, distribution, scale, or error dependence. Compare the written observation models.
Expecting every simulation output to contain residual error. A simulation may return structural predictions only. Confirm that the workflow actually generates an observation variable using residual variability.
Interpreting one simulated sample as a precise diagnostic. Random variation can cause sample means and variances to differ from their theoretical values. Use adequate replication and account for the expected Monte Carlo variability.

The most reliable cross-tool conclusion is often narrower than “the simulations match.” State whether the structural predictions match, whether the observation equations match, and whether the generated distributions agree under comparable conditions. Even identical observation equations do not imply identical record-by-record observations if independent random draws are used. Conversely, coincidentally similar simulated values do not establish that the equations are equivalent.

Worked Example: Interpret a Prediction-Only Output

In a hypothetical check, two tools both return $12.0\ \mathrm{mg/L}$ at a record. Tool A’s output is identified as a conditional prediction. Tool B returns a simulated response of $11.4\ \mathrm{mg/L}$ under an additive model with $\sigma_{\mathrm{add}}=0.8\ \mathrm{mg/L}$. Determine the residual draw represented by Tool B’s output and whether the two values show a prediction mismatch.

Under the additive model $Y=\mu+\epsilon$, the residual draw is:

\epsilon=Y-\mu=11.4\ \mathrm{mg/L}-12.0\ \mathrm{mg/L} =-0.60\ \mathrm{mg/L}.

The standardized draw is $-0.60/0.8=-0.750$. The two reported values do not show a prediction mismatch: both indicate a prediction of $12.0\ \mathrm{mg/L}$. They are different output types, and Tool B’s response includes a plausible residual draw. This one value cannot establish that the tools use the same residual model; that requires checking the equations and, ideally, repeated or controlled-draw comparisons.

Key takeaway: Compare predictions with predictions and simulated observations with simulated observations. For residual simulation, equivalence requires agreement on the observation equation, distribution, scale, and record-level application—not merely similar output labels or matching random seeds.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

A conditional prediction is $6.0\ \mathrm{mg/L}$. Under additive normal error with $\sigma_{\mathrm{add}}=0.5\ \mathrm{mg/L}$, a standard-normal draw is $z=-1.2$. What simulated observation results?
For a combined model with $\sigma_{\mathrm{add}}=0.3\ \mathrm{mg/L}$, $\sigma_{\mathrm{prop}}=0.08$, and prediction $5.0\ \mathrm{mg/L}$, calculate the conditional variance, assuming independent zero-mean components.
Two tools have matching predictions but different simulated observations when run with different seeds. Does this alone show that their residual models differ? Explain.
A saved output is called “prediction” and has no residual draw applied. Can it be compared directly with another tool’s simulated observation as an equivalence check? Why or why not?

Solutions. 1. The residual is $\epsilon=(0.5)(-1.2)=-0.60\ \mathrm{mg/L}$, so $Y=6.0-0.60=5.40\ \mathrm{mg/L}$. 2. The variance is $0.3^2+5.0^2(0.08)^2=0.09+0.16=0.25\ (\mathrm{mg/L})^2$. The conditional standard deviation is $0.50\ \mathrm{mg/L}$. 3. No. Different random-number streams can produce different draws even when the residual models agree. Compare the observation equations and, if possible, use common prescribed draws; otherwise compare distributions over repeated simulations. 4. No. The first quantity is a prediction and the second includes residual variability. Compare predictions with predictions, or generate a response from the first tool using the same observation model before comparing simulated observations.