Why the Output Table Is Part of the Simulation
A simulation can calculate the right values and still produce an output that is difficult to analyze correctly. If a result cannot be linked unambiguously to its scenario, uncertainty draw, virtual subject, time, and metric, later summaries can silently mix levels or discard pairing. A well-designed table makes those relationships explicit rather than relying on row order, filenames, or undocumented conventions.
The design decision comes first: state what one row represents. For time-varying output, one row can represent one metric measured at one time for one virtual subject under one uncertainty draw and one scenario. This row definition determines the table’s key and makes aggregation and comparison rules inspectable.
These identifiers describe different parts of the simulation hierarchy. A scenario distinguishes the input or regimen being evaluated. An uncertainty draw identifies a sampled set of uncertain quantities. A virtual-subject identifier identifies a subject within the simulated population; it may be local to a draw or deliberately shared across scenarios to preserve matching. Time identifies the observation time using a consistent convention and unit. A metric identifier states what was measured, such as concentration or a specified exposure summary.
The key is composite: no single identifier is expected to identify a row by itself. A subject can have many times and metrics; a time can occur in many scenarios and draws. The table is valid only when the complete key is unique for its stated grain.
Choose the Grain Before Choosing Columns
A common time-varying schema uses columns named scenario_id, draw_id, subject_id, time, metric_id, and value. A unit column or a metric-definition table should specify the units and meaning of each metric. For example, concentration might be in mg/L, while time might be in hours. A metric name alone is not enough if its units or definition can vary between outputs.
Identifiers should be stable and explicit. Do not use the row number as a subject identifier: sorting or filtering would change its meaning. Do not infer scenario or draw identity from row position. If subject identifiers are only unique within each draw, the key must include both draw and subject. If the same virtual subjects are intentionally reused across scenarios, retain the same subject identifiers in those scenarios; this is what makes later matching possible.
Not every result has a time dimension. A subject-level AUC, for example, may be one result per subject, draw, scenario, and metric, with no observation time. Do not invent a time such as zero to make such a result fit a time-varying schema. Use a separate table at the appropriate grain, or define a clearly documented time field that represents a real measurement occasion. The key must match the actual output, not a desired table shape.
Worked Example: Reshaping Concentrations Into Long Form
Suppose a hypothetical simulation has one scenario, one uncertainty draw, and two virtual subjects. Concentrations are recorded at 0 and 2 hours. The values are subject A: 0.0 and 4.2 mg/L; subject B: 0.0 and 3.6 mg/L. A wide table might place the two times in separate columns, but that layout makes time a column heading rather than an explicit identifier.
In long form, the four observations are represented as follows:
| scenario_id | draw_id | subject_id | time_h | metric_id | value | unit |
|---|---|---|---|---|---|---|
| S1 | D1 | V1 | 0 | concentration | 0.0 | mg/L |
| S1 | D1 | V1 | 2 | concentration | 4.2 | mg/L |
| S1 | D1 | V2 | 0 | concentration | 0.0 | mg/L |
| S1 | D1 | V2 | 2 | concentration | 3.6 | mg/L |
Each row is one subject’s concentration at one time. For example, the key (S1, D1, V1, 2, concentration) identifies the 4.2 mg/L observation. The two zero values are separate observations because their subject identifiers differ. The displayed time values are in hours, as indicated by the column name; the metric values are in mg/L.
Validate the Key and the Measurement Meaning
A table should be checked against its declared grain before it is summarized. For the time-varying schema, test whether each complete combination of scenario, draw, subject, time, and metric occurs once. A duplicate key can indicate that an intended identifier was omitted, that a simulation emitted duplicate records, or that two results with different meanings were collapsed into one metric label. It is not safe to remove duplicates automatically: first determine whether they are genuine duplicate records or distinct observations requiring another key field.
A valid key is necessary but not sufficient. Check that time uses one unit and convention throughout, metric identifiers have stable definitions and units, and identifiers do not change meaning between rows. If the time grid differs by scenario, that is acceptable when the actual time is recorded; comparisons at a common time must then match on time rather than assuming corresponding row positions.
Missing results also require an explicit interpretation. A missing value should not be confused with a value of zero, and a row that is absent should not silently be treated as an observed zero. When failures or nonfinite outputs occur, preserve the distinctions established in the simulation output and make the status interpretable; do not let table reshaping erase the reason a value is unavailable.
Worked Example: Finding an Incomplete Key
A hypothetical export contains these three rows, all for metric concentration:
| scenario_id | draw_id | subject_id | time_h | value_mg_L |
|---|---|---|---|---|
| S1 | D1 | V1 | 2 | 4.2 |
| S1 | D1 | V1 | 2 | 4.2 |
| S1 | D2 | V1 | 2 | 3.8 |
Under the declared grain, the first two rows have the same complete key: (S1, D1, V1, 2, concentration). The key is therefore not unique. The third row is not a duplicate of them because its draw identifier is D2.
The repeated first two rows could be a duplicate export, or they could be two distinct replicate records for which the schema lacks a replicate identifier. The values alone cannot establish which explanation is correct. The correct action is to inspect the output-generation logic and either remove a confirmed duplicate or add the missing identifier and revise the row definition. Averaging 4.2 and 4.2 would hide the key failure rather than resolve it.
Aggregate Without Collapsing the Hierarchy
A tidy table supports summaries because each grouping level can be named explicitly. For instance, a population summary at a particular scenario, draw, time, and metric groups over virtual subjects. A summary across uncertainty draws is a different operation: it first needs a defined draw-level quantity and then summarizes those quantities over draws. Retaining the identifiers until the intended summary is clear prevents accidental mixing of subjects and draws.
Consider a threshold-attainment indicator for a subject-level metric \(x\) and threshold \(c\). The indicator is 1 when \(x \ge c\), and 0 otherwise. Within a particular scenario and draw, the attainment proportion across \(n\) subjects is the sum of these indicators divided by \(n\). The group identifiers used in the calculation should remain in the resulting summary table.
Here \(s\) labels the scenario, \(d\) the uncertainty draw, and \(j\) the subject within that group. This expression defines a within-draw proportion. It does not by itself specify how to combine those proportions across draws, nor whether draws or subjects should receive equal weight in a further summary.
Worked Example: Summarizing Attainment Within Each Draw
In a hypothetical simulation, the threshold is 1.0 mg/L and the subject-level metric is a minimum concentration in mg/L. The tidy rows are:
| scenario_id | draw_id | subject_id | metric_id | value_mg_L |
|---|---|---|---|---|
| S1 | D1 | V1 | minimum_concentration | 1.2 |
| S1 | D1 | V2 | minimum_concentration | 0.8 |
| S1 | D1 | V3 | minimum_concentration | 1.5 |
| S1 | D2 | V1 | minimum_concentration | 0.9 |
| S1 | D2 | V2 | minimum_concentration | 1.1 |
For draw D1, two of three values meet or exceed 1.0 mg/L, so the within-draw proportion is \(2/3 = 0.6667\). For draw D2, one of two values meets the threshold, so the proportion is \(1/2 = 0.5000\). The result table should retain the draw identifier:
| scenario_id | draw_id | threshold | attainment_proportion |
|---|---|---|---|
| S1 | D1 | 1.0 mg/L | 0.6667 |
| S1 | D2 | 1.0 mg/L | 0.5000 |
Pooling all five subjects gives \(3/5 = 0.6000\), but that differs from the equal-weight mean of the two draw-level proportions, \((0.6667 + 0.5000)/2 = 0.5834\) using the displayed rounded values. The difference arises because D1 has three subjects and D2 has two. Which summary is appropriate depends on the intended estimand and simulation design; the table should preserve enough identifiers to make that choice explicit.
Keep Scenario Comparisons Aligned
Scenario comparisons require attention to whether outputs are independent or deliberately matched. When the same uncertainty draws and virtual subjects are reused across scenarios, those identifiers define the pairs. Compute a contrast only after matching rows on the common identifiers and on any other dimensions needed for the question, such as time and metric. Scenario is then the dimension that differs between the matched values.
For time-varying observations, a typical match uses draw, subject, time, and metric, with one value for each scenario at each matched key. For a subject-level metric without time, time is not part of the matching key. A one-to-one match must be checked: unmatched keys and multiple values for a scenario-key combination should be investigated rather than silently dropped or combined.
Worked Example: Comparing Two Scenarios at a Matched Time
A hypothetical simulation reuses the same draw and two virtual subjects under scenarios S1 and S2. At 2 hours, the concentration values are 4.2 and 3.6 mg/L under S1, and 3.8 and 4.1 mg/L under S2. Define the contrast as S2 minus S1.
| draw_id | subject_id | time_h | concentration_S1_mg_L | concentration_S2_mg_L | S2 minus S1 (mg/L) |
|---|---|---|---|---|---|
| D1 | V1 | 2 | 4.2 | 3.8 | -0.4 |
| D1 | V2 | 2 | 3.6 | 4.1 | 0.5 |
For V1, the contrast is \(3.8 - 4.2 = -0.4\) mg/L. For V2, it is \(4.1 - 3.6 = 0.5\) mg/L. Their mean matched contrast is \((-0.4 + 0.5)/2 = 0.05\) mg/L. This calculation is valid as a matched comparison because each contrast uses the same draw, virtual subject, time, and metric in both scenarios.
If the subject identifiers had been generated independently in the two scenarios, matching V1 to V1 would not be justified merely because the labels happen to agree. The table and simulation design must establish what the identifiers mean.
Common Mistakes and Practical Alternatives
A wide table can be convenient for a fixed report or a small number of known times. It is less suitable as a canonical simulation output when the number of times, metrics, scenarios, or draws varies. Long form makes those dimensions data rather than column names, so filtering, grouping, joining, and uniqueness checks can use the same fields across outputs. A wide report can be derived later from a well-defined long table when that presentation is useful.
- Putting scenario values in separate columns. This makes a new scenario require a new schema and can complicate matching. Store scenario identity in a column and scenario-specific values in rows.
- Dropping draw or subject identifiers after simulation. This prevents draw-level summaries and may destroy paired comparisons. Keep identifiers until the intended analysis level is explicit.
- Using row order to imply time or subject. Sorting, chunking, or parallel execution can change order. Store the actual identifiers and time values.
- Combining different metrics in one unlabeled value column. A value of 4.2 is uninterpretable without a metric definition and unit. Retain a metric identifier and its units.
- Averaging duplicate keys automatically. Duplicates can signal a missing dimension or an execution problem. Diagnose the cause before deciding how to handle them.
The central limitation is that tidy structure cannot repair an ambiguous simulation design. It cannot determine whether subjects are intentionally matched, whether two metric labels mean the same thing, or which weighting defines a desired population summary. Those decisions must be specified and reflected in the identifiers and metric definitions.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- A table has scenario, draw, subject, time, and concentration value, but no metric identifier. What is missing if the same table may also contain AUC results?
- Two rows have identical scenario, draw, subject, time, and metric identifiers. What should be done before removing or combining either row?
- In one draw, 7 of 10 subjects meet a threshold; in another, 3 of 5 do. Calculate each draw-level proportion and the pooled proportion across all 15 subjects.
- For a paired scenario contrast in concentration, which identifiers should match between the two scenario values?
Solutions. (1) Add a metric identifier so concentration and AUC are distinguishable, and specify the unit and definition for each metric. (2) Investigate whether the rows are duplicate records or whether the row grain is missing an identifier, such as a replicate; do not average them automatically. (3) The proportions are \(7/10 = 0.7000\) and \(3/5 = 0.6000\). The pooled proportion is \((7+3)/(10+5) = 10/15 = 0.6667\). (4) Match draw, virtual subject, time, and metric; scenario is the dimension that differs. Confirm that subject identifiers represent intentional matching across scenarios.