Why a Comparison Table Needs Structure
A run-comparison table is more than a list of objective-function values. Each number must remain attached to the run that produced it, the model version and changes used, the parameter it describes, and the execution outcome. If any of those links is unclear, a reader may attribute a result to the wrong candidate or mistake an unavailable result for a valid estimate.
The earlier material on defining and identifying runs, preserving run artifacts, recording outcomes, and comparing runs on common inputs provides the workflow context. Here the task is to turn that information into a compact, auditable table. The table should present facts consistently; it should not, by itself, imply that a model is preferable or explain why an objective-function value changed.
Define the Rows, Columns, and Statuses
Start by deciding what one row represents. In a run-level table, one row represents one run, so it is suited to run identifiers, input and model provenance, objective-function values, and overall outcomes. Parameter results can be placed in separate columns if the candidate set is small and fixed. When many parameters are estimated, a separate parameter-level table is usually clearer: one row represents one run-parameter pair.
In either layout, the identifier must be unique and stable. If a run was retried or resubmitted, preserve separate identifiers for separate executions and record their relationship rather than overwriting the earlier record. Otherwise, two different execution outcomes could appear to describe the same run.
Use explicit column names and consistent units. For example, label an estimate as “CL estimate (L/h),” not simply “CL”; report whether uncertainty is a standard error, a relative standard error, or an interval; and state the interval level if an interval is used. If objective-function definitions or scales could differ, record the definition or software-reported quantity alongside the value. A bare column called “OFV” can conceal a difference in what was calculated.
Execution outcome also needs more than a single ambiguous label. A process may have completed while the estimation or uncertainty calculation did not succeed. Record these stages separately when the distinction matters. For example, “execution: completed,” “estimation: converged,” and “covariance: failed” convey different facts. Use a controlled vocabulary and define its terms. Do not treat a missing covariance result as a zero standard error or as evidence that the estimate itself is zero.
Worked Examples of Table Construction
Worked Example: Assemble a Run-Level Table
Suppose two hypothetical runs used the same analysis dataset and preprocessing version. Run R-A01 is the reference model. Run R-B01 changes the residual-error specification. The software reports the same objective-function quantity for both runs. The estimates and standard errors shown below are hypothetical; clearance (CL) is in L/h and volume (V) is in L.
For R-A01, the objective-function value is 1240.6. The CL estimate is 5.20 L/h with standard error 0.42 L/h, and V is 48.0 L with standard error 5.6 L. For R-B01, the corresponding value is 1232.8; CL is 5.05 L/h with standard error 0.40 L/h, and V is 49.1 L with standard error 5.8 L. Both runs completed estimation, and their covariance steps succeeded.
If relative standard errors (RSEs) are useful and the reported standard errors are on the same scale as their estimates, calculate each as \(100 \times SE / |\text{estimate}|\). For R-A01, CL has \(100 \times 0.42/5.20 = 8.08\%\), and V has \(100 \times 5.6/48.0 = 11.7\%\). For R-B01, CL has \(100 \times 0.40/5.05 = 7.92\%\), and V has \(100 \times 5.8/49.1 = 11.8\%\). These calculations describe the reported estimate-scale uncertainty in this chosen measure; they do not replace the standard errors in the record.
| Run ID | Model change | Objective function (reported scale) | CL estimate (L/h); SE; RSE | V estimate (L); SE; RSE | Execution / estimation / covariance |
|---|---|---|---|---|---|
| R-A01 | Reference specification | 1240.6 | 5.20; 0.42; 8.08% | 48.0; 5.6; 11.7% | Completed / converged / succeeded |
| R-B01 | Changed residual-error specification | 1232.8 | 5.05; 0.40; 7.92% | 49.1; 5.8; 11.8% | Completed / converged / succeeded |
The table has one row per run, names the change relative to the reference, includes units for both parameters, and makes the outcome stages visible. A note accompanying a real table should identify the dataset and preprocessing version, model version or lineage, and exact objective-function definition when those details are not already present in linked metadata.
Worked Example: Record a Failed or Incomplete Run
Consider two further hypothetical executions. R-C01 stopped during estimation before a final objective-function value or parameter estimates were reported. R-C02 completed estimation and reported an objective-function value of 1218.4 and a CL estimate of 5.10 L/h, but its covariance step failed, so no standard error was reported.
The table should record unavailable values as “not reported” or “not available,” with the reason stated in the outcome field. It should not enter zero for R-C01’s objective function or for R-C02’s standard error: zero would be a numerical result, not a marker of missing information.
| Run ID | Model change | Objective function | CL estimate (L/h) | CL SE (L/h) | Execution / estimation / covariance |
|---|---|---|---|---|---|
| R-C01 | Changed initial estimate | Not reported | Not reported | Not reported | Stopped / incomplete / not reached |
| R-C02 | Changed initial estimate and retry settings | 1218.4 | 5.10 | Not available | Completed / converged / failed |
The rows distinguish a run that did not produce final estimation results from one that produced estimates but lacked a covariance result. If an output contains provisional or partial estimates, label them as partial and explain their status; do not place them in columns that imply final estimates. The run identifier and outcome should still be retained so the table remains an accurate record of what was attempted.
Worked Example: Use a Parameter-Level Table for a Larger Model
Suppose a hypothetical comparison has three runs and four estimated parameters. Putting every estimate and uncertainty measure in a single run-level row would require many columns and repeated status information. Instead, keep a run-level table for each run’s model change, objective function, and outcome, then use one row per run-parameter pair for estimates and uncertainty.
For illustration, the parameter-level records for two parameters from R-A01 and R-B01 could be arranged as follows. The values are hypothetical, and “SE” is explicitly identified as the uncertainty measure.
| Run ID | Parameter | Estimate | Unit | Uncertainty measure | Uncertainty value | Parameter result status |
|---|---|---|---|---|---|---|
| R-A01 | CL | 5.20 | L/h | SE | 0.42 L/h | Reported |
| R-A01 | V | 48.0 | L | SE | 5.6 L | Reported |
| R-B01 | CL | 5.05 | L/h | SE | 0.40 L/h | Reported |
| R-B01 | V | 49.1 | L | SE | 5.8 L | Reported |
If R-C02’s covariance result is unavailable, its parameter-level row can still record the estimate while setting the uncertainty measure to “SE” and the uncertainty value to “not available,” with the covariance failure linked in the run-level outcome record. Keeping run-level and parameter-level tables linked by the same run ID makes the status traceable without repeating run metadata for every parameter.
Checks Before Sharing the Table
A reliable table is both readable and reproducible. Check that each run ID maps to one execution record, that every estimate is linked to the intended model and parameter, and that all quantities have units or a clearly stated convention. Confirm that values in the table match the preserved run outputs; table formatting should not silently alter precision or substitute a derived quantity for a reported one.
Where a measure is calculated for display, state the formula and retain enough precision to avoid misleading rounding. For example, the RSE calculations above use the estimate and standard error on the same scale. If the estimate is zero, or sufficiently close to zero that the ratio is unstable or uninformative, do not present the RSE as an ordinary interpretable summary without qualification. Record the uncertainty measure that is available and explain the limitation.
For a compact table, combine estimate and uncertainty only when the notation is defined and the same uncertainty measure applies throughout. A cell such as “5.20 (0.42)” is ambiguous unless a note says that the first number is the estimate in L/h and the parenthetical number is its standard error in L/h. Do not mix standard errors, intervals, and RSEs in one unlabeled column.
Common Mistakes and Limitations
- Using a blank or zero for a failed result. A blank may be mistaken for an accidental omission, and zero is a numerical value. Use an explicit status such as “not reported” and state why.
- Conflating completion with successful estimation. A program can finish while estimation or covariance calculation fails. Record these outcomes separately where relevant.
- Omitting units or uncertainty definitions. A number without units, or parentheses without a defined meaning, cannot be interpreted reliably.
- Overloading one wide table. Many parameters can make a run-level table difficult to scan. Use a linked parameter-level table rather than compressing values into ambiguous cells.
- Letting the table imply a conclusion. A comparison table organizes evidence; it does not establish the reason for a numerical change or select the preferred model. Those questions require separate interpretation.
Tables also have limits. They cannot replace model files, data provenance, logs, or full estimation output. They summarize selected fields, so a reader needs access to the corresponding immutable artifacts and enough metadata to trace each entry. A comparison is only as interpretable as the consistency of its inputs, definitions, and reporting conventions.
Check Your Understanding
Answer each question, then check the solutions below.
- A run completed, but its covariance calculation failed. Which outcome fields should the table distinguish, and what should be entered for an unavailable standard error?
- A parameter estimate is 8.00 mg/L and its standard error is 0.60 mg/L. Calculate its RSE to three significant figures.
- Why should a failed run’s missing objective-function value not be recorded as zero?
- A model has many parameters and several runs. Which table layout is likely to be clearer, and what identifier links its records?
Solutions. (1) Record execution completion, estimation outcome, and covariance outcome separately; enter “not available” for the standard error and identify the covariance failure. (2) \(100 \times 0.60/8.00 = 7.50\%\). (3) Zero is a numerical value and could be mistaken for an observed result; use an explicit missing-result label and reason. (4) Use a run-level table for run metadata and a parameter-level table with one row per run-parameter pair; link them using the unique run ID.