Why Internal Consistency Matters
Estimation results are often distributed across several files: a parameter table, a run log, a standard-error report, and a summary generated by a separate script. Each file may be readable on its own while the collection contains a conflict. An estimate may have different values in two tables; a standard error may be attached to the wrong label; or a summary may call a run successful while its log reports that an iteration limit was reached.
The goal of an internal-consistency review is not to decide which result is statistically preferable. It is to establish whether the reported values refer to the same run, parameter, scale, and estimation result, and whether the files agree to the precision they display. This check complements earlier material on comparing submitted and reproduced run outputs and verifying fixed parameters and estimation controls. It does not replace reviewing model implementation or the methods used to calculate uncertainty.
Establish What Is Being Compared
Before comparing numbers, create a correspondence between each output field and the result it claims to represent. A label such as CL is not enough by itself. Check whether it denotes clearance or log-clearance, whether it is expressed in L/h or another unit, and whether it is a population parameter, an individual estimate, or a fixed value. Likewise, the name of a standard-error column does not establish which calculation produced it.
Run identity matters just as much. Two files that have similar names may come from different runs, and a single directory may contain outputs from an earlier attempt. Use available run identifiers, timestamps, configuration records, and file provenance to establish the intended relationship. The audit should compare like with like, not merely files stored together.
- Parameter identity: match the model quantity, index or covariate, and role in the model.
- Scale and units: distinguish a parameter from a transformation of that parameter and record units explicitly.
- Reported quantity: keep estimates, standard errors, fixed values, and other summaries separate.
- Method and run status: identify the estimation attempt and the method or status associated with each record.
Worked Example: Reconciling Clearance Estimates at Different Precision
Suppose two files are confirmed to refer to the same completed run and the same population clearance parameter in L/h. File A reports \(5.24\) L/h to the nearest \(0.01\) L/h. File B reports \(5.2\) L/h to the nearest \(0.1\) L/h. Are these displays compatible?
Under round-to-nearest reporting with intervals of the form \([a-u/2,\ a+u/2)\), a displayed value \(a\) with display increment \(u\) represents values in that interval. File A therefore represents \([5.235,\ 5.245)\) L/h, and File B represents \([5.15,\ 5.25)\) L/h.
The intervals overlap: every value from \(5.235\) up to, but not including, \(5.245\) is inside both intervals. Thus the two displays could result from rounding the same underlying estimate. This is evidence of compatibility, not proof that the files were generated from the same calculation; run identity and provenance still need checking.
Comparing Rounded Values Correctly
A displayed number is usually rounded, so exact equality is too strict when formats differ. Let \(a\) and \(b\) be two displayed values, and let \(u_A\) and \(u_B\) be their respective display increments. For example, a value shown to the nearest \(0.01\) has increment \(0.01\). With the usual half-open rounding intervals, the displays are compatible if their intervals overlap.
The strict inequality matters. At equality the intervals only touch at a point excluded by one interval, so they do not overlap. This criterion assumes that both values use the stated increments and the same units and scale. If a report uses a different rule for rounding ties, check its endpoint convention explicitly rather than assuming the half-open intervals apply.
For scientific notation, determine the increment from the last displayed digit. For instance, \(3.42\times 10^{-2}\) shown to three significant figures has increment \(0.01\times10^{-2}=0.0001\), not \(0.01\). Convert units before comparing; a numerical tolerance cannot repair a scale mismatch. If displays are compatible, record that conclusion as “compatible at displayed precision,” not “identical.”
Worked Example: Detecting a Genuine Estimate Discrepancy
Two confirmed output files report the same dimensionless parameter from the same run. File A reports \(0.842\) to the nearest \(0.001\); File B reports \(0.846\) to the nearest \(0.001\). Check whether rounding can explain the difference.
The intervals are \([0.8415,\ 0.8425)\) and \([0.8455,\ 0.8465)\). Their displayed-value difference is \(|0.842-0.846|=0.004\), while the sum of their half-increments is \(0.0005+0.0005=0.001\). Since \(0.004\) is not less than \(0.001\), the intervals do not overlap. Rounding at the stated precision cannot reconcile the estimates.
This is a discrepancy to investigate, not grounds to select one file’s value. Check run identity, parameter mapping, transformations, and whether one file is stale or was generated from a different configuration. Preserve both records and report the unresolved conflict until its source is established.
Check Estimates, Standard Errors, and Labels Separately
An estimate and its standard error are different quantities. There is no general arithmetic identity that lets a reviewer infer the correct standard error from the estimate alone. Compare standard errors across files only when the parameter, scale, units, run, and standard-error method match. A mismatch can be real, but it can also reflect different uncertainty calculations or a transformation that has not been accounted for.
Names and row order are especially fragile ways to match parameters. Prefer a stable parameter identifier or a documented mapping, then verify the meaning against the model specification and output headers. Check indices and covariate terms as well: a label for a typical value is not interchangeable with a label for a covariate effect. An apparently matching label can conceal a difference in parameterization.
Worked Example: Separating an Estimate Match From a Standard-Error Mismatch
Two reports are confirmed to describe the same population parameter, in the same units, from the same run, and both identify the same standard-error method. Report A gives an estimate of \(12.34\) mg/L and standard error \(0.18\) mg/L. Report B gives an estimate of \(12.3\) mg/L and standard error \(0.31\) mg/L. The estimate increments are \(0.01\) and \(0.1\) mg/L, respectively; both standard errors are displayed to the nearest \(0.01\) mg/L.
For the estimates, the intervals are \([12.335,\ 12.345)\) and \([12.25,\ 12.35)\) mg/L. They overlap, so the estimate displays are compatible. For the standard errors, the intervals are \([0.175,\ 0.185)\) and \([0.305,\ 0.315)\) mg/L. They are disjoint, so the standard-error displays are not compatible at the stated precision.
The correct conclusion is that the estimate displays are compatible but the standard-error records conflict, given the stated matching conditions. The estimate agreement does not resolve the standard-error discrepancy. Check the standard-error calculation, source file, and row mapping; do not silently substitute one value for the other.
A conversion of units must be applied consistently to the estimate and its standard error. For example, if a quantity and its standard error are both expressed in mg/L, converting to g/L multiplies both by \(0.001\). A parameter transformation is different: transformed estimates and uncertainty summaries do not necessarily correspond by simply applying the transformation to the standard error. Record the scale and method, and seek the relevant calculation rather than assuming the numbers should match directly.
Run Status Is Part of the Result
A status label is a claim about the run, not a decorative field. Compare it with the run log and other status records, ensuring that they refer to the same attempt. Terms such as “successful,” “completed,” or “converged” may have software-specific meanings; use the documented status definitions and controls established for that run. A file being present, or containing estimates, is not by itself evidence that estimation completed successfully.
If one summary says “successful” while the corresponding log reports an iteration limit or an error, the records are contradictory until explained. Do not infer that one status is correct from which file looks more polished. Determine whether the files belong to different attempts, whether a status was carried over, or whether the software reports separate stages. If the run status remains unresolved, mark dependent results as requiring review.
Confirm that the files refer to the same run and estimation attempt using recorded identifiers and run records.
Record parameter identity, estimate or standard-error type, scale, units, and any method needed for comparison.
Convert to a common scale and compare rounding intervals; do not demand exact equality from differently rounded displays.
Check status records against the run log, document compatible values and unresolved conflicts, and retain the evidence.
Diagnostics, Alternatives, and Limits
Useful warning signs include duplicate or missing parameter labels, unexpected row order, inconsistent units, a change in the number of reported parameters, a standard error attached to a different estimate row, or status statements that refer to different attempts. These patterns call for tracing records back to their source rather than applying a broad numerical tolerance. Automated checks can flag missing fields, duplicate identifiers, non-finite values, and incompatible rounding intervals, but they cannot establish that two differently named fields represent the same model quantity without a reliable mapping.
A strict record-by-record comparison is appropriate when outputs should be identical and use the same formatting. Rounding-interval comparison is more appropriate when the same result is printed at different precision. A tolerance-based rule can be useful in automated pipelines, but it must be prespecified, justified for the scale and units, and reported; a tolerance chosen after seeing the discrepancy can conceal errors. None of these approaches resolves differences in parameterization, estimation method, run provenance, or status semantics. Those require explicit metadata and review.
This audit establishes internal agreement, not statistical adequacy. Consistent files can all contain an incorrect result, and a mismatch does not by itself identify which record is correct. Keep the conclusion proportionate: state what matched, what failed, the precision and assumptions used, and what evidence is still needed.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- Two files show the same parameter in the same units: \(4.17\) to the nearest \(0.01\), and \(4.2\) to the nearest \(0.1\). Are their displays compatible?
- Two values are \(1.00\) and \(1.10\), each rounded to the nearest \(0.10\). Under half-open rounding intervals, do they overlap?
- One report gives a standard error of \(0.24\) and another gives \(0.29\), both to the nearest \(0.01\). What additional information is needed before calling this an inconsistency?
- A summary calls a run successful, but its log reports an iteration limit. What is the appropriate audit conclusion?
Solutions. (1) The intervals are \([4.165,\ 4.175)\) and \([4.15,\ 4.25)\), which overlap, so the displays are compatible at their stated precision. (2) The intervals are \([0.95,\ 1.05)\) and \([1.05,\ 1.15)\). They touch at \(1.05\), which is excluded by the first interval, so they do not overlap. (3) Confirm the same run, parameter identity, scale, units, and standard-error method. If all match, the intervals \([0.235,\ 0.245)\) and \([0.285,\ 0.295)\) are disjoint and the records conflict. (4) Treat the status as unresolved or contradictory until run identity and software-specific status definitions are checked; the presence of estimates does not establish success.