Why Cross-Section Consistency Matters
A model-informed report may present one result in several places: a parameter table, a figure, a caption, and a sentence describing the development implication. Those presentations need not use identical formatting, but they must refer to the same quantity, population, analysis, and uncertainty. A conversion from liters per hour to milliliters per minute can change the displayed number without changing the result. By contrast, a figure that plots a different confidence limit from the table may change what a reader understands about the evidence.
A consistency check is not a search for matching strings. It is a comparison of the meaning of each reported item and, where needed, of its numerical value after aligning units, definitions, and precision. Earlier material on traceability and analysis-set reconciliation provides the relevant principle: a claim should be traceable to its supporting model output and analysis population. Here the task is to test whether the different report sections preserve that meaning and value.
Set the Comparison Before Comparing Numbers
Before checking whether two numbers agree, establish what each is supposed to represent. A parameter value may be a population estimate, an individual estimate, or a covariate-specific prediction. A sample size may count participants, samples, records, or bootstrap replicates. A plotted band may be a confidence interval for an estimated mean, a prediction interval for future observations, or a simulation percentile range. Similar labels do not make these quantities interchangeable.
For each item to be checked, record the quantity and its definition, the population or analysis set, the model run or analysis version, the units, and the summary being reported. For uncertainty, record the interval level and construction. For a conclusion, record the claim and the statistical evidence it relies on. Then compare the relevant table entries, plotted data and axis labels, captions, and narrative sentences.
Match the parameter, estimand, endpoint, or prediction, including its population and time point.
Check denominators, unit conventions, transformations, and whether each display uses the same analysis version.
Determine whether the values can result from the same underlying number after the stated rounding or conversion.
Confirm that intervals, plotted limits, and prose conclusions agree with the aligned result.
Compare against the controlling analysis output or source artifact, correct the affected presentation, and record the reason.
Units and Rounding: When Different Numbers Agree
A unit check is often the fastest way to distinguish a true discrepancy from a harmless change of scale. Convert values using the stated units and compare them at a precision appropriate to the report. For instance, a rate in liters per hour and the same rate in milliliters per minute have different numerical values because both the volume and time units change.
Worked Example: Reconciling a Clearance Value Across Units
Suppose a hypothetical model report gives typical clearance as \(4.80\) L/h in a table. A figure labels the same parameter as \(80\) mL/min, while the narrative says \(0.080\) L/min. Do these displays agree?
Convert the table value. Since \(1\) L \(=1000\) mL and \(1\) h \(=60\) min,
The table and figure agree. Converting instead to liters per minute gives \(4.80/60=0.0800\) L/min, which agrees with the narrative at its displayed precision. This is a formatting difference, not a conflicting estimate. The report should still make the units explicit, because the number \(4.80\) without a unit is ambiguous.
Rounding should also be assessed against the underlying precision, not by demanding identical last digits in every section. If a value \(x\) is rounded to \(d\) decimal places using the usual nearest-value rule, its display represents values within approximately \(0.5\times10^{-d}\) of \(x\), subject to the tie-breaking convention. Thus, displays of \(2.35\) and \(2.4\) can both arise from a value near \(2.35\) when one is rounded to two decimals and the other to one decimal. If the displays cannot both be obtained from one underlying value under the stated rounding rules, the difference needs investigation.
Worked Example: Distinguishing Rounding From a Changed Estimate
In a hypothetical report, a table gives a model-predicted concentration of \(12.46\) mg/L, and the narrative gives \(12.5\) mg/L. A figure label instead says \(12.8\) mg/L. Determine which displays can be reconciled by rounding to one decimal place.
Rounding \(12.46\) to one decimal place gives \(12.5\) mg/L, because the hundredths digit is 6. The table and narrative are consistent. The figure label \(12.8\) mg/L is not the same value rounded to one decimal place: \(12.46\) lies \(0.34\) mg/L below \(12.8\), more than the \(0.05\) mg/L rounding tolerance for one decimal place. It is a discrepancy that must be traced to the figure source, data, or version; the available numbers alone do not establish which is correct.
Analysis-Set Counts and Denominators
A sample size is meaningful only with a defined unit of counting. In a pharmacometric analysis, one participant may contribute multiple samples or records. “\(n=120\)” could therefore mean 120 participants, 120 records, or another count. A caption that uses \(n\) without saying what it counts can appear to conflict with a table even when the counts differ for a legitimate reason.
Worked Example: Reconciling Participant Counts in a Figure Caption
Consider a hypothetical analysis with 120 enrolled participants. The model fit used evaluable pharmacokinetic data from 116 participants. The parameter table states “analysis set: 116 participants,” but a figure caption says “model fit, \(n=120\) participants.” The analysis log and model input dataset both identify 116 included participants.
The enrollment count and model-analysis count answer different questions: 120 were enrolled, while 116 contributed data to this fit. Because the caption specifically describes the model fit, its stated denominator conflicts with the analysis log and input dataset. The evidence supports changing the caption to “model fit, 116 participants,” while retaining 120 as the enrollment count where that is the intended quantity. The correction should preserve the distinction and be checked against all other locations using the same count.
This check does not establish why four participants were not included. That explanation belongs with the relevant analysis-set definition and should not be invented from the count alone.
Count checks should compare like with like: participants with participants, observations with observations, and the same inclusion rules with the same inclusion rules. If counts differ because a figure shows a subset or a later time point, the report should state that clearly rather than forcing the values to match.
Intervals, Figures, and Written Conclusions
A table, a plotted interval, and a sentence about a decision often convey the same uncertainty in different forms. Check that the plotted point estimate and limits correspond to the table, that the axis and caption identify the quantity and interval type, and that the prose conclusion follows from the reported result. Do not infer equivalence between intervals simply because their lines look close on a graph.
The reason is that an inverted confidence interval contains precisely the parameter values that would not be rejected by the corresponding test. Therefore, if the null value is outside the interval, it would be rejected by that test; if it is inside, it would not be rejected. The relationship does not justify comparing an interval from one model with a test from another, or treating a prediction interval as a confidence interval for an estimate.
Worked Example: Checking an Interval Against a Figure and Conclusion
A hypothetical report compares a model-based exposure ratio between two groups. The table reports an estimated ratio of \(0.82\) with a 95% confidence interval of \(0.68\) to \(0.99\). The narrative says the interval excludes 1 and describes the estimated ratio as below 1. A figure, however, appears to show an upper confidence limit of \(1.02\). Assume the table and figure are intended to show the same estimate, analysis, and interval method.
For a ratio, the no-difference value is 1. The table’s interval has upper limit \(0.99<1\), so it excludes 1. Under the stated matching two-sided 5% procedure, this is consistent with rejecting \(H_0\) that the ratio equals 1. It supports describing the estimated ratio as below 1, but it does not by itself establish a causal effect.
The figure’s limit \(1.02>1\) would include the null value and contradict both the table and the corresponding narrative. Since the report states that the displays refer to the same analysis and method, this is not a rounding difference. The reviewer should trace the plotted data and figure-generation source to the controlling analysis output. If that output confirms the table limits, correct the figure; if it instead confirms the plotted limit, the table and prose also require correction. Until that trace is completed, the report contains an unresolved inconsistency.
Diagnostics, Resolution, and Limits
Useful diagnostics are simple but systematic: compare values after unit conversion; verify decimal precision against the source value; check whether all counts refer to the same analysis set; and compare interval endpoints and plotted scales rather than relying on visual impression. A discrepancy that remains should be investigated through the output or artifact that generated the result. The corrected report should identify the affected locations and preserve a record of what changed and why.
A manual cross-reference is flexible and can catch meaning errors that a numerical check cannot, such as a participant count mislabeled as an observation count. Automated checks can efficiently flag exact differences in values, units, or sample sizes when report content is generated from structured outputs. Neither approach is sufficient alone: automated matching can miss a wrong label or mismatched estimand, while manual review can overlook repeated values or transcription errors. Both should be grounded in the same defined result and source.
Consistency does not prove that the model is correct, that the analysis is appropriate, or that a claim is justified. It shows only that the report presents its specified results coherently across sections. A perfectly consistent report may still have limitations in its data, assumptions, precision, or scope; those require their own evaluation. Conversely, a mismatch is a signal to trace the result, not proof that the underlying model or analysis is invalid.
- Do not resolve by majority vote. Three report sections can repeat the same transcription error. Check the controlling output.
- Do not confuse a display difference with a scientific difference. Convert units and account for rounding before raising a discrepancy.
- Do not compare undefined denominators. Specify whether a count is participants, records, or another unit.
- Do not let the narrative override the interval. Check that the conclusion follows from the matching estimate and uncertainty summary.
Check Your Understanding
Answer each question, then compare your reasoning with the concise solutions below.
- A table reports \(1.20\) mg/L and a figure reports \(1200\) µg/L for the same estimate. Are these necessarily inconsistent? Show the conversion.
- A report says 90 participants were enrolled and \(n=84\) were included in the model fit. What must be checked before concluding that the report is inconsistent?
- A matching two-sided test uses \(\alpha=0.05\), and its 95% confidence interval is \(1.03\) to \(1.40\) for a ratio whose null value is 1. What follows about the null value under the stated matching conditions?
- A table gives \(3.141\) and a narrative gives \(3.14\). State one condition under which these displays are consistent and one reason to investigate further.
Solutions. (1) They agree: \(1.20\ \mathrm{mg/L}\times1000\ \mathrm{\mu g/mg}=1200\ \mathrm{\mu g/L}\). (2) Check what each count represents and whether both refer to the same population, inclusion criteria, and analysis; enrollment and model-fit counts can legitimately differ. (3) The interval excludes 1, so the corresponding test rejects \(H_0\) at the 5% level, provided the test and interval are based on the same method, data, and estimand. (4) They are consistent if the narrative is the table value rounded to two decimal places. Investigate if both are claimed to be exact, if their units or definitions differ, or if the underlying source does not support the stated rounding.