Why Reference Test Cases Matter
A deployed dosing calculation is a chain of transformations: clinical records are interpreted, units and event times are aligned, model inputs are assembled, calculations are performed, and results are returned to a user or another system. A correct model equation cannot protect a service from a wrong unit conversion or an incorrectly mapped dose time. Computational verification uses known inputs and expected outputs to detect such errors before and after changes to the software.
A reference test case is more than a number saved from a previous program run. It is a documented, independently checked example specifying the relevant inputs, assumptions, units, expected output, and comparison rule. The expected result should come from a trusted derivation, a separately implemented calculation, or another justified reference—not merely from the same code being tested. Otherwise, the test can reproduce the same mistake on both sides of the comparison.
Tests establish that specified software behavior agrees with specified expectations for the cases examined. They do not show that the model is clinically appropriate, that the expected outputs are correct for every patient, or that all possible input combinations have been tested. These distinctions matter when interpreting a clean test report.
Specify the Test Before Running It
A useful case states the model and conditions clearly enough that another reviewer can reproduce the expected result. For a deterministic calculation, the same model version and inputs should produce the same result, subject to the declared numerical tolerance. Record units explicitly, including the units expected at each interface. If the result depends on dose history, sample time, or patient covariates, include those values and explain their interpretation.
For floating-point outputs, exact equality is often an inappropriate pass criterion. A common rule accepts an output \(y_{\mathrm{test}}\) when its difference from the reference \(y_{\mathrm{ref}}\) satisfies:
Here, \(\mathrm{atol}\) is an absolute tolerance in the output's units and \(\mathrm{rtol}\) is a dimensionless relative tolerance. The maximum allows an absolute comparison near zero and a scale-sensitive comparison when the reference is larger. Tolerances should reflect justified rounding and the computation being checked; making them so wide that a clinically important discrepancy passes defeats the test. Discrete outputs—such as a route category or an error code—usually require exact agreement.
The model assumptions must be part of the case, not silently inferred from its output. For example, a concentration formula for repeated doses may assume identical doses at a fixed interval, linear one-compartment elimination, and steady state. A test using that formula is meaningful only when its inputs and stated scenario meet those assumptions.
Worked Example: Unit Test for a Steady-State Trough Calculation
Consider a hypothetical one-compartment model with repeated intravenous bolus doses, first-order elimination, a fixed dosing interval, and steady state. Let \(D/V=10.0\ \mathrm{mg/L}\), and let \(e^{-k\tau}=0.0907180\), where \(k\) is the elimination rate constant and \(\tau\) is the interval. The steady-state trough immediately before the next dose is:
Substitute the hypothetical inputs:
A reference case can therefore record the model assumptions, \(D/V=10.0\ \mathrm{mg/L}\), \(e^{-k\tau}=0.0907180\), and an expected concentration of approximately \(0.997688\ \mathrm{mg/L}\). If the output is compared using \(\mathrm{rtol}=10^{-5}\) and \(\mathrm{atol}=10^{-8}\ \mathrm{mg/L}\), the allowed difference is about \(9.98\times10^{-6}\ \mathrm{mg/L}\). The test should pass for an output within that bound and fail outside it. This checks the calculation for this case; it does not establish that the assumptions fit a particular patient.
From Calculation Checks to Pipeline Checks
A unit test isolates the model calculation by supplying well-defined inputs directly. An integration test instead follows those inputs through connected parts of the service. That distinction helps make failures interpretable: if the direct model test passes but an integration case fails, investigate the mapping, units, event selection, or output handling before changing the model calculation.
A practical test set should include typical valid inputs, values near important decision boundaries, and invalid or incomplete inputs. It should also check transformations that commonly alter meaning: mass units, time units, date-time conventions, identifiers, and the distinction between a dose event and a sample event. A test case should specify expected behavior for missing or invalid information, not merely omit those cases.
Worked Example: Integration Test for Dose, Clearance, and Interval Mapping
Suppose a hypothetical dosing service receives a dose of \(500\ \mathrm{mg}\), clearance of \(4.00\ \mathrm{L/h}\), and a dosing interval of \(12.0\ \mathrm{h}\). The service is specified to calculate average steady-state concentration for a linear model as dose divided by clearance times interval. Assume the dose is correctly mapped in milligrams, clearance in litres per hour, and the interval in hours.
The reference test should verify more than the final value. It can assert that the incoming dose is interpreted as \(500\ \mathrm{mg}\), that the interval reaches the model as \(12.0\ \mathrm{h}\), and that the returned concentration is about \(10.42\ \mathrm{mg/L}\), within the declared tolerance. If a mapping mistakenly supplies \(12.0\) minutes while the model expects hours, the expected output will not be reproduced. Checking intermediate interface values helps locate that failure rather than obscuring it behind a single final-output mismatch.
This case assumes the stated average-concentration formula and consistent units. It is not a claim that average concentration alone determines an appropriate dose for an individual; it tests the specified computational path.
Worked Example: Checking Event-Time Alignment
In a hypothetical single-dose scenario, a \(250\ \mathrm{mg}\) dose is administered at time \(t=0\), the apparent volume of distribution is \(25.0\ \mathrm{L}\), and \(k=0.100\ \mathrm{h^{-1}}\). A sample is taken \(6.00\ \mathrm{h}\) after the dose. Under a one-compartment model with first-order elimination, the expected concentration is:
An integration case should give the service both event timestamps and verify that it identifies the sample as \(6.00\ \mathrm{h}\) after the dose, then returns approximately \(5.488\ \mathrm{mg/L}\). If a time mapping interprets the elapsed value as \(6.00\) minutes, the test will expose the mismatch. The expected calculation assumes the stated dose, timing, volume, and elimination model; the test does not validate those assumptions for real clinical use.
Justifying the Comparison Rule
The tolerance rule combines two useful scales. When \(|y_{\mathrm{ref}}|\) is large enough that \(\mathrm{rtol}|y_{\mathrm{ref}}|\) exceeds \(\mathrm{atol}\), the permitted difference is proportional to the reference magnitude. When the reference is near zero, the absolute tolerance prevents the relative term from demanding an unrealistically tiny difference.
This follows directly by considering the two regions. If \(\mathrm{rtol}|y_{\mathrm{ref}}|\geq\mathrm{atol}\), then the maximum is the relative allowance and the test requires \(|y_{\mathrm{test}}-y_{\mathrm{ref}}|\leq\mathrm{rtol}|y_{\mathrm{ref}}|\). If \(\mathrm{rtol}|y_{\mathrm{ref}}|<\mathrm{atol}\), the maximum is the absolute allowance and the test requires \(|y_{\mathrm{test}}-y_{\mathrm{ref}}|\leq\mathrm{atol}\). Thus a single rule avoids division by a reference value near zero while retaining a relative scale for larger values. The tolerances still require a reasoned choice in the output units.
For a vector of outputs, apply the comparison component by component, using the appropriate units and tolerances for each component. Do not use one broad tolerance across quantities with different scales or meanings. If an output is rounded for display, test the underlying numerical result separately from the display-formatting rule.
Test Selection, Diagnostics, and Alternatives
A compact reference suite should be designed to make failures informative. Include ordinary inputs, edge cases that are valid under the specification, and cases expected to produce a defined error or no recommendation. For example, a negative dose or an unsupported route should not quietly enter a calculation that assumes a positive dose and an intravenous bolus. The expected result may be a specific validation error rather than a concentration.
- Model unit mismatch: direct inputs are correct but the calculated output differs; inspect equation implementation, parameter ordering, and reference derivation.
- Mapping or conversion mismatch: a direct unit test passes but an integration test fails; inspect source fields, units, and transformations at interfaces.
- Time-alignment mismatch: the case fails when dose and sample histories are included; verify event selection, timestamps, and interval interpretation.
- Output mismatch: the internal numerical result is correct but the returned value or units differ; inspect serialization, rounding, and labels.
- Inadequate test case: tests pass despite a known defect; add a case that exercises the omitted input pattern or boundary.
Reference tests are one approach among several. They are strongest when a small number of carefully derived cases have unambiguous expected outputs. Property-based tests check general relations across many generated inputs, but a property can be implemented incorrectly or fail to reflect the intended clinical specification. Code review can identify design and logic concerns that a finite set of cases misses, but it does not execute the system. Simulation-based checks can explore broad ranges, yet they still depend on the simulated scenarios and do not replace trusted reference values.
| Approach | Main question answered | Important limitation |
|---|---|---|
| Reference test cases | Does this specified input produce the specified output? | Only tested cases and assertions are covered. |
| Property-based tests | Does a general relation hold across many inputs? | The chosen property may not fully specify correct clinical behavior. |
| Code review | Is the implementation logic understandable and plausible? | Review alone does not demonstrate executed behavior. |
| Simulation-based checks | How does behavior vary across generated scenarios? | Results depend on scenario generation and do not provide an independent oracle by themselves. |
A test suite should be run whenever relevant model code, data mappings, interfaces, or dependencies change. Reference cases must also be versioned and reviewed: changing an expected output simply to make a failing test pass can conceal a regression. First establish whether the code changed incorrectly, the reference was wrong, or the intended specification changed; document the reason for any revised expectation.
Common Pitfalls and Limits
A common mistake is to treat agreement with a saved output as proof that the calculation is correct. The reasoning is: “The output matches the reference, therefore the model is correct.” The correction is that a passing case establishes agreement only for that input, output, implementation path, and acceptance rule. If the expected value was generated by the same defective code, the agreement may be circular. Independent derivation and varied test cases reduce that risk but cannot remove all uncertainty.
Another pitfall is using an excessively large tolerance to suppress small differences without checking their meaning. A tolerance should be justified in the output's units and by the required precision; it should not erase a discrepancy that could change a recommendation or its interpretation. Conversely, exact equality can make a valid test brittle when a calculation is represented with slightly different floating-point rounding. Keep computational comparison distinct from clinical validation: reference tests verify software behavior, while model appropriateness and clinical consequences require their own evaluation.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- A model function returns \(2.50\ \mathrm{mg/L}\) when the reference is \(2.500\ \mathrm{mg/L}\). With \(\mathrm{rtol}=10^{-4}\) and \(\mathrm{atol}=10^{-6}\ \mathrm{mg/L}\), does the result pass?
- A direct model unit test passes, but a service-level test using the same scenario fails. Which parts of the service should be examined first, and why?
- For \(D=100\ \mathrm{mg}\), \(CL=5.00\ \mathrm{L/h}\), and \(\tau=10.0\ \mathrm{h}\), calculate the average steady-state concentration under \(C_{\mathrm{avg,ss}}=D/(CL\tau)\).
- Why is a reference output copied from the software under test a weak oracle?
Solutions. (1) The difference is zero, so it passes. More generally, the allowed difference is \(\max(10^{-6},10^{-4}\times2.500)=0.000250\ \mathrm{mg/L}\). (2) Check input mapping, unit conversion, event selection and timing, then output formatting; the passing direct test makes the isolated calculation a less likely source, though it does not rule out all model-call differences. (3) \(C_{\mathrm{avg,ss}}=100/[(5.00)(10.0)]=2.00\ \mathrm{mg/L}\). (4) The tested code and expected value may share the same error, so agreement does not independently verify the calculation; derive or obtain the reference through a separately checked route.