Why Use Independent Test Cases?
A model can run successfully and still implement the wrong input conversion, parameter transformation, or prediction equation. A useful component test asks a narrower question: for this precisely specified input, does this part of the model produce the result required by its specification? Small cases make the answer inspectable. They are especially useful when a full estimation or simulation run would make it difficult to tell which component caused a discrepancy.
An independent test case has an expected result determined without relying on the same calculation pathway being tested. For example, an expected concentration can be calculated from a stated equation and simple values, then compared with the model output. Copying a model's output into a test file does not provide an independent check: it merely preserves whatever that version of the model produced.
The focus here is deterministic component behavior, not statistical inference. Tests should state the component's contract, the input and units, the expected output, and how close the actual output must be to that expectation. A test is only meaningful if the assumptions and input conventions are explicit.
Specify the Contract Before Testing
A test cannot determine whether a result is correct until the intended behavior is defined. For a dose input, the contract might specify accepted units, the internal unit, route, dose time, and how invalid amounts are handled. For a parameter transformation, it should specify the parameter scale and transformation. For a prediction, it must specify the model, input values, time, and output units.
Tests also need to distinguish behavior from implementation details. For example, a dose of zero may be valid and should produce a zero dose event, while a negative dose may be rejected under the stated input contract. Another interface could have different validation rules. The test must check the agreed rule rather than assume that every model handles input identically.
For numeric outputs, floating-point calculations may differ by a small amount because of rounding. A common comparison rule for an actual value \(y\) and expected value \(y^*\) is:
Here, \(\mathrm{tol}_{\mathrm{abs}}\) is an absolute tolerance in the output's units, and \(\mathrm{tol}_{\mathrm{rel}}\) is a dimensionless relative tolerance. An absolute tolerance is important near zero, where relative error alone is unhelpful. Tolerances should be tight enough to catch meaningful errors, but not so tight that harmless rounding differences cause failures.
Write down the component's intended inputs, units, valid range, and expected behavior, including any specified error handling.
Use simple values for which the expected result can be computed independently. Include cases that exercise distinct rules, not merely several similar ordinary inputs.
Show the conversion, transformation, or equation used to obtain the expected result. Keep its source separate from the function being tested.
Apply a stated tolerance to numeric outputs, or check the specified error behavior for invalid inputs. A failure indicates a discrepancy to investigate; it does not by itself identify the cause.
Worked Example: Checking Dose-Unit Conversion
Worked Example: Checking Dose-Unit Conversion
Suppose an input interface accepts a dose in grams, while the model's dose event is recorded in milligrams. The contract specifies that \(1\ \mathrm{g}=1000\ \mathrm{mg}\), that a zero dose is accepted as zero, and that a negative dose is rejected. Design independent cases for these behaviors.
Positive dose. Enter \(0.25\ \mathrm{g}\). The expected internal amount is obtained directly from the unit conversion:
The test passes if the recorded dose event is \(250\ \mathrm{mg}\), within a suitable tolerance for the implementation. This case checks both the conversion factor and the resulting unit.
Zero dose. Enter \(0\ \mathrm{g}\). The expected internal amount is \(0\times1000=0\ \mathrm{mg}\). This case checks that zero is not mistakenly treated as missing or replaced by a default dose.
Invalid dose. Enter \(-0.25\ \mathrm{g}\). Under the stated contract, the expected behavior is rejection with the specified input error, not conversion to \(-250\ \mathrm{mg}\) and silent continuation. This test checks validation as well as arithmetic.
These cases have separate purposes. The positive amount checks unit conversion; zero checks a valid edge case; the negative amount checks a stated validation rule. If the specification does not say what should happen to negative doses, the test designer must resolve that ambiguity before treating either acceptance or rejection as correct.
Test Parameter Transformations Directly
Model parameters are often represented on a scale that guarantees a constraint after transformation. A log transformation maps any real-valued input to a positive parameter. A logistic, or logit-inverse, transformation maps a real-valued input to a value strictly between zero and one. Tests should verify the transformation actually specified by the model; confusing a parameter with its transformed representation can yield plausible-looking but incorrect values.
Worked Example: Testing Log and Logistic Transformations
Suppose a model uses \(\theta=\exp(\eta)\) for a positive clearance parameter and \(p=1/(1+\exp(-z))\) for a fraction. Test the log transformation at \(\eta=\ln(4)\), and the logistic transformation at \(z=0\) and \(z=\ln(3)\).
For the positive parameter:
Thus the expected transformed clearance is \(4\) in the clearance units specified by the model. A result of \(\ln(4)\), approximately \(1.386\), would indicate that the model may have returned the unconstrained value instead of applying the transformation.
For \(z=0\):
For \(z=\ln(3)\), \(\exp(-z)=1/3\), so:
These cases check a central value and a noncentral value. The expected results are calculated directly from the specified formulas, not by passing the same values through a second call to the model's transformation function. In a fuller test set, values on both sides of zero and values near, but not at, the range limits can help reveal sign or overflow problems. The logistic transformation produces values strictly inside \((0,1)\) for finite \(z\); it does not produce exactly zero or one.
Check Predictions Against an Independent Calculation
The earlier material on hand-checking a one-compartment prediction provides a useful setting for a prediction test. Consider an intravenous bolus model with dose \(D\), volume \(V\), and clearance \(CL\), and assume a single dose at time zero, positive \(V\) and \(CL\), no additional doses, and no residual or observation error in the predicted concentration. The concentration at time \(t\geq0\) is:
The formula also gives a simple justification for a useful check. For fixed \(V\), \(CL\), and \(t\), concentration is proportional to dose because \(D\) is a multiplicative factor. Therefore doubling the dose should double the predicted concentration at the same time. This relationship is a secondary check on dose handling and prediction calculations.
Worked Example: Testing a One-Compartment Prediction
Use a hypothetical intravenous bolus dose of \(100\ \mathrm{mg}\), \(V=20\ \mathrm{L}\), \(CL=4\ \mathrm{L/h}\), and \(t=5\ \mathrm{h}\). Assume the model reports concentration in \(\mathrm{mg/L}\). Calculate the expected prediction independently.
First, the initial concentration term is:
The elimination-rate term is \(CL/V=(4\ \mathrm{L/h})/(20\ \mathrm{L})=0.2\ \mathrm{h}^{-1}\), so its product with time is \(0.2\ \mathrm{h}^{-1}\times5\ \mathrm{h}=1\). Substitution gives:
A component test can compare the model's prediction with \(1.8394\ \mathrm{mg/L}\), using, for example, \(\mathrm{tol}_{\mathrm{abs}}=0.000001\ \mathrm{mg/L}\) and \(\mathrm{tol}_{\mathrm{rel}}=0.000001\). These are illustrative tolerances; the appropriate values depend on the numerical precision and the model's output conventions.
Two companion checks make the case more diagnostic. At \(t=0\), the formula gives \(C(0)=5.000\ \mathrm{mg/L}\). At \(t=5\ \mathrm{h}\), doubling the dose to \(200\ \mathrm{mg}\) doubles the expected prediction to \(3.6788\ \mathrm{mg/L}\). If the first prediction fails but the initial concentration is correct, the time-dependent part deserves scrutiny. If the doubled-dose result is not twice the original, dose scaling or event handling may be implicated.
Diagnostics, Alternatives, and Limitations
A failed test identifies disagreement between the specified expectation and the model output, not necessarily the faulty line of code. First check that the case uses the intended units, route, time origin, parameter scale, and output definition. Then inspect the component boundaries: an error in a unit conversion can propagate into an otherwise correct prediction, while an incorrect transformation can make an input appear to be a prediction-equation error.
Tests are more informative when their expected results are independent and cases target different failure modes. Useful signs of a weak test include:
- The expected value was copied from the implementation being tested.
- The test checks only a typical input and omits a meaningful edge case, such as a zero dose.
- The expected value is rounded so heavily that a consequential error still passes.
- The tolerance is so tight that harmless floating-point variation fails, or so loose that a wrong result passes.
- The expected behavior for invalid inputs is not specified, leaving the test to encode an undocumented assumption.
For a deterministic component, a direct analytic calculation is often clearer than checking only whether the program completes. A second implementation can be helpful, but it is not automatically independent: if it copies the same formula, convention, or coding error, both implementations may agree and still be wrong. A full model run can reveal interactions among components, but may be harder to diagnose than a small test. Visual inspection can identify conspicuous patterns, yet it does not replace a numeric expected result for a specific input.
Small test cases have limits. Passing them does not prove that every input, parameter combination, or model pathway is correct. A case may miss an error that occurs only at an extreme value, with repeated events, or under a different route. Test results also depend on the validity of the specification and the oracle calculation. Use component tests as focused checks, not as evidence that the entire model is correct.
Check Your Understanding
For each question, decide what the test should expect and why. Solutions follow the questions.
- A model accepts grams and stores milligrams. What internal dose should a \(0.08\ \mathrm{g}\) input produce?
- Under \(\theta=\exp(\eta)\), what is the expected value of \(\theta\) when \(\eta=0\)? What class of error could a result of zero suggest?
- Under \(p=1/(1+\exp(-z))\), what is \(p\) when \(z=\ln(3)\)?
- For the prediction example, what concentration is expected at time zero, and what happens at \(5\ \mathrm{h}\) if the dose is halved while the other inputs stay fixed?
- Why is copying a model's output into the expected-value file not an independent test oracle?
Solutions. (1) \(0.08\times1000=80\ \mathrm{mg}\). (2) \(\exp(0)=1\), a positive value; a result of zero could indicate that the transformation was omitted or incorrectly implemented. (3) Since \(\exp(-\ln3)=1/3\), \(p=1/(1+1/3)=0.7500\). (4) At time zero, \(C(0)=100/20=5.000\ \mathrm{mg/L}\). Halving the dose to \(50\ \mathrm{mg}\) halves the prediction at \(5\ \mathrm{h}\), giving \(1.8394/2=0.9197\ \mathrm{mg/L}\). (5) The implementation supplies both the tested result and its supposed expectation, so the same error can appear in both and remain undetected.