Why Change Control Matters
A deployed dosing service is not a single model file. Recommendations depend on a chain of components: model parameters and equations, executable code, incoming data mappings, and the interface that displays results. A seemingly small change in one component can alter a dose recommendation, suppress a warning, or change what a clinician sees. Change control is the defined process for assessing, testing, approving, releasing, and recording such changes.
The goal is not to prevent all change. Models may need updated parameters, software defects need correction, and data feeds or user interfaces may need revision. The goal is to make each change deliberate and traceable, and to detect unintended effects before the changed version is used for patient care. Audit Trails for Reconstructing Recommendations address how to reconstruct a past recommendation; change control addresses how a proposed change is evaluated before release and how the released version is identified.
A change request should identify the affected component, the reason for the change, the proposed implementation, and the intended behavior. The affected components may include:
- Model: equations, parameter values, covariate effects, or the population from which a model is intended to be used.
- Code: algorithms, numerical routines, dose-selection logic, or error handling.
- Data mapping: field definitions, units, code translations, missing-value handling, and rules for selecting the relevant record.
- Interface: display, rounding, warnings, labels, or the transfer of a recommendation to another system.
The proposed change must be compared with a defined baseline: the currently approved model, code, mappings, and interface. Without that baseline, reviewers cannot reliably tell what changed or identify the version against which test results should be interpreted.
Assessing Scope and Risk Before Testing
Impact assessment starts by tracing the path from the changed component to the clinical output. For example, a revised parameter may affect predicted exposure and dose selection; a mapping change may alter the patient covariate supplied to that model; an interface change may leave the internal recommendation unchanged but display or transmit it differently. Assess both direct effects and dependencies downstream.
The assessment should state the intended behavior, the plausible failure modes, and the evidence required to support release. The level of testing should match the consequences and reach of the change. A spelling correction in a nonclinical help label usually has a narrower impact than a change to a covariate mapping that influences all recommendations. A narrowly scoped change still needs a check that it has not altered related behavior.
Tests are meaningful only if their conditions are explicit. Reference inputs and expected outputs must be defined, including units and rules for rounding or missing data. For a deterministic service, repeated runs with the same version and inputs should give the same result. If a component is stochastic or depends on external services, the test plan must specify how variability and dependencies are controlled or evaluated. The test environment should represent the release configuration closely enough that passing results are relevant to the deployed system.
Describe the reason, requested behavior, affected components, and current approved baseline. Assign an identifier so the request and its evidence can be connected.
Identify the inputs, calculations, outputs, warnings, and workflows that could be affected. Include dependencies across the model, code, mappings, and interface.
Choose representative cases, expected results, tolerances where appropriate, and criteria for both acceptable differences and blocking failures.
Run the planned tests against the candidate version, investigate discrepancies, and have an appropriate reviewer assess whether evidence satisfies the stated criteria.
Identify the exact released artifacts, confirm deployment and a small set of critical behaviors, and retain a defined route to restore the prior version if necessary.
Quantifying a Change in Model Output
An impact assessment should quantify changes when the model makes that possible. A changed prediction is not automatically a defect: it may be the intended effect of an updated parameter. The comparison should state which inputs were held constant, which version was changed, the output units, and whether the size or pattern of the difference is acceptable for the intended use.
Worked Example: A Changed Clearance Parameter
Suppose a hypothetical model predicts steady-state exposure as dose divided by clearance. For one fixed test case, the dose is 600 mg. The approved model uses clearance of 5.0 L/h, and a proposed parameter update uses 5.4 L/h. Calculate and interpret the predicted exposure change, assuming the model equation and dose are otherwise unchanged.
Check the comparison assumptions. This calculation isolates the parameter change: the same 600 mg dose and the same equation are used for both versions. It is an impact check for this hypothetical case, not evidence by itself that the updated parameter is appropriate for all patients.
The model gives \(AUC = \text{dose}/CL\), where the resulting units are mg·h/L:
For the candidate parameter:
The absolute change is \(111.1-120=-8.9\) mg·h/L. Relative to the old prediction, it is approximately \((-8.9/120)\times100=-7.41\%\). The candidate predicts about 7.41% lower exposure for this test case. The result should be compared with the impact criteria defined for this change; it is not enough to label the difference acceptable or unacceptable without such criteria and clinical review.
The comparison has a simple general justification. If the dose \(D\) is fixed and \(AUC=D/CL\), then the ratio of new to old predictions is:
The dose cancels because it is held constant. This explains why increasing clearance from 5.0 to 5.4 L/h lowers the predicted exposure, and why a relative change in clearance need not produce an equal-sized relative change in exposure.
Regression Tests for Mappings and Interfaces
Regression testing should include both the change itself and important previously verified behavior. A useful test set is not just a large count of typical inputs. It should cover relevant boundaries, unusual but valid inputs, missing or conflicting data where those are handled, and cases whose outputs trigger important warnings or workflow branches. Each case should have a stated expected result or a defined comparison rule.
Numerical comparisons may use an exact match, a tolerance, or an explicitly approved difference. A tolerance must have a rationale and units; it should not be enlarged simply to make a failing test pass. For displayed doses, test the underlying calculated value as well as the displayed and transmitted value. A correct internal result can still produce an incorrect clinical display through a unit conversion, rounding rule, or interface defect.
Worked Example: Detecting a Data-Mapping Error
A hypothetical model calculates an adjustment factor \(F=\sqrt{100/C}\), where \(C\) is a mapped covariate in the model's specified units. For one test record, the verified source value is \(C=81.9\), but a candidate mapping incorrectly supplies \(C=25\). Compare the factors and assess the impact.
Verify the inputs first. The test assumes the source record and its intended units have been independently checked. It also assumes the same model equation is used in both calculations, so the mapping is the only changed component.
With the verified value, \(F_{\text{correct}}=\sqrt{100/81.9}=\sqrt{1.2210}\approx1.105\). With the candidate mapping, \(F_{\text{mapped}}=\sqrt{100/25}=\sqrt{4}=2.000\). The ratio of mapped to correct factors is:
Thus the mapped factor is about 1.81 times as large, an increase of approximately \(((2.000-1.105)/1.105)\times100=81.0\%\). This is a substantial discrepancy for this test input and should block release until the mapping is corrected or the difference is otherwise explained and approved. It does not establish that a final dose would also increase by 81%; that conclusion would require the full dose-selection logic and downstream rules.
Worked Example: Testing a Dose Display Change
A hypothetical interface displays a calculated dose rounded to the nearest 25 mg. In a regression suite of 40 cases, two cases have raw calculated values of 488 mg and 588 mg. The approved interface displays 500 mg and 600 mg, respectively. A candidate interface truncates values to the lower multiple of 25 mg. Determine the candidate displays and whether the suite passes if all clinically critical cases must match their approved displays exactly.
Check the test conditions. The raw values, rounding specification, and critical-case status are fixed by the test definition. The comparison concerns the display transformation; it does not assume that the dose calculation itself has changed.
For 488 mg, the closest multiples of 25 mg are 475 mg and 500 mg. Their distances are 13 mg and 12 mg, so nearest-25 rounding gives 500 mg; truncation gives 475 mg. For 588 mg, the closest multiples are 575 mg and 600 mg, at distances 13 mg and 12 mg. Nearest-25 rounding gives 600 mg; truncation gives 575 mg.
Both candidate displays therefore differ from the approved display by 25 mg. If these are clinically critical cases and the release criterion requires exact display agreement, the candidate fails, even if the other 38 cases pass. A score of \(38/40=95.0\%\) passing cannot override a criterion that requires every critical case to pass. The interface must be corrected or the intended rounding behavior must be formally reassessed before release.
Review, Release, and Failure Modes
A release decision should connect the request, impact assessment, test evidence, review, and exact candidate version. It should state which tests passed, which failed, how discrepancies were resolved, and what residual limitations remain. A difference may be acceptable when it is the planned effect of a change and is supported by the agreed criteria; an unexplained difference is not made acceptable merely by documenting it.
Where practical, avoid relying on the person who made a change as the only reviewer of its impact and test evidence. Before release, verify that the intended version of each affected artifact is included and that the deployment configuration corresponds to the tested candidate. A release check should confirm critical end-to-end behaviors, including that inputs are interpreted correctly and displayed recommendations and warnings are transmitted as intended. A rollback plan needs to identify the prior approved version and the conditions under which restoration would be considered.
Common warning signs include unexplained changes in a reference output, tests run against a different version from the one proposed for release, missing test cases for affected boundaries, and inconsistent outputs between the model and interface. If stochastic behavior or external dependencies cause variable results, investigate whether the variability is expected and controlled before interpreting a pass or failure. Do not hide a mismatch by changing the expected result without recording the reason and obtaining appropriate review.
A full revalidation of every component may be unnecessarily burdensome for a genuinely limited change, while testing only the changed line of code may miss important dependencies. A risk-based regression suite is the practical middle course: test the changed function, the plausible downstream effects, and representative previously verified behavior. Its limitation is that tests only detect failures that the test cases and criteria can reveal. They cannot establish that the original model is clinically appropriate, that every possible input is safe, or that an untested workflow behaves correctly. Keep those limits explicit when interpreting passing results.
Check Your Understanding
Use the definitions and release principles above. Each solution is provided after the questions.
- A mapping change converts a laboratory value from one unit to another. Name three aspects that should be assessed or tested before release.
- A candidate version changes a prediction from 80 to 84 units. What is the absolute change and the percentage change relative to the approved prediction?
- Why is a test suite with 99% passing results not necessarily sufficient for release?
- A code change passes unit tests but has not been tested through the interface that displays the recommendation. What important class of failure remains untested?
Solutions. (1) Verify the source and target units and conversion rule; test representative values, including relevant boundaries; and trace whether the mapped value changes model outputs, warnings, or downstream display. (2) The absolute change is \(84-80=4\) units. The relative change is \((4/80)\times100=5.00\%\), an increase of 5.00% relative to the approved prediction. (3) The remaining 1% may contain a critical case, and release criteria may require all such cases to pass; a percentage alone does not convey failure severity. (4) Interface failures remain possible, including incorrect rounding, labels, warnings, or transfer of the calculated result. End-to-end tests are needed for those behaviors.