Why Domain Checks Matter
A model can produce a precise-looking dose recommendation for a patient whose clinical profile was never represented in the evidence used to develop or validate that model. The calculation may run, and its output may be numerically plausible, but the relationship the model learned may not apply to that patient. Before a recommendation is acted on, a dosing system therefore needs to ask whether the patient and relevant observations fall within the model’s validated domain.
This check serves a different purpose from an abstention rule based on high predictive uncertainty. A model can be uncertain for a patient well represented in its development data; it can also be confidently wrong when asked to extrapolate beyond those data. Domain screening addresses the latter concern. It does not establish that a recommendation is unsafe, nor does passing the check prove that it is correct. It identifies whether the recommendation is supported by the scope of evidence established for the model.
Specify What the Model Has Been Validated For
A usable domain definition begins with the model’s intended use and the evidence supporting it. It should name the relevant population, clinical setting, covariates, and observations. For example, a model may have been evaluated only in adults receiving a particular formulation, with a specified laboratory method, and with renal-function values represented over a stated range. If the deployment population or measurement process differs materially, a dosing recommendation may involve extrapolation even when each numerical input appears ordinary in isolation.
Domain boundaries can be hard or statistical. A hard boundary is an explicit scope limit, such as a supported age range or a covariate category included in validation. A statistical boundary describes the region of covariate combinations represented in the validation data. A patient might satisfy every one-variable range yet have an unusual combination of values. Both kinds of boundary can be necessary: marginal ranges do not describe joint support.
The boundary should be established before evaluating individual deployment cases. It may be based on the validation cohort, external evidence, and clinical scope, but it should not be chosen after seeing whether a particular patient’s recommendation seems convenient. A high empirical percentile can be a practical statistical boundary, but it does not automatically define what is clinically acceptable. The chosen limits and their rationale must be documented.
Three Complementary Detection Strategies
No single detector captures every kind of domain violation. A practical screen may combine explicit scope limits with data-based checks for unusual covariate combinations and observations. The following approaches answer different questions.
- Univariate boundary checks ask whether a value lies within a prespecified interval or allowed set. They are transparent and easy to audit, but can miss unusual combinations of individually common values.
- Category checks ask whether a categorical level was represented and evaluated. An unrepresented category is not made supported merely by assigning it a numerical code or by grouping it with another category without justification.
- Joint-support checks compare a vector of covariates with the multivariate distribution represented in the reference data. They can flag combinations that are unusual even when each component is within its observed marginal range.
Checks may also apply to observations used by the model, such as a laboratory measurement or exposure value, when the validation evidence only supports a particular measurement range or process. A domain screen should state which observations are covered and why. It should not silently replace an out-of-domain value with a boundary value: clamping can conceal extrapolation while producing an output that looks routine.
A Joint-Support Measure and Its Justification
One common joint-support measure is the squared Mahalanobis distance. Let \(x\) be a patient’s vector of \(p\) continuous covariates, and let \(\mu\) and \(\Sigma\) denote the mean vector and covariance matrix used to describe a reference population. The distance measures separation from the reference mean while accounting for covariate scales and correlations:
This is not simply Euclidean distance. If two covariates are strongly correlated, a combination that goes against that correlation can be far from the reference data even when neither value is individually extreme. For a positive-definite \(\Sigma\), a linear transformation that whitens the covariates turns the quadratic form into a sum of squared standardized coordinates. Under the specific assumption that the reference covariates follow a multivariate normal distribution with known \(\mu\) and \(\Sigma\), that sum has a chi-square distribution with \(p\) degrees of freedom:
Thus, under those assumptions, a threshold at the \(q\) quantile of \(\chi^2_p\) encloses probability \(q\) of that reference distribution. In practice, \(\mu\) and \(\Sigma\) are estimated, the data may not be multivariate normal, and the validation sample may be limited. The theoretical threshold therefore needs careful qualification; an empirical threshold, assessed on suitable reference or validation data, may be more appropriate. Either approach flags statistical atypicality, not clinical unsafety.
Worked Example: Flagging a Value Outside a Supported Range
Suppose a hypothetical dosing model was validated for patients aged 18 through 75 years, inclusive. The rule is specified before deployment: ages below 18 or above 75 are out of domain. A patient is 82 years old.
The domain check is \(18 \leq \text{age} \leq 75\). Since \(82 > 75\), the patient is outside the stated age domain by \(82-75=7\) years.
The correct result is an out-of-domain flag, not a conclusion that the dose is necessarily wrong or unsafe. The model-generated recommendation should not be presented as validated for this patient. The system should route the case to the prespecified review or approved non-model pathway. It should not cap the age at 75 and issue the resulting recommendation as though the patient were 75.
Worked Example: Detecting an Unusual Covariate Combination
Consider a hypothetical validation population summarized by two standardized covariates \(Z_1\) and \(Z_2\), each with mean 0 and variance 1 and with correlation 0.80. For this illustration, assume a bivariate normal reference distribution with known parameters, and use the 95th percentile of \(\chi^2_2\), approximately 5.991, as the joint-support threshold. A patient has \(z_1=2\) and \(z_2=-2\).
The covariance matrix and its inverse are \( \Sigma=\begin{pmatrix}1&0.8\\0.8&1\end{pmatrix} \) and \( \Sigma^{-1}=\frac{1}{1-0.8^2}\begin{pmatrix}1&-0.8\\-0.8&1\end{pmatrix} =\frac{1}{0.36}\begin{pmatrix}1&-0.8\\-0.8&1\end{pmatrix} \). Thus,
Since \(40>5.991\), the patient is flagged by this illustrative joint-support rule. Each value is only two standard deviations from its mean, but the opposite-direction combination is unusual under the strong positive correlation in the reference population. The chi-square interpretation depends on the stated normality and known-parameter assumptions; with estimated parameters or a non-normal population, a threshold would need empirical evaluation. The flag indicates atypical covariate support, not a 95% probability that the model is wrong.
Worked Example: An Observation Beyond the Validated Measurement Range
Suppose a hypothetical model was evaluated using drug concentrations from 1.0 to 20.0 mg/L measured with a specified assay. Its prespecified observation-domain rule flags concentrations outside that interval for review. A patient’s measured concentration is 24.0 mg/L.
The value exceeds the upper boundary by \(24.0-20.0=4.0\) mg/L, so it fails the observation-domain check. The system should not use the model’s dose recommendation as if a concentration of 24.0 mg/L had been covered by validation. It should surface the flag and use the designated review pathway. Substituting 20.0 mg/L would hide the fact that the model is being asked to operate outside its supported measurement range.
This example concerns the scope of model evidence, not whether the laboratory result is technically valid. A measurement may be accurate and still lie outside the range for which the dosing model was evaluated.
Turn Flags Into a Safe and Testable Rule
A domain detector has value only when its output leads to a clear action. The action should be designed with the clinical and technical teams responsible for the model’s use. Depending on the intended setting, a flagged case may be prevented from receiving an automated model recommendation, sent for clinical review, or routed to another approved pathway. The specific action belongs in the implementation protocol; a detector should never leave the response to an informal interpretation of the flag.
List the patient characteristics, categories, measurement contexts, and observation ranges supported by the validation evidence.
Set explicit bounds and, where needed, a joint-support rule before evaluating deployment patients. Record the data and assumptions behind each threshold.
State how a flagged recommendation is withheld or qualified, where the case is routed, and what information is recorded.
Test the rule on representative cases, including known in-domain and deliberately out-of-domain examples, and inspect both missed flags and unnecessary flags.
Diagnostics should include the number and proportion of cases flagged, the types of boundary violations, and whether flags cluster in particular sites, subgroups, or measurement contexts. A very high flag rate may indicate that the deployment population differs from the validation population, or that the rule is too restrictive. A very low rate does not prove that the screen is adequate: the detector may simply fail to recognize a relevant type of extrapolation. Review borderline cases and compare the flagged distribution with the population for which the model is intended.
Re-evaluate the domain rule when the model, data collection, intended population, or practice setting changes. Expanding the domain requires evidence that the model performs acceptably in the newly included region; it is not justified just because many patients are flagged. Conversely, a rule that routinely blocks patients within the intended population may need refinement, but the refinement should be tested before it is used to permit recommendations.
Alternatives, Limitations, and a Common Mistake
| Approach | What it detects well | Main limitation |
|---|---|---|
| Prespecified univariate bounds | Clear violations of documented ranges or levels | May miss unusual combinations of ordinary values |
| Joint-support distance | Covariate combinations far from a reference distribution | Depends on the chosen representation, reference data, and threshold |
| Predictive-uncertainty abstention | Cases where the recommendation is too uncertain under the uncertainty rule | Uncertainty can be low even when the case is outside validated support |
| Clinical review | Context requiring judgment beyond a numerical rule | Needs a reliable workflow and does not itself define the model’s validated domain |
Statistical support is not the same as clinical applicability. A patient can fall within the observed covariate distribution but differ in an important unrecorded way. A domain detector can also be distorted by small samples, inappropriate covariate scaling, unstable covariance estimates, or a reference cohort that does not represent the intended deployment population. No single distance threshold resolves those limitations. Domain rules are safeguards around the evidence, not a substitute for assessing whether that evidence matches the use case.
Common mistake: “The model returned a recommendation with low predictive uncertainty, so the patient must be in domain.” This reasoning confuses two separate questions. Predictive uncertainty describes uncertainty under a model and its assumptions; domain screening asks whether the patient’s characteristics and observations are supported by validation evidence. A model can be confident while extrapolating. Check the domain independently, and apply the prespecified action when it fails.
Check Your Understanding
Answer each question, then compare with the solutions below.
- A model was validated for weights from 40 to 120 kg. A patient weighs 118 kg but has a joint covariate combination flagged by the prespecified support rule. Is the patient necessarily in domain?
- In one sentence, distinguish an out-of-domain flag from high predictive uncertainty.
- For two standardized covariates with correlation 0, a patient has values \(z_1=1\) and \(z_2=2\). Calculate \(D^2\). Would this patient exceed a hypothetical threshold of 5.991?
- Why is replacing an out-of-range observation with the nearest boundary value not a valid way to make the model recommendation supported?
Solutions. (1) No. Passing the marginal weight check does not override the joint-support flag; the specified action for an out-of-domain patient applies. (2) A domain flag concerns support in the validated population or measurement scope, whereas predictive uncertainty concerns uncertainty in the recommendation under the model. (3) With zero correlation, \(\Sigma\) is the identity matrix, so \(D^2=1^2+2^2=5\). Since \(5<5.991\), this patient does not exceed that illustrative threshold. (4) Replacing the observation conceals the actual input and does not add validation evidence for the patient’s true value; it can therefore create an apparently supported output without making the recommendation supported.