Why Observed Support Matters
A simulation can generate a covariate profile that was not represented in the data used to develop or evaluate a prediction model. This can happen because a continuous covariate falls below its observed minimum or above its maximum. It can also happen because a combination of covariates is unusual or absent, even though each covariate value separately falls within its observed range. A model may still return a numerical prediction in either case. A computed value, however, is not evidence that the prediction is reliable in that region.
This distinction matters when standardizing predictions to a reference population: a population average depends both on the sampled covariates and on the model’s predictions for those covariates. If some simulated profiles lie beyond the data-supported region, their predictions may depend heavily on assumptions about the model form, covariate relationships, or the target population. The task is not automatically to remove such profiles. It is to identify them, understand why they occur, and make the assumptions and consequences explicit.
Ranges Are Not the Whole Support
For a single continuous covariate, the minimum and maximum provide a simple screening boundary. A simulated value outside that interval is beyond the observed range. But the boundary is only a summary of the sample: observing no values above a particular maximum does not establish that the target population cannot contain larger values. Conversely, a value inside the minimum-to-maximum interval is not necessarily well supported. The sample might contain only a few observations near that value, or the intended population might differ from the data source.
With several covariates, separate ranges can be misleading. Suppose age and body weight each cover a broad interval in the data. A simulated patient whose age and weight both fall inside those intervals may nevertheless have a combination not represented in the joint data. A useful geometric screen for continuous covariates is the convex hull, the smallest convex region containing the observed profiles. A point outside that region is certainly outside the convex hull of the data. But a point inside it is not necessarily well supported: the actual observations may leave gaps, concentrate in a narrow band, or have few profiles near that point. For categorical covariates, a combination of levels can likewise be unobserved even when every level appears in the data.
These distinctions are about available evidence, not a verdict that a model cannot be evaluated. Mechanistic knowledge, external data, or a defensible structural model may support predictions beyond the observed region. The interpretation should then identify that basis rather than present the prediction as directly supported by the data.
Worked Example: A Weight Above the Observed Range
Suppose a hypothetical pharmacometric analysis has observed body weights from 45 to 100 kg. Its clearance model is \(CL=5(WT/70)^{0.75}\) L/h, where \(WT\) is weight in kg. A simulated individual has a weight of 110 kg. Calculate the model prediction and identify what it means.
The simulated weight exceeds the observed maximum, so this is univariate extrapolation beyond the observed range. Substituting the weight into the stated model gives:
The model returns a clearance prediction of approximately 7.018 L/h. That arithmetic establishes what the model predicts; it does not establish that the prediction is empirically validated for a 110 kg individual. Its interpretation depends on whether the allometric relationship remains appropriate above the observed range and whether the simulated weight is plausible for the target population. If the analysis has a defensible external or mechanistic basis for those assumptions, it can report the prediction with that qualification. Without such a basis, it should be described as assumption-dependent extrapolation.
Assessing Joint Covariate Support
A joint support assessment asks whether simulated profiles resemble combinations represented in the data, rather than checking each column independently. The convex hull offers one screen for continuous covariates, but it is not a complete definition of empirical support. For example, an observed sample may contain two distinct clusters with a large empty region between them. A point in that gap can lie inside the convex hull while still having little nearby data support. In mixed data, it is also important to check whether relevant combinations of continuous values and categorical levels occur together.
The model’s structure affects how consequential an unsupported combination may be. An additive model may produce a finite prediction by combining effects estimated over separate ranges. But if the effect of one covariate changes with another, the interaction may be poorly informed where the combination is absent. Nonlinear relationships can also change beyond the observed range in ways that a simple extrapolation cannot reveal. These are not reasons to assume every unsupported prediction is wrong; they are reasons to state which model assumptions determine it.
Worked Example: In-Range Values, Unsupported Combination
In a hypothetical data set, the observed age-weight profiles are \((20,50)\), \((20,90)\), and \((60,50)\), with age in years and weight in kg. Consider a simulated profile with age 60 years and weight 90 kg. Each value falls within its separate observed range, but the combination is outside the triangle formed by the observed profiles. Suppose an already specified clearance model is \(CL=4+0.03A-0.01WT\) L/h.
At the simulated profile, the model gives:
This 4.90 L/h value is a model prediction for a combination not represented in the data. The separate range checks alone would miss the support problem. In particular, these data do not directly show how clearance behaves among older individuals at the higher end of the observed weight range. If the model assumes that age and weight effects combine as specified, its prediction follows from that assumption; it should not be described as directly supported by observed profiles. External evidence or a justified mechanistic model could strengthen the interpretation, but the calculation by itself cannot do so.
Quantifying How Often Sampling Exceeds an Interval
When a univariate covariate is sampled from a specified distribution, the probability of falling outside an observed interval can quantify how often a simulation is expected to enter an extrapolation region. Let the observed interval be \([L,U]\), and let the sampled covariate have cumulative distribution function \(F\). For a continuous distribution, the probability of a value below \(L\) or above \(U\) is:
If \(N\) covariates are sampled independently from that same distribution, the number \(K\) outside the interval is a binomial random variable with mean \(Nq\) and variance \(Nq(1-q)\). This describes the expected frequency of range exceedance under the chosen sampler; it does not determine whether those values are scientifically plausible or whether their predictions are reliable. In multiple dimensions, a single marginal tail probability is not enough to describe unsupported combinations.
Worked Example: Expected Number of Ages Outside the Observed Interval
Suppose a hypothetical development data set includes ages from 18 to 75 years. A proposed target-population sampler draws age from \(X\sim N(60,12^2)\), with age measured in years. Calculate the probability that a draw falls outside the observed interval, then find the expected number outside it in 1,000 independent draws.
Standardizing the two endpoints gives \(z_L=(18-60)/12=-3.50\) and \(z_U=(75-60)/12=1.25\). Thus:
About 10.59% of draws are expected to fall outside the observed age interval under this sampler. For \(N=1{,}000\), the expected number is:
This is an expectation, not a guarantee that exactly 106 simulated ages will be outside the interval. Under the stated independent-sampling assumption, the count has standard deviation \(\sqrt{Nq(1-q)}=\sqrt{1{,}000(0.10588)(0.89412)}\approx 9.73\) draws. The calculation also relies on the proposed normal distribution being an appropriate representation of target ages. It does not, by itself, justify either that distribution or predictions at the ages it generates beyond the observed interval.
Choosing What to Do With Unsupported Profiles
There is no universal correction for extrapolation. The appropriate response depends on the scientific target and on why the profiles are unsupported. Different actions can change the estimand, the sampled population, or the assumptions behind predictions. The following approaches answer different questions:
| Approach | What it does | Main implication |
|---|---|---|
| Empirical resampling | Resamples observed covariate records. | Stays within observed profiles but may fail to represent target groups absent from the data. |
| Truncation or rejection | Removes draws outside a chosen region, or redraws them. | Changes the sampled population unless that restriction matches the intended target. |
| Clipping or winsorization | Replaces values beyond a boundary with a boundary value. | Creates artificial concentration at the boundary and changes the covariate distribution. |
| Explicit extrapolation | Retains profiles and evaluates the model beyond observed support. | Preserves the proposed target sampling scheme, but requires defensible model assumptions in unsupported regions. |
| Restricting the target population | Defines the target to exclude profiles outside a justified region. | Answers a narrower population question; it does not describe the original target. |
Empirical resampling is not automatically safer: if the target population includes covariate patterns not present in the resampling data, resampling cannot represent them. Conversely, retaining every parametric draw is not automatically better: a mathematically convenient sampling distribution may place substantial mass in implausible regions. Truncation and clipping are not neutral housekeeping steps. They alter the covariate distribution and can shift the population prediction. A principled choice states the target and explains how the chosen strategy relates to it.
For explicit extrapolation, the analysis should name the extrapolated covariates or combinations, state the relevant model assumptions, and show how much of the simulated population is affected. Where scientifically reasonable, sensitivity analyses can compare predictions under alternative defensible model forms or target-distribution assumptions. Such comparisons reveal dependence on assumptions; they do not turn unsupported predictions into observations.
Diagnostics and Common Pitfalls
A practical support screen starts with the simulated covariates and the data used for model development. For each continuous covariate, count or summarize draws below the observed minimum and above the observed maximum. For joint profiles, examine whether simulated combinations fall beyond the observed multivariate region and whether categorical combinations are represented. Report the number or proportion flagged, and inspect the predictions associated with those profiles. A single extreme prediction may matter even if the proportion flagged is small, particularly when population summaries are sensitive to that prediction.
These checks identify potential support problems; they do not establish that the target distribution is wrong or that the model prediction is invalid. Follow-up requires scientific judgment about the data source, intended population, and model. In particular, distinguish an observed-data boundary from a biological or operational boundary. An observed maximum is a property of the sample. A true impossibility, such as a covariate combination prohibited by the study or physiology, requires separate justification.
- Wrong reasoning: “Every simulated value is between the observed minimum and maximum, so the profiles are supported.” Correction: Check joint combinations and density of representation as well as marginal ranges.
- Wrong reasoning: “The model returned a number, so the result is data-supported.” Correction: A model can produce predictions beyond observed support; their validity depends on assumptions not established by the arithmetic.
- Wrong reasoning: “Clipping extreme draws only prevents numerical problems.” Correction: Clipping changes the sampled covariate distribution and can change the target-population prediction.
- Wrong reasoning: “Nothing was observed above the maximum, so the target population has no values above it.” Correction: The sample maximum does not define the population’s true upper boundary.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- A covariate has observed values from 10 to 50 units. What can and cannot be concluded about a target draw of 52 units?
- Age and weight are each within their observed marginal ranges, but their combination is outside the convex hull of observed profiles. Is this extrapolation? Explain.
- A continuous target sampler has probability \(q=0.04\) of generating a value outside an observed interval. For 500 independent draws, what is the expected number outside?
- Why can clipping every value above an observed maximum change the scientific interpretation of a population prediction?
Solutions. (1) The draw is beyond the observed range, so a prediction at 52 units is extrapolation relative to those data. The observed maximum does not establish that the target population cannot contain 52-unit values, nor does it validate a model prediction there. (2) Yes. The profile is outside the convex hull, despite both marginal values being in range, so the joint combination is not represented by the observed profiles. The prediction depends on assumptions about behavior in that region. (3) The expected number is \(Nq=500(0.04)=20\) draws. This is an expectation, not a guarantee of exactly 20. (4) Clipping replaces distinct high values with the same boundary value, changing the target covariate distribution and potentially its predicted population average. The resulting estimate answers a modified, boundary-capped population question unless that modification is scientifically intended.