Why Alignment Comes Before Comparison
Two models can report predictions in the same units and still answer different questions. One may describe younger participants receiving a tablet under closely monitored dosing; another may describe an older, less adherent population receiving a different formulation. Comparing their overall predictions without accounting for these differences can make a change in population or conditions look like a model discrepancy—or conceal a genuine discrepancy.
Alignment means making the populations and clinical conditions represented by two model components sufficiently comparable for the intended comparison or transfer. It does not mean forcing them to be identical. It means identifying relevant differences, deciding which can be adjusted using evidence, and stating what remains unsupported. The intended target population and conditions should be specified before calculating an adjusted prediction.
The relevant differences depend on the quantity being compared. Demographics and organ function may affect model inputs or modify a drug effect. Adherence and formulation can change the exposure generated by a prescribed regimen. Food, dosing schedule, concomitant treatments, sampling design, and other study conditions can alter what was measured or the setting represented by the model. A difference is important when it can change the prediction, the meaning of the parameter, or the applicability of the evidence.
Define the Context and the Comparison
Start by describing the source and target in terms that connect to the model. “Adults” is often too broad: age distribution, renal function, disease severity, adherence, formulation, and treatment conditions may matter. Record the relevant attributes, their distributions or levels, and how each is represented. Also specify the model output, its units, and the time point or summary being compared. This tutorial concerns contextual comparability; it does not replace the earlier material on harmonizing quantities across models.
A useful representation is a conditional prediction \(m(x,c)\), where \(x\) denotes patient attributes such as age or organ function and \(c\) denotes clinical conditions such as formulation or dosing conditions. A population-level prediction averages this conditional prediction over the population and conditions of interest. If the target has discrete strata \(k\), a standardized prediction is:
Here, \(m_k\) is the prediction for stratum \(k\) under the specified conditions, and \(w_k\) is that stratum’s proportion in the target population, with \(w_k \geq 0\) and \(\sum_k w_k=1\). The result has the same units as \(m_k\). This calculation is meaningful only if the stratum-specific predictions refer to the same output definition and relevant clinical conditions, or have been validly adjusted to them.
The key assumption is that the conditional predictions are applicable to the target individuals within each stratum. In practice, that requires relevant effect modifiers to be represented, comparable outcome and condition definitions, and enough source evidence to support the target strata. If the source has no evidence for an important target subgroup, weighting an unsupported prediction does not solve the problem.
Worked Example: Standardizing for a Different Age Distribution
Hypothetical data. A model predicts the percentage reduction in a disease measure at a specified dose and time point. For ages 20–64, the prediction is 30%; for ages 65 and older, it is 20%. The source population is 40% ages 20–64 and 60% ages 65 and older. The target population is 70% and 30%, respectively. Assume the predictions are valid in both age groups under the same dose, formulation, and study conditions.
The source-population prediction is \(0.40(30\%)+0.60(20\%)=12\%+12\%=24\%\). The target-standardized prediction is:
The target prediction is 27%, three percentage points higher than the source aggregate prediction of 24%. This difference arises from the different age composition under the stated stratum predictions; it does not establish that age itself causes a change. The calculation does not adjust any other difference, and it depends on the stratum predictions being applicable to the target.
Separate Patient Attributes From Clinical Conditions
A population difference and a condition difference should not be collapsed into one adjustment. Patient attributes describe who is represented; clinical conditions describe how treatment is given or how evidence is generated. They can interact, but they are conceptually distinct. For example, renal function is a patient attribute, while formulation and dosing schedule are treatment conditions. Adherence describes medication-taking behavior and may vary across individuals and settings. A prescribed dose alone does not establish the exposure associated with that regimen.
For each difference, ask three questions: Is it represented in the model? Is its relationship to the output supported by evidence? Does the source evidence cover the target values or conditions? If the model has an evidence-supported function for a covariate, an adjustment may be possible. If a factor is absent, poorly measured, or outside the evidence range, the responsible choice may be to qualify the comparison, restrict its scope, or gather more evidence rather than apply an invented correction.
Worked Example: Reconciling Organ Function, Adherence, and Formulation
Hypothetical model. Suppose a simplified exposure index, in arbitrary units, is predicted as \(E=100\sqrt{90/G}\,a f\), where \(G\) is estimated renal function in mL/min, \(a\) is the fraction of prescribed doses taken, and \(f\) is relative formulation availability. The model is assumed valid over the values used here, and the factors are assumed to combine multiplicatively.
The source context has \(G=45\) mL/min, \(a=0.80\), and \(f=0.90\). Its prediction is:
The target context has \(G=90\) mL/min, \(a=0.95\), and \(f=1.00\). Its prediction is:
Under this hypothetical model, the target prediction is 95.0 exposure-index units, compared with 101.8 in the source context. The lower target prediction reflects the specified changes in all three inputs; it cannot be attributed to renal function alone. The adjustment is only defensible if the model’s functional form, input definitions, and range of validity are supported. In particular, a population-average adherence fraction may not adequately represent individual variation or its association with other attributes.
A Justification for Standardizing Predictions
The weighted average follows from partitioning the target population into mutually exclusive strata. Let \(M\) be the model prediction for a randomly selected target individual, and let \(K\) identify that individual’s stratum. The law of total expectation gives:
This identity justifies reweighting conditional predictions to a new composition. It does not justify using a source estimate for \(m_k\) in the target automatically. That substitution requires the conditional prediction to be transportable: after accounting for relevant attributes and conditions, the source estimate must represent the target stratum for the stated model output. If important effect modifiers remain unaccounted for, or the target stratum lies beyond the evidence, the equality cannot be used as a practical transfer claim merely because the algebra is valid.
Worked Example: Checking a Condition-Specific Transfer
Hypothetical data. A model predicts a treatment-related reduction in a clinical measure, in units, at a fixed time point. Under a specified background treatment, it predicts 5 units for mild baseline disease and 9 units for severe baseline disease. The source population is 80% mild and 20% severe; the target population is 30% mild and 70% severe. Assume the same endpoint, time point, formulation, dosing, and background treatment in both contexts, and assume the subgroup predictions are supported for both populations.
The source aggregate is \(0.80(5)+0.20(9)=4+1.8=5.8\) units. Standardizing to the target composition gives:
The target-standardized prediction is 7.8 units, two units above the source aggregate. The subgroup predictions have not changed; the composition has. If the target also used a different background treatment, this calculation would not align that condition, and its 7.8-unit result could not be interpreted as a prediction for the changed treatment setting without further evidence.
Diagnostics, Alternatives, and Limitations
An alignment assessment should expose where a comparison is fragile. Check whether the source and target use compatible definitions for covariates and conditions; whether their measurement timing and categories match; whether all important target strata occur in the source; and whether conclusions change under plausible target weights or alternative assumptions. Extreme weights, very small strata, missing covariates, and predictions outside the supported range are warning signs. A numerical adjustment can look precise while depending on unsupported extrapolation.
The main alternatives serve different purposes:
| Approach | When it can help | Main limitation |
|---|---|---|
| Direct comparison or pooling | Contexts are already sufficiently comparable for the stated output. | Can confound population or condition differences with model differences. |
| Stratified standardization | Relevant strata are defined, supported, and their target weights are known. | Cannot repair unsupported strata or unmeasured effect modifiers. |
| Covariate or condition adjustment | An evidence-supported model function represents the relevant difference. | Results depend on the function, input quality, and applicability range. |
| Restriction to a common context | A defensible comparison is possible in the overlap of source and target. | May narrow the question and exclude important parts of the target population. |
These approaches are not interchangeable. Direct pooling is not a substitute for checking comparability. Standardization addresses composition only when conditional predictions are valid. Covariate adjustment requires an appropriate relationship, not merely a covariate measurement. Restriction can avoid extrapolation but changes the population to which the conclusion applies. Some differences may remain unresolved; report them explicitly instead of implying that alignment has been achieved.
Alignment also does not establish that a model is correct, that an observed difference is causal, or that a transferred parameter is certain. Model misspecification, measurement error, unmeasured modifiers, and uncertainty in target weights can remain. The purpose is narrower: clarify which population and conditions a prediction represents, and whether the evidence supports using it in the context of interest.
Common Mistake
A related mistake is to adjust for several differences at once and describe the result as the effect of one factor. As the exposure example shows, if renal function, adherence, and formulation all change, the resulting difference reflects their combined contribution under the model. Attribution to one factor requires a comparison that holds the others fixed or a suitable decomposition.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- A source population is 60% group A and 40% group B. Predictions are 12 mg/L for group A and 8 mg/L for group B. The target is 25% group A and 75% group B. What is the target-standardized prediction, and what assumption is needed?
- A target differs from the source in formulation, but the model contains no formulation term and the available evidence does not compare formulations. Is it valid to adjust using the formulation ratio alone? Explain.
- A model predicts 4 units in mild disease and 10 units in severe disease. The target is 50% in each group. Calculate the standardized prediction.
- Why can a target subgroup with no source evidence make standardization unreliable even when its target weight is known?
Solutions. (1) \(0.25(12)+0.75(8)=3+6=9\) mg/L. The subgroup predictions must apply to the target groups under compatible conditions, with relevant modifiers represented. (2) No. A numerical formulation adjustment is unsupported without evidence for its relationship to the model output; qualify or restrict the comparison, or obtain relevant evidence. (3) \(0.50(4)+0.50(10)=2+5=7\) units. (4) The weight gives the subgroup’s contribution but not its prediction. Without supported conditional evidence, assigning a value for that subgroup requires unsupported extrapolation.