Why a Measurement Change Matters
At a seamless phase transition, the treatment, population, or randomization ratio may change, but so can the way an outcome is measured. A new questionnaire, laboratory assay, or assessment schedule may seem like a practical update. Statistically, however, the key question is whether the stage-specific results still refer to the same treatment effect on the same clinical estimand. A change in recorded numbers does not by itself imply a change in the treatment effect; a change in what is measured, or when it is measured, may do so.
A valid stagewise combination test requires stagewise evidence that is valid for the prespecified hypothesis. If the measurement change alters the estimand, combining stagewise p-values can produce a precisely calculated answer to a different question than the trial intended. The decision to calibrate, analyze stages separately, or change the estimand should therefore be made before interpreting confirmatory evidence.
First Decide Whether the Estimand Is Unchanged
An instrument change can alter only the numerical scale, or it can change the construct being measured. For example, replacing a symptom questionnaire with a validated revised form may preserve the intended symptom construct but change score units. In contrast, switching from a symptom score to a biomarker does not automatically preserve the same clinical outcome. A mathematical correlation between two measurements is not sufficient to establish that their treatment effects answer the same clinical question.
An assay change likewise requires more than a high correlation. A new assay may have a different offset, scale, precision, or sensitivity near clinically important concentrations. The bridge should cover the range expected in the trial and include specimens representative of both treatment groups and relevant clinical conditions. If the relationship is nonlinear or varies across that range, a single linear conversion may not be adequate.
An assessment-schedule change is especially consequential. A measurement at week 16 and a measurement at week 12 are outcomes at different times. If the treatment effect changes over time, these are different estimands even when the same instrument is used. A longitudinal model might estimate a common target-time effect, but doing so requires appropriate data and assumptions about the trajectory; it is not achieved merely by rescaling one stage's result.
Before choosing an analysis, specify the outcome construct, target time, population, treatment contrast, and scale. Then ask whether the bridge supports translating the new measurement to that target without treatment-dependent or clinically important distortion.
Calibration and Combination: Conditions and Justification
Suppose the new measurement \(Y_{\text{new}}\) and old measurement \(Y_{\text{old}}\) are related by a prespecified linear calibration, \(Y_{\text{new}}=a+bY_{\text{old}}+\varepsilon\), with \(b>0\). If the residual error has mean zero and the same calibration applies to both treatment groups, a contrast in the new measurement can be translated to the old scale by dividing by \(b\). The additive offset \(a\) cancels from a treatment contrast. This translation is defensible only if the relation is sufficiently stable in the trial population and across the outcome range, and if calibration uncertainty is either negligible or included in the standard error.
For a stagewise test, let \(Z_1\) and \(Z_2\) be valid standard-normal test statistics in the direction specified by the alternative. One common inverse-normal combination uses prespecified weights \(w_1,w_2\) satisfying \(w_1^2+w_2^2=1\). For equal weights, each is \(1/\sqrt{2}\). The following justification explains why stagewise validity and independence matter.
Under the null hypothesis, if \(Z_1\) and \(Z_2\) are independent standard normal variables, the weighted sum has mean zero and variance \(w_1^2+w_2^2=1\). Therefore \(Z_{\text{comb}}\sim N(0,1)\), and its upper-tail p-value is \(P(Z\ge Z_{\text{comb}})\). Independence commonly follows when stagewise evidence is based on distinct participants and the analysis respects the design; if independence does not hold, the variance and combination procedure must reflect the dependence. Valid combination does not repair an invalid stagewise statistic or a change of estimand.
Worked Example: Converting a New Instrument to the Original Scale
Hypothetical data. Stage 1 uses an established symptom score. In a bridge sample, the new instrument is treated as having the fixed relation \(Y_{\text{new}}=10+1.2Y_{\text{old}}\). The prespecified target is the treatment contrast in the old-score units, higher scores indicating greater symptom burden. In Stage 1 the estimated treatment benefit is 3.0 old-score points, with \(SE=1.5\). In Stage 2 the estimated benefit is 4.7904 new-score points, with \(SE=2.4\). Assume the calibration relation is appropriate for both randomized groups and that its uncertainty is negligible for this illustration.
The Stage 2 contrast on the old scale is
The standard error also divides by the positive scale factor:
Thus \(Z_1=3.0/1.5=2.000\) and \(Z_2=3.992/2.0=1.996\). Suppose the design specified equal inverse-normal weights and the stagewise statistics are independent. Then
The one-sided standard-normal p-value is approximately \(0.00236\). Under the stated assumptions, this is evidence against the null in the prespecified direction. It is not a probability that the null is true. If the calibration slope had been estimated with material uncertainty, treating 1.2 as fixed would understate uncertainty; the analysis would need to propagate that uncertainty or use an appropriate design-based calibration procedure.
Check the Bridge, Not Just the Conversion Formula
A bridge study should be designed to detect the kinds of disagreement that matter for the trial. Paired measurements on the same specimens or participants allow direct assessment of differences, but the bridge sample must represent the trial's measurement range and relevant subgroups. Examine the relationship graphically and quantitatively for offset, scale, nonlinearity, heteroscedasticity, and subgroup differences. A high correlation can coexist with a clinically important systematic bias.
For assay replacement, comparability is often framed in terms of an acceptable difference on the clinical scale. If \(D=Y_{\text{new}}-Y_{\text{old}}\), an equivalence assessment asks whether an interval for the mean difference lies entirely within prespecified margins \((-\delta,\delta)\). Failure to establish equivalence does not prove the assays differ; it means the available bridge has not shown that any difference is small enough for the proposed use. The margin must be clinically justified rather than selected after seeing the data.
Worked Example: Assessing Paired Assay Comparability
Hypothetical data. Twenty stored specimens are measured using both the old and new assay. Define \(D\) as new minus old, in assay units. The paired mean difference is \(\bar{D}=0.030\), and the sample standard deviation of the paired differences is \(s_D=0.200\). Suppose differences are reasonably modeled as independent, approximately normal paired differences, and the prespecified acceptable mean-bias margins are \(-0.10\) and \(0.10\) units.
The standard error is \(s_D/\sqrt{n}=0.200/\sqrt{20}=0.04472\). With \(df=n-1=19\), the 95% t interval uses \(t_{0.975,19}=2.093\):
The interval is approximately \((-0.0636,\ 0.1236)\) assay units. Its upper endpoint exceeds the acceptable margin of \(0.10\), so this bridge does not establish that the mean bias lies within the specified limits. The result does not demonstrate that the assays are inequivalent; it indicates that this sample, with this variability, is insufficient to support the proposed equivalence claim. The trial should not assume interchangeability solely from the small observed mean difference.
When Separate Stage Analyses Are Needed
Calibration is not always possible or appropriate. If the new measurement represents a different construct, if the bridge is weak in the clinically relevant range, or if an assessment-time change alters the target outcome, report stage-specific results for their respective estimands. A combined p-value cannot make distinct clinical questions identical. If the protocol instead defines a new common estimand, justify how each stage contributes valid evidence about it and explain any required model assumptions.
Worked Example: A Change in Assessment Schedule
Hypothetical data. Stage 1 measures a continuous clinical score at week 12 and estimates a treatment benefit of 1.8 points with \(SE=0.8\). Stage 2 measures the same score at week 16 and estimates a benefit of 2.6 points with \(SE=0.9\). Assume higher scores indicate improvement. The two treatment contrasts are measured in the same units, but the stage-specific target times differ.
The stage-specific standardized estimates are \(1.8/0.8=2.25\) at week 12 and \(2.6/0.9=2.89\) at week 16. These calculations do not show that the effects differ, nor do the common units establish that the effects are estimates of the same time-specific parameter. If treatment effects evolve between weeks 12 and 16, the contrasts answer different questions. Combining these statistics as if they measured one fixed-time effect would therefore require an additional, prespecified estimand and a justified model linking the two times. Without that basis, present the week-12 and week-16 results separately and describe their uncertainty in context.
Choosing an Analysis and Diagnosing Failure
The main options have different requirements. Calibration is useful when the construct and target time are unchanged and a reliable bridge maps the scales. A model that includes measurement method may be appropriate when both measurements are observed and the method effect can be estimated under defensible assumptions. Separate stage analyses are preferable when comparability is not established or the estimands differ. The choice is not between a more powerful and a less powerful test; it is between analyses that answer the intended question and analyses that may not.
| Situation at transition | Defensible approach | Key requirement |
|---|---|---|
| Same construct and time; stable scale change | Calibrate to a common scale, then use valid stagewise evidence | Representative bridge and calibration uncertainty addressed |
| Same assay construct but uncertain bias or precision | Validate further or model method differences | Bias and precision acceptable over the relevant range |
| Different construct or target assessment time | Separate stage results, or define and justify a new common estimand | Do not equate outcomes merely because both are numerical |
Warning signs include a calibration slope that changes across the outcome range, differences by treatment group or clinical subgroup, systematic residual patterns, and bridge observations concentrated in a narrow range unlike the trial data. A calibration model should also be checked for influential observations and adequate precision. If calibration is estimated from a small sample, its uncertainty may be substantial even when the fitted line looks plausible.
A frequent mistake is to argue: “Both stages report points on a numeric scale, so we can pool the observations and run the planned test.” This ignores possible differences in construct, units, time, and calibration error. The correction is to establish commensurability first, transform effects and their uncertainty where justified, and use the stagewise testing method with its design assumptions. If that cannot be done, analyze and interpret the stages separately rather than treating an unverified conversion as fact.
Check Your Understanding
Answer each question, then compare with the brief solutions below.
- A new instrument is related to the old one by \(Y_{\text{new}}=5+2Y_{\text{old}}\). A Stage 2 treatment contrast is 6 new-scale units with standard error 3. What are the converted contrast and standard error on the old scale?
- Why does a high correlation between two assays not, by itself, establish interchangeability?
- A bridge interval for mean assay bias extends beyond one prespecified equivalence margin. What can and cannot be concluded?
- Stage 1 measures an outcome at month 3 and Stage 2 at month 6. Name one reason their treatment contrasts may not estimate the same parameter.
Solutions. (1) Divide both by the slope: the contrast is \(6/2=3\) old-scale units and its standard error is \(3/2=1.5\) old-scale units; the offset cancels from a contrast. (2) Correlation can be high despite a systematic offset, scale difference, nonlinear bias, or subgroup-specific disagreement. (3) Equivalence has not been established under the chosen margins; this does not prove inequivalence. (4) The treatment effect may change over time, so month-3 and month-6 contrasts refer to different target times.