Changing Outcome Assessment Schedules

Why Assessment Timing Matters

A trial can retain the same treatments, participants, and measurement instrument yet change what its outcome means by changing when it is measured or how often it is assessed. A score at week 12, an average score across scheduled visits, and the proportion meeting a response criterion at week 12 are distinct summaries. A schedule revision preserves an estimand only when the target remains the same: the same outcome variable, population, treatment conditions, and time-based summary.

This distinction is especially important in an adaptive trial. An interim decision might lead to a revised visit calendar for later participants. If the revised calendar records the same outcome at the target time, the target may be retained. If it changes the time point or the weights assigned to observations, it may change the target, even if the protocol continues to call the outcome by the same name.

Definition: Let \(Y^a(t)\) denote a participant’s continuous outcome at time \(t\) under treatment condition \(a\), measured on a specified scale. In a fixed target population, the treatment effect at time \(t\) is \(\Delta(t)=E[Y^1(t)]-E[Y^0(t)]\). An assessment schedule is the set of times at which outcomes are planned to be collected. A schedule preserves this fixed-time estimand only if the target remains the contrast at the same \(t\), on the same scale, in the same population and under the same treatment conditions.

For a response outcome, define \(R^a(t)=1\) if the participant meets a prespecified response criterion at time \(t\), and \(R^a(t)=0\) otherwise. The response risk difference at time \(t\) is \(P(R^1(t)=1)-P(R^0(t)=1)\). Changing \(t\) can change which participants count as responders, so it can change the estimand even when the response rule itself is unchanged.

Fixed-Time Outcomes and Schedule Revisions

Suppose the scientific question concerns mean symptom severity at week 12. The target is the contrast in mean severity at week 12, not the contrast at whichever visit happens to be nearest for each participant. A protocol that moves the nominal assessment from week 12 to week 10 therefore changes the time point being targeted. Calling both measurements “the endpoint” does not make them the same outcome summary.

A shifted schedule may still allow estimation of the original fixed-time contrast, but that requires a method and assumptions connecting the collected data to the target time. For example, measurements at weeks 10 and 14 can be used to estimate a week-12 mean by interpolation under an appropriate model for change over time. Interpolation is not a design-based guarantee: if outcomes change nonlinearly between visits, the interpolated mean may be inaccurate. Extrapolating beyond the observed time range usually requires stronger assumptions still.

Worked Example: Estimating a Fixed-Time Mean Between Visits

In a hypothetical trial, the target is the mean outcome at week 12, where higher scores indicate more severe symptoms. A revised schedule collects mean scores at weeks 10 and 14. In the treatment arm, the observed means are 8 and 12 points; in the control arm, they are 5 and 7 points. Assume, for this calculation, that each arm’s mean changes linearly from week 10 to week 14.

For a time \(t\) between \(t_0\) and \(t_1\), linear interpolation estimates the mean at \(t\) as the mean at \(t_0\) plus the fraction of the interval elapsed times the change across the interval:

\widehat{\mu}(t)=\bar{Y}(t_0)+\frac{t-t_0}{t_1-t_0}\left[\bar{Y}(t_1)-\bar{Y}(t_0)\right].

At week 12, the elapsed fraction is \((12-10)/(14-10)=2/4=0.5\). The treatment-arm estimate is \(8+0.5(12-8)=8+2=10\) points. The control-arm estimate is \(5+0.5(7-5)=5+1=6\) points. The estimated week-12 treatment contrast is \(10-6=4\) points, with units of symptom-score points.

This calculation estimates the original week-12 contrast only if the linear interpolation assumption is reasonable for the arm means and the data are representative of the target population. The two scheduled visits alone do not establish that assumption. If the trajectory is curved or changes abruptly between weeks 10 and 14, the estimate can miss the week-12 mean.

When the target is specified at a fixed time, adding measurements at other times does not by itself change the estimand. Those extra measurements may help describe the trajectory or support estimation, but they must not silently replace the fixed-time contrast with a different summary. Conversely, collecting fewer visits may leave the estimand intact on paper while making it harder to estimate reliably.

Visit-Averaged Summaries Depend on Visit Weights

Some outcomes are defined as a summary across multiple times rather than at one time. Such a summary must specify both the time interval and how outcomes over that interval contribute. An equal-weight average across observed visits assigns the same weight to each visit, regardless of how far apart the visits are. Adding a visit then changes the weights and can change the target summary.

More generally, suppose a treatment effect is summarized over prespecified times \(t_1,\ldots,t_J\), with weights \(w_1,\ldots,w_J\) that sum to 1. The target is \(\theta=\sum_{j=1}^{J}w_j\Delta(t_j)\). The times and weights are part of the definition, not merely a data-collection convenience.

Justification: If a revised schedule adds a time \(t_{J+1}\) and the new summary gives equal weight to all \(J+1\) visits, then \(\theta_{\text{new}}=[J\theta_{\text{old}}+\Delta(t_{J+1})]/(J+1)\), where \(\theta_{\text{old}}\) is the equal-weight average over the original \(J\) visits. Thus, \(\theta_{\text{new}}-\theta_{\text{old}}=[\Delta(t_{J+1})-\theta_{\text{old}}]/(J+1)\). The two summaries are equal only if the added-time effect equals the original average, or if the summary is explicitly defined to keep the original weights and target.

Worked Example: An Added Visit Changes an Equal-Weight Summary

Consider a hypothetical repeated continuous outcome measured at weeks 4, 8, and 12. Suppose the treatment effects at those times are 2, 6, and 10 points, respectively. The protocol defines its summary as the equal-weight average across the three scheduled visits.

The original summary is \((2+6+10)/3=18/3=6\) points. A schedule revision adds week 16, where the treatment effect is 14 points, and the summary is redefined as the equal-weight average across all four visits. The revised summary is \((2+6+10+14)/4=32/4=8\) points.

The increase from 6 to 8 points is not evidence that the treatment’s effect at weeks 4, 8, or 12 changed. It occurs because the revised summary includes a later time with a larger effect and gives that time weight. The schedule and the definition of the summary together have changed the estimand. If the intended target were the original average over weeks 4, 8, and 12, the added week-16 measurement could be collected but would not be included with equal weight in that target.

An alternative is to define a time-weighted summary over a fixed interval, with weights reflecting duration or a prespecified integration rule. That can be more appropriate when the scientific question concerns average burden over time, but it is not automatically equivalent to a visit average. Changing the interval, the weights, or the rule for handling gaps can change the target. The protocol should define these features before the schedule is adapted.

Response at a Specified Time

For a binary response outcome, the response criterion and assessment time are both essential. “Response at week 20” and “response at week 24” are different variables if a participant can improve, relapse, or otherwise change status between those times. The same threshold applied at both visits does not make the response proportions interchangeable.

Worked Example: Response Rates at Two Assessment Times

In a hypothetical randomized trial, 100 participants per arm have response status recorded at both weeks 20 and 24. At week 20, 60 treatment-arm participants and 45 control-arm participants meet the response criterion. At week 24, the corresponding counts are 55 and 35. Assume for this example that response is ascertained for every participant at both times and that the criterion is unchanged.

At week 20, the estimated response proportions are \(60/100=0.60\) and \(45/100=0.45\). The estimated risk difference is \(0.60-0.45=0.15\), or 15 percentage points, favoring treatment.

At week 24, the estimated proportions are \(55/100=0.55\) and \(35/100=0.35\). The risk difference is \(0.55-0.35=0.20\), or 20 percentage points, favoring treatment. These calculations are consistent with the stated counts and denominators.

The two estimates answer different questions: the treatment-control response contrast at week 20 and the contrast at week 24. The change from 15 to 20 percentage points does not show that one estimate is a correction of the other. A protocol choosing one assessment time over the other changes the response-at-a-time-point estimand unless it specifies a method that continues to target the original time.

Auditing an Adapted Schedule

Before implementing a schedule change, compare the original and revised definitions of the outcome rather than comparing visit calendars alone. State the target time or interval, the measurement scale or response rule, and any weights used to combine measurements. Then check whether the revised data collection can estimate that same target without relying on unsupported assumptions.

Write the target precisely.
For example, specify a mean score contrast at week 12, or a risk difference for response at week 20. Name the scale, population, and treatment conditions.

Compare the scheduled times.
Identify visits that moved, were added, or were removed. Determine whether the target time is still measured directly or must be estimated from other times.

Compare the summary rule.
Check whether visit weights, the averaging interval, or the response time changed. If any did, assess whether the target itself changed.

Check the measurement process.
Assess whether the instrument and response criterion remain comparable and whether assessment frequency could alter care or participant behavior. If assessment affects treatment decisions, the treatment condition may also differ.

State estimation assumptions and limitations.
For an unobserved target time, state the trajectory or interpolation model and assess whether it is plausible. Do not treat a model-based estimate as a direct measurement.

Useful diagnostics include plotting arm-specific outcome means against actual assessment time, checking the distribution of actual visit times around the target, and comparing the frequency of missing assessments by schedule and trial stage. Sparse data near the target, visibly nonlinear trajectories, or different patterns of missed visits are warnings that a simple interpolation or pooled summary may not be reliable. These checks diagnose weaknesses; they do not prove that an assumed trajectory is correct.

The main alternatives answer different questions. A directly measured fixed-time contrast is clearest when that time is observed for the target population. A model-based estimate at a fixed time can use data from nearby visits, but depends on the model and its assumptions. A prespecified average over times summarizes a trajectory, but depends on its interval and weights. A response risk at one time is easy to interpret as status at that time, but does not summarize response over the whole course of follow-up. These approaches should not be substituted for one another without an explicit change in the scientific target.

Common Mistakes and Limitations

“The same endpoint name means the same estimand.” It does not. A symptom score at week 10 and the same score at week 12 are measurements at different times.
“More frequent measurement only adds information.” It can add information when the target stays fixed, but an equal-weight visit average changes when visits are added or removed.
“A nearby visit is the target visit.” It is not. Substituting a nearby observation or interpolating requires an explicit estimation rule and assumptions about change over time.
“The response criterion is unchanged, so the response outcome is unchanged.” The time point also defines the response variable; response at week 20 can differ from response at week 24.

Even a carefully specified schedule cannot eliminate every limitation. Interpolation may be sensitive to nonlinear change; fewer assessments may leave the target weakly supported by data; and frequent assessments may burden participants or influence care. If assessment frequency itself affects treatment decisions or participant behavior, it is not merely a change in measurement logistics. The implications depend on how the trial is conducted, and should be addressed before treating observations from different schedules as directly comparable.

Key takeaway: A schedule change preserves an outcome estimand only when it preserves the outcome variable and its time-based summary. Fixed-time contrasts, weighted averages across visits, and response proportions at a specified time each require their own explicit definition.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions below.

A trial targets mean lung function at week 16, but a revised schedule measures it at week 14. Is the original target directly measured? What additional step would be needed to estimate it?
A protocol averages treatment effects equally at weeks 6 and 12. A revision adds week 18 and averages equally across all three times. Does the summary necessarily preserve the original target? Explain.
In a hypothetical trial, response proportions at week 10 are 0.52 for treatment and 0.40 for control. What is the response risk difference at week 10, in percentage points?
Name two diagnostics that could flag problems when outcomes from different assessment schedules are combined.

Solutions. (1) No. Week 14 is not week 16; estimating the week-16 mean would require data or a model that supports estimation at week 16, together with stated assumptions. (2) No. The added time receives weight and may have a different treatment effect, so the equal-weight target can change. It is preserved only under special conditions, such as the week-18 effect equaling the original average, or if the original weights remain fixed. (3) The risk difference is \(0.52-0.40=0.12\), or 12 percentage points favoring treatment. (4) Examples include plotting mean outcomes against actual assessment time and checking the distribution of visit times near the target; comparing missing-assessment patterns across schedules is another useful diagnostic.