Why Assessment Time Matters
A dose-response conclusion is always tied to when the outcome was assessed. A dose that produces a large improvement after four weeks may have a smaller effect after twelve weeks; another dose may show a modest early effect that is larger later. A conclusion based on only one assessment time can therefore misrepresent the evidence relevant to the intended treatment duration.
This tutorial focuses on comparing treatment effects at prespecified times, especially an early and a later assessment, and using those comparisons in dose justification. A time-specific effect describes the contrast at that assessment. It does not, by itself, describe what happened between assessments or establish that an effect will continue beyond the observed period.
For an endpoint where lower values are better, the outcome can be re-expressed or the contrast reversed so that positive values still indicate benefit. The orientation must be stated and used consistently. If the endpoint is measured as change from baseline, the treatment effect is the difference between treatment groups in that change at the specified time, not the change within the active group alone.
The comparison assumes that the groups being contrasted are comparable, as expected under proper randomization, that the endpoint and its measurement are consistent across groups and times, and that the analysis accounts for repeated observations from the same participants. The assessment times, endpoint, and handling of events such as treatment discontinuation should be specified in the estimand and analysis plan. Otherwise, a change in the population or outcome being analyzed over time can be mistaken for a change in treatment effect.
Estimating Effects at Prespecified Times
A repeated-measures model can estimate the mean outcome for each treatment group at each planned assessment, while representing the correlation among observations from the same participant. For time-specific dose justification, time can be treated as a categorical factor. This allows the estimated effects at the scheduled visits to differ without requiring a particular curve between visits. The model should include the treatment, assessment time, and treatment-by-time terms needed to estimate the planned contrasts, along with appropriate baseline adjustment where applicable.
The key quantities are the estimated treatment-placebo contrasts at each assessment, \(\hat{\theta}_d(t_E)\) and \(\hat{\theta}_d(t_L)\), and their difference, \(\hat{\kappa}_d\). The repeated-measures model also supplies the standard errors and covariance needed to quantify uncertainty in these contrasts. The uncertainty of the difference cannot generally be calculated as though the two estimates came from unrelated groups: many of the same participants contribute outcomes at both times.
Worked Example: Calculating Effects at Two Assessment Times
Suppose a hypothetical randomized trial measures improvement on a scale where higher values indicate greater benefit. The table gives mean improvement from baseline in points at Weeks 4 and 12.
| Group | Week 4 mean improvement | Week 12 mean improvement |
|---|---|---|
| Placebo | 1.0 points | 2.0 points |
| 50 mg | 4.0 points | 5.0 points |
| 100 mg | 7.0 points | 8.0 points |
At Week 4, the estimated treatment effects versus placebo are \(4.0-1.0=3.0\) points for 50 mg and \(7.0-1.0=6.0\) points for 100 mg. At Week 12, they are \(5.0-2.0=3.0\) points and \(8.0-2.0=6.0\) points, respectively.
The mean improvement within each group increases by one point from Week 4 to Week 12. However, the active-placebo contrasts are unchanged: \(\hat{\kappa}_{50}=3.0-3.0=0\) points and \(\hat{\kappa}_{100}=6.0-6.0=0\) points. In this hypothetical example, the increasing improvement over time is also seen on placebo, so it does not represent an increase in the estimated treatment-placebo effect. The estimates alone do not establish whether the effects differ statistically or are clinically meaningful; uncertainty and a clinical benchmark would also be needed.
Comparing Early and Later Effects With Uncertainty
To assess whether the estimated treatment effect differs between two assessments, the relevant contrast is \(\hat{\kappa}_d=\hat{\theta}_d(t_L)-\hat{\theta}_d(t_E)\). If the estimates have standard errors \(SE_L\) and \(SE_E\), and covariance \(\operatorname{Cov}(\hat{\theta}_d(t_L),\hat{\theta}_d(t_E))\), then the variance of their difference follows from the variance rule for a difference:
The subtraction of twice the covariance matters. If the two estimated effects are positively correlated, the difference is typically estimated more precisely than it would be under an independence assumption. The covariance should come from the fitted model or an appropriate analysis of the repeated data, not be guessed after seeing the results.
When the estimate is approximately normally distributed, an approximate 95% confidence interval is the estimated contrast plus or minus 1.96 standard errors. This interval quantifies uncertainty in the early-to-later difference under the model; it is not a probability statement about the fixed true contrast. If several doses or several time contrasts are examined as formal claims, the analysis should address multiplicity or clearly identify which comparisons are primary and which are exploratory.
Worked Example: Quantifying a Change in Treatment Effect
In a hypothetical repeated-measures analysis, the estimated active-placebo benefit is 5.0 points at Week 4, with \(SE_E=0.70\) points, and 3.0 points at Week 12, with \(SE_L=0.80\) points. The model estimates a correlation of 0.60 between these two treatment-effect estimates. Assume the estimates are approximately jointly normal and that these model-based standard errors and covariance are appropriate.
The estimated change from early to later is
The covariance is \(0.60(0.70)(0.80)=0.336\) points squared. Therefore,
The approximate 95% confidence interval is
Under the stated model and approximation, the estimated Week 12 benefit is 2.0 points smaller than the Week 4 benefit, and the interval for that difference excludes zero. This supports a decrease between these two assessed times on the modeled scale. It does not show that the benefit decreased continuously, identify when any decrease occurred, or predict the effect beyond Week 12.
Using Time-Specific Evidence in Dose Justification
A dose should be justified for the treatment duration and assessment that matter to the intended use. The decision may depend on whether a clinically meaningful benefit is estimated at the later assessment, whether the dose ranking changes with time, and whether the additional benefit of a higher dose at that time is worth its added burden or risk. The clinical benchmark and assessment time should be chosen before interpreting the estimates; a favorable result at an unplanned time is weaker support for a decision than a result aligned with the planned treatment use.
Worked Example: Applying a Clinical Benefit Threshold at Both Times
Consider a hypothetical dose-ranging study with a prespecified clinical benefit threshold of 4.0 points on an improvement scale. The estimated treatment-placebo benefits are 4.2 and 6.0 points for 50 mg and 100 mg, respectively, at Week 4; at Week 12 they are 4.1 and 4.3 points.
At Week 4, both point estimates exceed the threshold, and the higher dose has an estimated incremental benefit of \(6.0-4.2=1.8\) points relative to 50 mg. At Week 12, both point estimates still exceed the threshold, but the estimated incremental benefit of 100 mg over 50 mg is only \(4.3-4.1=0.2\) points.
If the intended assessment is Week 12, the Week 4 difference of 1.8 points should not be presented as the expected added benefit at Week 12. The later estimates suggest that 50 mg may already meet the point-estimate threshold and that the higher dose adds little estimated benefit at that time. This is not a final dose recommendation: confidence intervals, safety, adherence, and the prespecified decision criteria would also be needed. In particular, point estimates alone do not establish that either dose truly exceeds the threshold.
Several analysis choices can be useful, but they answer different questions. A model with categorical assessment time estimates effects at the planned visits with relatively few assumptions about the shape of change between them. A model with continuous time can estimate a smooth trajectory or effects at unobserved times, but requires a defensible functional form; a misspecified curve can distort the estimated time-specific contrasts. Separate analyses at each visit may be easy to interpret, but can ignore within-participant correlation and complicate multiplicity control. A time-averaged summary can be useful when the decision truly concerns average benefit over a period, but may conceal an important early-to-later difference. It should not replace a planned assessment-specific effect when the decision is about that specific time.
Assumptions, Diagnostics, and Failure Modes
The interpretation depends on more than the choice of model. Check whether the same endpoint definition and scale are used at each assessment; whether the visit windows are sufficiently consistent; and whether the analysis population and treatment-policy strategy remain meaningful over time. For example, if participants who discontinue are systematically less likely to provide later measurements, an analysis of observed cases may compare a selected subset at Week 12 with a broader group at Week 4. The assumptions used to handle missing outcomes must therefore be stated and examined in sensitivity analyses where appropriate.
Useful diagnostics include plots of group means and their uncertainty by assessment time, the numbers of participants contributing data at each visit, and observed patterns of missingness and discontinuation. Check that the fitted model can represent the observed time pattern and that the estimated covariance structure is plausible. Investigate whether conclusions change materially under reasonable alternative covariance or missing-data assumptions. A model that fits the average trend but misses a clear dose-by-time pattern may produce misleading contrasts.
- Using within-group change as the drug effect. Improvement from baseline in the active group does not isolate drug benefit. Compare against placebo at the same assessment time.
- Comparing significance labels instead of effects. A significant estimate at Week 4 and a nonsignificant estimate at Week 12 do not, by themselves, show that the effects differ. Estimate and assess the early-to-later contrast directly.
- Ignoring repeated-measures covariance. Treating the time-specific estimates as independent can give the wrong standard error for their difference. Use the covariance from the repeated-measures analysis.
- Choosing a favorable visit after looking at the data. Selective emphasis on the most favorable assessment can overstate evidence. Prespecify the primary time contrasts and account for additional formal comparisons.
- Claiming more than the observations support. Two assessments do not reveal the complete course of response between visits or establish what happens after the last assessment. Keep the conclusion tied to the times and population analyzed.
Check Your Understanding
Use the definitions and principles above to answer each question. Short solutions follow the questions.
- A treatment-placebo benefit is estimated as 4 points at Week 4 and 6 points at Week 12. What is the estimated early-to-later contrast, and how should its sign be interpreted?
- Why is it incorrect to infer that the treatment effect changed merely because a Week 4 estimate is statistically significant and a Week 12 estimate is not?
- Two time-specific effect estimates have standard errors 0.5 and 0.6 points and covariance 0.10 points squared. Calculate the standard error of their difference.
- A higher dose has a larger estimated benefit at the early assessment but nearly the same estimated benefit as a lower dose at the later assessment. What should a dose justification for use assessed at the later time emphasize?
Solutions. (1) The contrast is \(6-4=2\) points, so the estimated treatment-placebo benefit is 2 points larger at Week 12 than at Week 4. (2) The two significance results concern separate comparisons with zero; they do not directly test whether the estimates differ. Estimate the time contrast and its uncertainty. (3) The variance is \(0.5^2+0.6^2-2(0.10)=0.41\) points squared, so the standard error is \(\sqrt{0.41}=0.6403\) points. (4) Emphasize the later time-specific effects and the higher dose's incremental benefit there, with uncertainty and safety context; the early advantage should not be substituted for the later comparison.