Choosing Assessment Times for Delayed Responses

Why assessment timing matters

A treatment may not produce its full clinical response as soon as dosing begins. The delay can vary between patients, and the response may build gradually after it starts. An assessment scheduled too early can therefore measure a smaller treatment contrast than the contrast expected after responses mature. Moving the assessment later may reduce this shortfall, but also postpones the result and can increase exposure to dropout, disease progression, or other changes in follow-up.

Simulation helps compare candidate schedules under explicit assumptions about response delays, response trajectories, outcome variability, and trial conduct. The goal is not simply to find the date with the largest estimated effect. A useful comparison reports the contrast and its bias relative to a defined target, the probability of a prespecified decision, and the time at which that decision could be made.

Definition: An assessment schedule specifies the times at which the endpoint is measured and, if applicable, the times at which decisions may be made. For a treatment effect that changes over time, let \(\delta(t)\) be the mean treatment-versus-control contrast at time \(t\), and let \(\delta_{\infty}\) be the mature contrast under the model. The schedule-specific bias relative to this mature-effect target is \(\delta(t)-\delta_{\infty}\). This is a difference between two estimands at different response times; it is not automatically bias in an estimator of \(\delta(t)\).

Representing delayed and variable responses

One transparent starting model gives each treated participant a response delay \(D\), a plateau effect \(A\), and a ramp duration \(R\). Time \(t\) and delay \(D\) use the same units. For a linear ramp, define the fraction of the plateau reached \(g(t-D)\) as follows.

g(u)= \begin{cases} 0, & u\leq 0,\\ u/R, & 0<u<R,\\ 1, & u\geq R. \end{cases} \qquad \tau(t)=A g(t-D).

Here \(\tau(t)\) is the individual treatment effect at time \(t\); the control treatment effect is set to zero. If the delay and plateau effect vary across people, the population treatment contrast is

\delta(t)=E\{A g(t-D)\}.

If every response eventually reaches its plateau and the mean plateau effect exists, then \(\delta_{\infty}=E(A)\). When \(A\) and \(D\) are associated, the expectation must preserve their joint distribution. Replacing it with separate average values can give the wrong contrast.

A simulation can sample delays and effects from specified distributions, generate outcomes at each candidate assessment, and apply the planned analysis. Other features, including baseline values, residual variability, discontinuation, and missing measurements, should reflect the trial question and be held comparable across schedules unless the schedule itself changes them. The assumed response model is a working description, not proof that real responses follow a linear ramp.

Deriving the expected contrast

For a finite set of delay groups, the population contrast is a weighted average of the group-specific effects. The justification is linearity of expectation: if group \(j\) has probability \(w_j\), delay \(D_j\), and plateau \(A_j\), then the expected effect equals the sum of each effect multiplied by its group probability.

\delta(t)=\sum_j w_j A_j g(t-D_j), \qquad \sum_j w_j=1.

This calculation is a useful check on simulation output. At a fixed time, the simulated mean contrast should be close to this population value, allowing for Monte Carlo error. A discrepancy much larger than simulation error suggests a coding error, a mismatch between the stated and implemented response model, or an unintended feature such as differential missingness.

Worked Example: Comparing Four Assessment Times

Hypothetical data-generating model. Suppose the plateau effect is \(A=0.60\) outcome units for every treated participant, the ramp duration is \(R=4\) weeks, and response delays are 0 weeks for 25% of participants, 4 weeks for 50%, and 10 weeks for 25%. The mature contrast is \(\delta_{\infty}=0.60\) outcome units.

At week 6, the delay-0 group has reached its plateau, the delay-4 group has completed half its ramp, and the delay-10 group has not started. Thus

\delta(6)=0.25(0.60)+0.50(0.60\times 2/4)+0.25(0)=0.300

outcome units, giving bias relative to the mature target of \(0.300-0.600=-0.300\) outcome units. At week 10, the first two groups have reached their plateaus, but the delay-10 group has not started:

\delta(10)=0.25(0.60)+0.50(0.60)+0.25(0)=0.450.

At week 12, the delay-10 group has completed only \(2/4\) of its ramp. Its effect is \(0.60\times2/4=0.30\), not \(0.60\). Therefore,

\delta(12)=0.25(0.60)+0.50(0.60)+0.25(0.60\times2/4)=0.525.

The week-12 bias is \(0.525-0.600=-0.075\) outcome units. At week 14, all groups have reached the plateau, so \(\delta(14)=0.600\) and the bias is zero under this model. This comparison makes the delay distribution consequential: knowing only the average delay would not reveal the contributions of all three groups at week 12.

Relating assessment time to power

A simple calculation can illustrate how a changing contrast affects power. Assume a two-arm randomized trial with \(n\) participants per arm at the assessment, independent normally distributed outcomes with common variance \(\sigma^2\), and no missing observations. For a difference in sample means, the standard error is \(\sqrt{2\sigma^2/n}\). A two-sided normal-approximation test at level \(\alpha\) has approximate power

P\left(|Z|>z_{1-\alpha/2}\right), \qquad Z\sim N\left(\frac{\delta(t)}{\sqrt{2\sigma^2/n}},1\right).

This approximation assumes the treatment contrast and variance used in the calculation correspond to the assessment being evaluated. With estimated variance, small samples, non-normal outcomes, or complex analysis, simulate the planned analysis rather than relying on this approximation. Power is the probability of rejecting \(H_0\) under the specified alternative and model; it is not the probability that the treatment works.

Worked Example: Correcting the Week-12 Power Calculation

Continue the hypothetical three-delay model. Assume \(n=100\) per arm, outcome variance \(\sigma^2=1.10\) outcome-units squared, a two-sided \(\alpha=0.05\), and the normal approximation above. These assumptions give

SE=\sqrt{\frac{2(1.10)}{100}}=\sqrt{0.022}=0.1483

outcome units. At week 12, use the correctly weighted contrast, \(0.525\) outcome units:

\lambda(12)=\frac{0.525}{0.1483}\approx3.540.

With \(z_{0.975}\approx1.96\), the approximate two-sided power is

P(Z>1.96)+P(Z<-1.96) =\Phi(3.540-1.96)+\Phi(-1.96-3.540) \approx0.9429.

Thus approximate power is about 0.943, or 94.3%, under the stated assumptions. Using the mature contrast of 0.60 at week 12 would be incorrect because the group whose response begins at week 10 has completed only half its ramp. At week 10 the contrast is \(0.450\), giving \(\lambda(10)=0.450/0.1483\approx3.034\) and approximate power \(0.8586\). The later assessment increases power in this example, but delays the result by two weeks.

Worked Example: A Different Distribution of Response Delays

Consider a second hypothetical trial model with \(A=0.60\) outcome units, a four-week linear ramp, and delays of 2 weeks for half the participants and 6 weeks for the other half. At week 8, the first group is past its ramp, while the second has completed half its ramp. Therefore,

\delta(8)=0.50(0.60)+0.50(0.60\times2/4)=0.450

outcome units. At week 10 both groups have reached the plateau, giving \(\delta(10)=0.600\) outcome units. Relative to the mature target, the week-8 bias is \(-0.150\) outcome units and the week-10 bias is zero.

For an illustrative power comparison, assume \(n=64\) per arm and \(\sigma^2=1.44\) outcome-units squared. Then \(SE=\sqrt{2(1.44)/64}=\sqrt{0.045}=0.2121\) outcome units. At week 8, \(\lambda=0.450/0.2121\approx2.121\), giving approximate two-sided power \( \Phi(2.121-1.96)+\Phi(-2.121-1.96)\approx0.564\). At week 10, \(\lambda=0.600/0.2121\approx2.828\), and power is approximately \(\Phi(0.868)+\Phi(-4.788)\approx0.807\). This schedule comparison depends on the assumed delay mixture; it is not a universal recommendation to assess at week 10.

Simulating decision timing

For a decision made at one prespecified assessment, the simulation can estimate power and the distribution of the analysis result at that time. If decisions may be made at several assessments, define the decision rule before comparing schedules. Repeatedly applying an unadjusted two-sided 0.05 test and declaring success if any assessment is significant does not preserve a 0.05 overall false-positive rate. The rule must be designed and calibrated for the repeated opportunities to decide.

For each simulated trial, record whether the rule produces a positive decision and, if it does, the first assessment time at which it does so. A useful schedule summary includes the proportion with a positive decision by the final assessment and the distribution of decision time among those with a positive decision. Also report the proportion not reaching the decision by the end; a mean decision time calculated only among successes does not describe all trials. Evaluate the false-positive rate under a null scenario as well as performance under treatment scenarios.

Worked Example: Interpreting Simulated Decision Times

Suppose 2,000 hypothetical trial replicates are simulated under a treatment scenario. Two assessment schedules use prespecified decision rules calibrated to the same overall type I error target. With assessments at weeks 8, 12, and 16, 300 replicates first meet the rule at week 8, 650 at week 12, and 550 at week 16. The estimated probability of a positive decision by week 16 is

\widehat{P}(\text{decision by week 16})=\frac{300+650+550}{2000}=0.750.

Among the 1,500 replicates with a positive decision, the mean first-decision time is

\frac{300(8)+650(12)+550(16)}{1500} =\frac{19000}{1500}=12.67\text{ weeks}.

For a denser schedule with assessments at weeks 8, 10, 12, 14, and 16, suppose the corresponding first-decision counts are 250, 350, 400, 300, and 260. The estimated decision probability is \(1560/2000=0.780\), and the mean time among decisions is

\frac{250(8)+350(10)+400(12)+300(14)+260(16)}{1560} =\frac{18660}{1560}=11.96\text{ weeks}.

In these illustrative simulation results, the denser schedule has a 0.030 higher estimated probability of a positive decision by week 16 and a 0.71-week shorter mean decision time among positive decisions. These summaries alone do not establish which schedule is preferable: check the null error rate, Monte Carlo uncertainty, costs, burden, and performance across plausible response models before drawing a conclusion.

Diagnostics, alternatives, and limitations

Before using a schedule comparison, check that the simulation implements the intended response and assessment times. Useful diagnostics include confirming that effects are zero before the delay and reach the intended plateau after the ramp; comparing simulated mean contrasts with the weighted calculation for simple delay mixtures; and checking that outcomes, missingness, and analysis inputs have the intended units and distributions. Under the null, verify that the decision procedure's false-positive rate is close to its target within simulation uncertainty. The earlier material on estimating operating characteristics from replicates and quantifying Monte Carlo error provides the basis for interpreting these simulated proportions.

Common failure modes include using the mature effect at an assessment where some responses are still ramping, confusing an imprecise estimate with a biased estimand, and comparing schedules using different decision rules. If response delays are uncertain, repeat the comparison over plausible delay distributions and ramp durations. If discontinuation or missing measurements depend on response or prognosis, model those processes explicitly; analyzing only participants who remain observed can change the population represented by the contrast.

A single fixed assessment is a straightforward alternative when one clinical time point defines the primary endpoint. A longitudinal analysis can use measurements from several times and may estimate a time course, but it requires an appropriate model and does not make an early endpoint automatically equivalent to a mature response. Simulation of candidate schedules answers a different practical question: how much information and how soon a specified decision rule may provide under stated assumptions. It cannot remove uncertainty about those assumptions or replace clinical justification of the endpoint.

Common pitfall: Treating every response group as if it has reached the plateau at a late-looking assessment can inflate the assumed contrast and power. For each group, first calculate elapsed response time \(t-D_j\), then apply the ramp function \(g(t-D_j)\), and only then weight the group effects. Assessment-specific power must use that resulting \(\delta(t)\), not \(\delta_{\infty}\) by default.

Key takeaway: Compare assessment schedules against an explicit target and a stated response-time model. Report how the schedule affects contrast bias, decision probability, and decision timing, while checking null calibration and sensitivity to plausible delays.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

A response has plateau effect 0.40 units, a three-week linear ramp, and a fixed delay of 5 weeks. What is the effect at week 6, and what is the bias relative to the plateau?
In a delay mixture, 60% of participants have delay 0 and 40% have delay 6 weeks. All have plateau effect 0.50 units and a four-week ramp. Calculate the mean contrast at week 8.
Why is it invalid to use the mature contrast to calculate power at an assessment where some participants have not completed their response ramp?
Two schedules allow testing at different numbers of assessments. What must be checked before comparing their probabilities of a positive decision?

Solutions. (1) At week 6, elapsed response time is \(6-5=1\) week, so the response has completed \(1/3\) of its ramp. The effect is \(0.40(1/3)=0.1333\) units, and bias relative to the plateau is \(0.1333-0.400=-0.2667\) units. (2) At week 8, the delay-0 group has reached its plateau; the delay-6 group has completed \(2/4\) of its ramp. The contrast is \(0.60(0.50)+0.40(0.50\times2/4)=0.400\) units. (3) The assessment-specific contrast is the weighted mean of the effects actually reached by that time. Substituting the mature contrast overstates the effect if some participants are still ramping, which can overstate calculated power. (4) Check that the repeated-assessment decision rules are specified and calibrated to a comparable overall type I error rate, and compare performance under both null and treatment scenarios.