Scheduling Looks for Intermediate Outcomes

Why the Timing of a Look Matters

A selection rule can only use the information available when the review occurs. Schedule a look too early, and many enrolled participants may not yet have an observed intermediate outcome; event-based information may also be sparse. Wait longer, and outcomes mature, but an unpromising arm continues to accrue participants and the decision to stop it is delayed. Scheduling therefore involves more than choosing a convenient calendar date: it means making explicit what level of outcome maturity and information is needed, and when that information is likely to become available.

This tutorial focuses on timing selection looks in a time-to-event MAMS setting with an intermediate outcome. The calculations below describe potential information at candidate looks; they do not determine the selection threshold or establish the operating characteristics of a complete design. The intermediate and final outcomes may also differ in their implications for treatment benefit, as emphasized in the earlier material on selection when intermediate and final effects disagree.

Define the Look and Its Information

Definition: A selection look is a prespecified review at which information available from the trial is used to decide whether treatment arms continue or are dropped. A calendar-time look is scheduled for a stated time after trial start. An event-triggered look is scheduled when a stated number of events has accrued. For an intermediate outcome assessed after a fixed window, matured participants are those whose assessment window has elapsed and whose outcome is available by the look.

A useful schedule should specify the time or trigger, the population whose outcomes can contribute, and any practical delay between an outcome occurring and becoming available for review. When the intermediate outcome is assessed at a fixed time after enrollment, calendar time alone does not imply that every enrolled participant has a result. For a time-to-event outcome, enrollment duration and the event-time distribution both affect how many events are available.

For a transparent planning calculation, suppose participants accrue at a constant rate \(r\) per month from month 0 through month \(A\), and stop accruing thereafter. Suppose the event time after enrollment has an exponential distribution with hazard \(\lambda\), independently across participants, with no censoring before the look. These assumptions are simplifying: they exclude staggered changes in accrual, loss to follow-up, competing events, and treatment-specific event rates. The expected event count per arm at calendar time \(t\) can then be calculated directly.

Expected Events Under Constant Accrual

A participant enrolled at time \(u\) has been followed for \(t-u\) months at a look at time \(t\). Under an exponential event-time model, the probability that this participant has experienced the event by the look is \(1-\exp\{-\lambda(t-u)\}\). With accrual at rate \(r\), the expected event count is the integral of that probability over the enrollment times that have occurred by \(t\):

E\{D(t)\} = r\int_0^{\min(t,A)} \left[1-\exp\{-\lambda(t-u)\}\right]\,du.

If accrual has ended by the look, so \(t\geq A\), integration gives

E\{D(t)\} = r\left[ A-\frac{\exp\{-\lambda(t-A)\}-\exp(-\lambda t)}{\lambda} \right].

The expression separates the number enrolled, \(rA\), from those expected to have experienced the event by time \(t\). The expected event count is not itself a guarantee that the realized count will equal that value. It is a planning summary under the stated assumptions; actual counts vary.

Worked Example: Expected Events at an Early Look

Suppose, hypothetically, that one arm accrues 10 participants per month for 8 months, and the event hazard is \(0.10\) per month. What is the expected event count at month 11.2?

Here \(r=10\), \(A=8\), \(\lambda=0.10\), and \(t=11.2\), so \(t\geq A\) and the ended-accrual formula applies:

\begin{aligned} E\{D(11.2)\} &=10\left[ 8-\frac{\exp\{-0.10(11.2-8)\}-\exp(-0.10\times11.2)}{0.10} \right]\\ &=10\left[ 8-\frac{\exp(-0.32)-\exp(-1.12)}{0.10} \right]\\ &\approx10[8-3.998692]\\ &\approx40.0131. \end{aligned}

Thus the expected count is about 40.01 events per arm. This is a modest amount of event information relative to the 80 participants enrolled; many participants have not yet experienced the event by month 11.2. The calculation depends on the assumed accrual pattern and hazard, not just on the number enrolled.

Outcome Maturity for a Fixed Assessment Window

For an intermediate outcome measured at a fixed window \(W\) after enrollment, with an additional availability or processing delay \(L\), a participant enrolled at time \(u\) is expected to have an available outcome at look time \(t\) only if \(u+W+L\leq t\). Under constant accrual through month \(A\), the number of participants whose outcomes are mature is therefore \(r\min\{A,\max(0,t-W-L)\}\), before allowing for missing outcomes. This is a count of participants eligible to contribute, not a claim that every outcome will actually be observed.

Worked Example: Counting Mature Intermediate Outcomes

In a hypothetical trial, 8 participants enroll per week for 12 weeks. The intermediate outcome is assessed 6 weeks after enrollment, and results take 1 additional week to become available. At the end of week 17, how many of the 96 enrolled participants can be expected to have an available outcome, assuming no missing outcomes?

An outcome is available by week 17 for participants enrolled by week \(17-6-1=10\). Accrual has been 8 per week through week 12, so the number enrolled by week 10 is

8\times 10=80\text{ participants}.

The total enrollment is \(8\times12=96\), so \(80/96=0.8333\), or about 83.3%, of enrolled participants can contribute at that look. The remaining 16 participants are enrolled but their assessment and availability period has not elapsed. A review at week 19 would allow all 96 participants to reach the 7-week outcome-availability point, assuming the outcomes are recorded as planned.

Compare Candidate Schedules

A practical schedule can be assessed against prespecified planning targets, such as a minimum expected event count and a minimum proportion of participants with an available intermediate outcome. These targets are design choices, not universal standards. They make the timing trade-off visible: an early look may provide an earlier chance to stop an arm but use less mature information; a later look may improve maturity and event accumulation but delay that decision.

Worked Example: Choosing Among Calendar-Time Looks

Consider a hypothetical arm with accrual at 12 participants per month for 10 months and an exponential event hazard of \(0.12\) per month. The intermediate outcome has a 2-month assessment window and a 0.5-month availability delay. Compare looks at months 10, 16, and 20. For illustration, suppose the planning targets are at least 80 expected events per arm and at least 90% of the 120 enrolled participants with a mature intermediate outcome.

At month 10, accrual has just ended. The expected event count is

\begin{aligned} E\{D(10)\} &=12\left[ 10-\frac{1-\exp(-0.12\times10)}{0.12} \right]\\ &=12\left[10-\frac{1-\exp(-1.2)}{0.12}\right]\\ &\approx50.1194. \end{aligned}

At month 16, the ended-accrual formula gives

\begin{aligned} E\{D(16)\} &=12\left[ 10-\frac{\exp(-0.12\times6)-\exp(-0.12\times16)}{0.12} \right]\\ &\approx85.9855. \end{aligned}

At month 20, it gives

\begin{aligned} E\{D(20)\} &=12\left[ 10-\frac{\exp(-0.12\times10)-\exp(-0.12\times20)}{0.12} \right]\\ &\approx98.9524. \end{aligned}

For the intermediate outcome, the maturity cutoff is \(t-2.5\) months. At month 10, only participants enrolled by month 7.5 can contribute: \(12\times7.5=90\), or 75% of the 120 enrolled. At months 16 and 20, the cutoff is beyond the end of accrual, so all 120 outcomes can be mature, assuming none are missing.

Month 10 meets neither illustrative target. Month 16 meets both: it provides about 85.99 expected events and mature outcomes for all 120 participants. Month 20 also meets both and provides about 98.95 expected events, but the additional four months yield only about \(98.9524-85.9855=12.9669\), or 12.97, further expected events per arm. The choice between months 16 and 20 therefore depends on whether that additional expected information justifies waiting longer. This calculation does not establish that either schedule has desirable selection performance; the selection rule and its consequences must be considered as part of the full design.

Calendar-Time and Event-Triggered Scheduling

Calendar-time scheduling fixes the review date in advance. It is operationally predictable and makes it straightforward to plan the maturity of fixed-window outcomes. Its limitation is that the event count at that date is uncertain and may be lower than anticipated if accrual or event occurrence is slower than assumed.

An event-triggered schedule instead reviews when a prespecified number of events has accrued. It ties the review to event information more directly, but the calendar date is uncertain. If a fixed-window intermediate outcome is also needed, reaching the event trigger does not ensure that the required outcomes are mature. A trial can therefore specify both an event trigger and a minimum maturity condition, or use calendar-time looks chosen to make the relevant outcome information available. Each choice changes when a selection decision can occur.

A third practical approach is to schedule looks after a stated proportion of enrollment has completed and then allow a prespecified follow-up and data-availability period. This can be easier to coordinate with an intermediate assessment, but enrollment completion alone is not information completion. In each approach, the design should state what happens if a trigger is reached before the necessary outcomes are available; an undefined delay or an improvised review date can make the actual schedule differ from the planned one.

Assumptions, Diagnostics, and Limitations

The formulas above are useful only insofar as their inputs reasonably describe the trial. Check that the assumed accrual rate and accrual duration reflect the operational plan, that the event hazard is a plausible approximation over the relevant follow-up, and that assessment and processing delays are realistic. If enrollment varies over calendar time, the constant-rate calculation should be replaced by a calculation using the planned accrual pattern. If event rates differ by arm, use arm-specific assumptions rather than treating a common rate as established fact.

Observed accrual, event counts, and assessment completion can be compared with planning expectations as the trial proceeds. Large discrepancies are a reason to revisit predictions of when information will be available, not a reason to silently change the selection rule. In particular, delayed outcome recording, missing assessments, or unexpectedly low event rates can make an apparently mature scheduled look less informative than intended.

A schedule with more looks offers more opportunities to make a decision, but those decisions are not independent: information accumulates over time, and successive looks usually contain overlapping participants and events. More looks do not automatically improve a design, and scheduling calculations alone do not establish error control, power, or the probability of dropping an arm. Those properties require evaluation of the full selection procedure and its decision rules. This tutorial’s calculations help set plausible times and information expectations; they are not a substitute for that evaluation.

Key takeaway: Choose a selection-look schedule by relating calendar time to both outcome maturity and expected event information. State the assumptions, include any assessment and reporting delays, and distinguish expected information from guaranteed information. Then evaluate the complete selection design rather than treating the schedule itself as proof of performance.

Check Your Understanding

Answer each question, then compare with the brief solutions.

A fixed-window intermediate outcome requires 4 months after enrollment and has a 1-month reporting delay. At a look at month 14, by what month must a participant have enrolled to have an available outcome?
An arm accrues 5 participants per month for 8 months. At month 10, how many participants have been enrolled, and how many have had at least 5 months for assessment and reporting?
Under constant accrual and an exponential event-time model, what does \(E\{D(t)\}\) represent? Is it the realized event count at time \(t\)?
Give one advantage and one limitation of an event-triggered look when selection also uses a fixed-window intermediate outcome.
In the third worked example, how many additional expected events per arm are gained by moving from month 16 to month 20, to two decimal places?

Solutions. (1) Enrollment must occur by \(14-4-1=9\) months. (2) Total enrollment is \(5\times8=40\). At month 10, an available outcome requires enrollment by month \(10-5=5\), giving \(5\times5=25\) participants, assuming no missing outcomes. (3) It is the model-based expected number of events by time \(t\), not the realized count; the realized count is variable. (4) An advantage is that the review is tied to a stated amount of event information. A limitation is that reaching the event trigger does not ensure the fixed-window outcomes are mature, and the calendar date is uncertain. (5) The gain is \(98.9524-85.9855=12.9669\), or 12.97 expected events per arm.