Delayed Intermediate Outcomes

Why Delayed Intermediate Outcomes Change the Look Schedule

In a multi-arm, multi-stage (MAMS) trial, arm selection may depend on an intermediate outcome collected before the final endpoint. But “intermediate” does not necessarily mean “immediately available.” A landmark outcome may require several months of follow-up, and even after that period, data transfer or adjudication may add delay. Thus, the number enrolled by a planned selection look can substantially exceed the number whose intermediate outcomes can actually inform the decision.

The practical task is to plan the calendar time of selection using both the enrollment schedule and the outcome-availability schedule. This is distinct from deciding how to rank or retain arms once data are available: here the focus is on how much usable information is expected to be available at each look, and when a prespecified information target can be reached.

Definition: Let \(t\) denote calendar time since trial opening, and let \(u\) denote a participant’s enrollment time. The readout delay \(D\) is the elapsed time from enrollment until the intermediate outcome is available for the selection analysis. An enrolled participant contributes an available outcome at look \(t\) only if \(u+D\leq t\).

For an outcome defined at a clinically meaningful follow-up landmark, the delay includes the time needed to reach that landmark. It may also include a prespecified data-processing or adjudication interval. For example, if the outcome is status at six months and the data take two weeks to finalize, a participant generally cannot contribute that six-month status until approximately six months plus two weeks after enrollment. The trial protocol should specify what “available” means operationally, rather than treating the date of enrollment or the date of a clinic visit as sufficient.

Expected Numbers Available at a Selection Look

Index the control arm by \(a=0\) and the experimental arms by \(a=1,\ldots,K\). Let \(r_a(u)\) be the planned enrollment rate in arm \(a\) at time \(u\), measured in participants per unit of calendar time. Let \(G_a(v)=P(D\leq v)\) be the probability that an intermediate outcome in arm \(a\) is available within \(v\) time units after enrollment. The expected number of available outcomes in that arm by look \(t\) is

E\{N_a(t)\}=\int_0^t r_a(u)G_a(t-u)\,du.

This expression adds contributions from enrollment at every time \(u\). Participants enrolled earlier have had longer to complete follow-up and processing, so their probability of being available by \(t\) is \(G_a(t-u)\). The formula is an expectation; it does not guarantee that the realized number available will equal that expectation. It requires a plausible enrollment-rate plan and a plausible description of readout delays. Arm-specific rates or delays should be used when those differ across arms.

Fixed-delay result: If every participant’s outcome is available exactly \(d\) time units after enrollment, then \(G_a(v)=0\) for \(v<d\) and \(G_a(v)=1\) for \(v\geq d\). Writing \(A_a(x)=\int_0^x r_a(u)\,du\) for cumulative planned enrollment, the expected number available at look \(t\) is \(A_a(t-d)\) when \(t\geq d\), and zero when \(t<d\).

The fixed-delay result follows by substituting the step-function availability probability into the integral. Only participants enrolled by \(t-d\) can have an outcome available by \(t\), so the integral reduces to enrollment accumulated through that time. If accrual is constant at rate \(r_a\) from trial opening, then \(A_a(x)=r_ax\), and availability grows as \(r_a(t-d)\) after the delay has elapsed.

Specify the readout.
State the follow-up landmark, any processing interval, and the rule for when an outcome counts as available.

Map the enrollment plan to availability.
Use cumulative accrual for a fixed delay, or the delay distribution in the integral when readout times vary.

Check each arm at candidate looks.
Compare expected available numbers with the prespecified information requirements, including the requirement for the shared control arm.

Plan for uncertainty and contingencies.
Recognize that realized accrual and availability vary; use an appropriate probability calculation or simulation when meeting a minimum is important.

Worked Planning Examples

Worked Example: A Fixed Landmark and Adjudication Delay

Suppose a hypothetical three-arm trial enrolls participants at a constant rate of 12 per month in each arm, beginning at time zero. The intermediate outcome is defined at a four-month follow-up landmark, and adjudication takes an additional half-month. A selection look is planned at month 10. How many outcomes are expected to be available in each arm?

The fixed delay is \(d=4+0.5=4.5\) months. Only participants enrolled by \(10-4.5=5.5\) months can have an available outcome. Therefore,

E\{N_a(10)\}=12(10-4.5)=12(5.5)=66.

The expected number available is 66 per arm. By month 10, 120 participants per arm have been enrolled, since \(12(10)=120\). The difference is not a discrepancy: the 54 participants enrolled after month 5.5 have not yet completed the landmark and adjudication interval. This calculation assumes the planned constant accrual rate is achieved and the stated delay applies to all participants.

Worked Example: Finding the Earliest Look for Mature-Sample Targets

Consider a hypothetical trial with two experimental arms and one shared control arm. Each experimental arm is planned to enroll 10 participants per month, and the control arm 15 per month. All outcomes have a fixed five-month delay. The design requires at least 50 available outcomes in each experimental arm and 75 available control outcomes before selection. Find the earliest look meeting these expected-count targets.

With a fixed five-month delay, the expected available count at time \(t\) is the arm’s accrual rate multiplied by \(t-5\), for \(t\geq5\). Each experimental arm requires

10(t-5)\geq 50 \quad\Longrightarrow\quad t-5\geq 5 \quad\Longrightarrow\quad t\geq 10.

The control requirement gives

15(t-5)\geq 75 \quad\Longrightarrow\quad t-5\geq 5 \quad\Longrightarrow\quad t\geq 10.

Thus month 10 is the earliest look meeting all three expected-count targets. At that time, 50 outcomes are expected in each experimental arm and 75 in the control arm. The control participants are shared across the two comparisons; they are counted once as a control cohort, not as two separate groups of 75. If the targets are minimum realized counts rather than expected counts, this calculation alone is insufficient: the schedule must also account for variability in accrual and readout.

Worked Example: Variable Readout Delay

Suppose, hypothetically, that one arm enrolls at a constant rate of eight participants per month. Its readout delay varies uniformly between four and six months, so \(D\sim\mathrm{Uniform}(4,6)\). At a look at month 10, calculate the expected number with an available outcome.

For elapsed time \(v\), the availability probability is zero if \(v<4\), equals \((v-4)/2\) for \(4\leq v<6\), and equals one for \(v\geq6\). Using \(v=10-u\), the expected count is

E\{N_a(10)\} =8\left\{\int_4^6 \frac{v-4}{2}\,dv+\int_6^{10}1\,dv\right\}.

The first integral is \(\left[(v-4)^2/4\right]_4^6=1\), and the second is \(10-6=4\). Hence

E\{N_a(10)\}=8(1+4)=40.

Eighty participants are expected to have enrolled by month 10, but only 40 outcomes are expected to be available. This is an average across the delay distribution, not a promise that exactly 40 will be available at the look. A design with a hard minimum should quantify the probability of falling below that minimum under the planned accrual and delay variability.

Choosing and Checking the Schedule

The main scheduling alternatives are to keep a fixed calendar look, delay selection until enough outcomes mature, or modify enrollment or follow-up operations to make information available sooner. A fixed calendar look is operationally simple, but it may yield few usable outcomes when accrual starts slowly or delays are longer than planned. Waiting for a mature-sample target makes the information requirement explicit, but can extend the trial calendar. Increasing accrual can bring more participants into the pipeline, although those newly enrolled participants still need the required follow-up and processing time.

In a MAMS design, each experimental arm and the shared control should be checked against their relevant availability requirements. Allocation ratios affect these counts: a control arm enrolling more slowly can determine the earliest feasible look even when experimental-arm targets have already been met. If accrual rates vary over time—for example, because sites open in stages—use the cumulative planned accrual curve rather than assuming a constant rate. If readout delays vary by site, participant group, or arm, a single average delay may mask important shortages; use an appropriate delay distribution or stratum-specific schedule.

A useful diagnostic is to compare, at each planned look, three quantities for every arm: total enrolled, expected outcomes available, and the target needed for selection. Large gaps between enrollment and availability are expected when follow-up is long, but they should be visible in the schedule. Revisit the calculation if recruitment is slower than planned, landmark visits are missed, adjudication takes longer, or data transfer creates a queue. These operational changes alter the availability schedule even if the nominal selection date does not change.

The calculation does not itself establish that the intermediate outcome is a suitable basis for selecting arms, nor does it adjust for differences between participants whose outcomes are available and those whose outcomes are not. It answers a timing and information-availability question. In particular, a participant without an available outcome at the look should not be silently treated as having a negative or unfavorable outcome. The protocol needs a clear approach to incomplete readouts, and any resulting selection procedure must be evaluated under the trial’s planned assumptions.

Common Pitfalls and Limitations

Counting everyone enrolled by the look. This overstates usable information whenever enrollment occurs within the required follow-up or processing window. Count only participants whose readout can be available by the look.
Using the follow-up landmark but omitting data-processing time. If the outcome requires adjudication or data transfer, include that delay in the availability definition or explicitly model its variation.
Treating an expected count as guaranteed. The integral gives an expected number. Realized enrollment and readout may be lower or higher, so a strict minimum calls for a probability-based check.
Applying one arm’s schedule to all arms without justification. Different recruitment rates, site mixes, or readout processes can produce different availability schedules.
Confusing outcome maturity with outcome quality. A mature outcome has completed the required observation and processing interval; maturity alone does not make it an unbiased or informative selection measure.

The schedule is limited by the accuracy of its inputs. Planned accrual can differ from actual recruitment, and delay distributions estimated from an earlier setting may not match the trial’s operational conditions. Sensitivity analyses across plausible accrual rates and delays are therefore valuable, especially when the selection date is difficult to move. These checks are about timing and availability, not about predicting final-endpoint success.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions below.

With a fixed delay of three months and constant accrual of 20 participants per month from time zero, how many outcomes are expected to be available at month 8?
An arm needs 36 available outcomes, accrues at nine participants per month, and has a fixed four-month delay. What is the earliest look meeting the expected-count target?
Why can a look with 100 participants enrolled still have fewer than 100 available outcomes?
For a variable readout delay, what does \(G_a(t-u)\) represent in the availability integral?

Solutions. (1) Only participants enrolled by month \(8-3=5\) can be available, so the expected count is \(20(5)=100\). (2) Solve \(9(t-4)\geq36\), giving \(t-4\geq4\) and therefore an earliest look at month 8. (3) Participants enrolled too recently to complete the required follow-up and processing interval are not yet available; enrollment and availability are different counts. (4) It is the probability that a participant enrolled at time \(u\) has an outcome readout delay no longer than the remaining time \(t-u\), and hence is available by look \(t\).

What You'll Learn