Why Monitor Event Accrual Against Plan?
In an event-driven trial, interim looks or final analysis may depend on reaching a target number of events. A planned schedule gives expected event counts at calendar dates, based on assumptions about enrollment, follow-up, event rates, and data processing. Monitoring compares that reference schedule with the events actually available at each date. It helps the team see whether the event target appears to be arriving sooner or later than planned and whether an operational check is warranted.
This comparison is a monitoring description, not an efficacy analysis. Being ahead of schedule does not mean treatment is more effective, and being behind does not show that treatment is less effective. Counts may differ from plan because of chance, enrollment patterns, follow-up, endpoint ascertainment, or delays in cleaning and adjudication.
Make the Comparison Fair
A meaningful comparison requires the observed and planned counts to refer to the same endpoint, population, and time scale. Before calculating deviations, specify the event definition, the calendar-time origin, the cutoff dates, and what counts as an event that is available for reporting. For example, a plan might count confirmed primary-endpoint events by the data cutoff, while a live operational report might include only events already adjudicated. Comparing those counts without accounting for the difference would confound event accrual with processing delay.
Use a documented version of the plan as the reference. The plan's assumptions should match those used to generate its expected counts, including the enrollment and follow-up patterns and any specified allowance for loss to follow-up or administrative censoring. If the plan is revised during monitoring, retain the original reference or clearly label the new one; otherwise, a change in the benchmark can look like a change in observed performance.
The observed counts should be cumulative counts as of the stated cutoff, not a mixture of event dates and data-availability dates. The earlier material on data cleaning and adjudication delays is relevant: an event may have occurred by a cutoff but not yet be verified and available in the count. Record the data extraction date and any known backlog alongside the counts.
Tables: Cumulative Counts and Interval Changes
A basic monitoring table should show the cutoff date, planned cumulative count, observed cumulative count, and deviation. It is often useful to add the counts accumulated since the preceding cutoff. If \(d_j=D_j-D_{j-1}\) is the observed number in interval \(j\), and \(m_j=M_j-M_{j-1}\) is the planned increase, then the interval deviation is \(q_j=d_j-m_j\). This distinguishes a current change in pace from the total difference carried forward from earlier periods.
The identity follows by writing \(D_j=\sum_{k=1}^{j}d_k\) and \(M_j=\sum_{k=1}^{j}m_k\), assuming both cumulative series start at zero. It explains why interval changes are a useful companion to cumulative deviations: an early surplus can be offset by later shortfalls, or a current surplus can coexist with an overall cumulative deficit.
Worked Example: Reading a Cumulative Monitoring Table
Suppose a hypothetical trial uses the same confirmed endpoint definition at each monthly cutoff. Its frozen plan expects cumulative counts of 18, 42, and 66 events at months 3, 6, and 9. The observed counts available at those cutoffs are 20, 39, and 62.
| Cutoff | Planned cumulative \(M_j\) | Observed cumulative \(D_j\) | Deviation \(R_j\) | Interval observed / planned |
|---|---|---|---|---|
| Month 3 | 18 | 20 | +2 | 20 / 18 |
| Month 6 | 42 | 39 | -3 | 19 / 24 |
| Month 9 | 66 | 62 | -4 | 23 / 24 |
At month 3, \(R_1=20-18=2\), so the observed count is 2 events ahead of plan. Between months 3 and 6, the observed increase is \(39-20=19\), versus a planned increase of \(42-18=24\). The interval deviation is \(19-24=-5\). Thus the cumulative deviation at month 6 is \(2+(-5)=-3\), which agrees with \(39-42=-3\). Between months 6 and 9, the increase is \(62-39=23\), one event below the planned increase of \(66-42=24\); the cumulative deviation becomes \(-3+(-1)=-4\).
The trial was initially ahead, then fell behind the cumulative schedule. In the last interval, accrual was close to the planned increase, but the earlier shortfall remained. Reporting only the latest interval would miss that cumulative context; reporting only the cumulative difference would hide the recent near-match in pace.
Plots for Seeing the Pattern
A cumulative plot places calendar time on the horizontal axis and event count on the vertical axis. Show the planned expected count as a line and the observed cumulative counts as points connected in time order. Label the event-count units and cutoff dates; state whether observed counts represent events occurring by the date or events available by the data extraction date. A vertical difference between the observed point and the planned line at a cutoff represents \(R_j\).
A second plot can show interval counts \(d_j\) and planned increments \(m_j\), or their differences \(q_j\). This view is sensitive to a recent change in pace, while the cumulative plot emphasizes the total gap to date. If the plan includes prediction intervals for event counts, a band around the planned curve can provide context for ordinary variability. A point outside a band is a reason to investigate assumptions and data quality; it is not automatically a formal hypothesis-test result, especially when many cutoffs are inspected.
Worked Example: A Count Outside the Planned Range
At month 6 in a hypothetical trial, the planned cumulative count is 50 events. The projection work supplied a 95% prediction interval of 42 to 58 events for that cutoff under its stated assumptions. The observed, confirmed cumulative count is 59.
The deviation is \(R=59-50=9\) events. Relative to the planned expectation, this is \(9/50=0.18\), or 18% above plan. The observed count is also one event above the upper end of the supplied prediction interval, since \(59>58\).
This is a useful flag to check the extraction date, endpoint definition, event processing, enrollment and follow-up assumptions, and whether the interval was intended for this cutoff. It does not establish that the underlying event rate has changed, nor does it give a 5% probability that the plan is wrong. The interval describes prediction uncertainty under a specified model; its interpretation depends on how that interval was constructed.
Look for Hidden Differences Across Groups
An overall count can conceal offsetting differences. If one site or enrollment period is ahead while another is behind, the aggregate deviation may be small even though important operational patterns differ. When appropriate and supported by the plan, inspect counts by site, region, or enrollment cohort as well as overall. Keep the same cutoff and event rules within each comparison, and avoid interpreting noisy small-group deviations as stable trends.
Worked Example: An Aggregate Count Masks Site Differences
At a hypothetical month-4 cutoff, a trial planned 20 events at Site A and 28 at Site B. The observed counts are 28 and 18, respectively. The total planned count is \(20+28=48\), and the total observed count is \(28+18=46\), so the overall deviation is \(46-48=-2\) events.
By site, however, Site A is \(28-20=+8\) events ahead, while Site B is \(18-28=-10\) events behind. The site deviations sum to \((+8)+(-10)=-2\), matching the overall difference. The aggregate count alone makes the trial appear close to plan, but the separate comparisons identify contrasting patterns that merit operational review.
These differences do not by themselves explain why the sites differ. Check whether site-specific enrollment, follow-up, event definitions, and data processing are comparable before drawing conclusions. A small count or a short observation period can also make a site-level deviation unstable.
Common Pitfalls and Alternatives
- Comparing different data cutoffs. A plan for events by month 6 cannot be fairly compared with a report extracted earlier unless the delay is explicitly accounted for. State both the event cutoff and data-availability date.
- Treating expected counts as required counts. The planned line is a reference expectation, not a deterministic quota. Random variation means observed counts need not equal the plan at every date.
- Calling a positive deviation evidence of efficacy. More events than planned says nothing by itself about the treatment contrast or the direction of a treatment effect.
- Confusing cumulative difference with recent pace. Use interval counts alongside cumulative counts to tell whether the current period is gaining or losing ground.
- Overreacting to repeated visual checks. A plot is a monitoring aid. Repeated inspection or points outside prediction bands should prompt investigation, not an unplanned formal claim about the model or treatment.
- Letting aggregation hide operational variation. Where relevant, inspect coherent strata, but account for small counts and differences in data completeness.
A cumulative table is the most direct way to document the comparison at scheduled cutoffs; a cumulative plot makes the direction and persistence of the gap easier to see. Interval summaries are better for recent pace, and stratified displays can reveal offsetting patterns. These are complementary monitoring views. Updating future event projections is a separate step: a deviation from plan alone does not specify which assumptions to change or how to calculate a revised forecast.
Check Your Understanding
Answer each question using the stated counts. Brief solutions follow so that the calculations and interpretations can be checked.
- At a cutoff, the plan is 35 cumulative events and 32 are observed. What is the deviation, and is accrual ahead or behind?
- Planned cumulative counts at two successive cutoffs are 24 and 40; observed counts are 27 and 39. Find the planned and observed interval increases and the interval deviation.
- A count is outside a supplied prediction interval. What does that observation warrant, and what does it not establish?
- Two sites have deviations of +6 and -5 events. What is their combined deviation, and what information would the combined number conceal?
- Name two checks needed before comparing an observed count with a planned count.
Solutions. 1. \(R=32-35=-3\), so the observed count is 3 events behind plan. 2. The planned increase is \(40-24=16\); the observed increase is \(39-27=12\); the interval deviation is \(12-16=-4\) events. 3. It warrants checking data quality, cutoff alignment, and model assumptions; it does not by itself prove the plan is wrong or constitute a formal test. 4. The combined deviation is \(+6-5=+1\) event. It conceals that one site is ahead and the other behind. 5. For example, verify that the event definition and population match, and that the calendar cutoff and data-availability rule are aligned.