Diagnosing Secular Trends in Control Outcomes

Why Examine Control Outcomes Over Calendar Time?

In a platform trial, control participants may be enrolled over a long period while treatments, eligibility criteria, clinical practice, or outcome assessment change. A control outcome rate that varies across calendar time can therefore be a warning that participants or the conditions of observation are not stable. It can also be an ordinary consequence of sampling variation. The diagnostic task is to describe the pattern carefully and decide whether it is substantial and credible enough to investigate further.

A secular trend is a systematic change in an outcome over calendar time, rather than a difference caused only by random variation among the observed participants. In this tutorial, the outcome is a binary event, such as response by a fixed assessment time or death by a prespecified follow-up time. A rate from one period is interpretable only if the outcome definition and the way follow-up is handled are comparable across periods.

The periods should be specified in a way that is clinically and operationally meaningful, such as calendar quarters or intervals defined by major changes in standard care. The outcome and assessment window should be consistent, and each denominator must be clear. For example, participants who have not yet reached the fixed assessment time should not be silently treated as non-events. If follow-up completion or ascertainment differs, the rate comparison may reflect that difference rather than a change in outcomes.

Summarize Before Interpreting

A useful first display is a table with the calendar period, event count, denominator, observed rate, and—when useful—an uncertainty measure. Counts alone are insufficient: 12 events could represent a low rate among many participants or a high rate among few. Rates alone are also incomplete because a rate based on 20 participants is less precise than the same rate based on 200.

If outcomes are independent Bernoulli observations within a period and participants have a common event probability \(p_j\), then \(Y_j\sim\mathrm{Binomial}(n_j,p_j)\). The variance of the observed rate is \(p_j(1-p_j)/n_j\). Replacing the unknown \(p_j\) with \(\hat{p}_j\) gives the estimated standard error:

When the sample is not too small and the rate is not near zero or one, an approximate 95% interval is \(\hat{p}_j\pm1.96SE(\hat{p}_j)\). This is a rough uncertainty summary, not a guarantee that the interval captures the true rate. For small denominators or extreme rates, this normal approximation can be poor; use an interval method suited to binomial data rather than relying on a negative lower endpoint or an upper endpoint above one.

A plot should make the time ordering visible. Plot each period’s rate at its calendar position, and show the number of participants and preferably uncertainty intervals. A connected line can guide the eye, but it does not imply that the rate changed smoothly between periods. If period lengths differ, label them clearly; do not let unequal time spans appear equivalent merely because each has one plotted point.

Separate the Pattern From Its Precision

A trend diagnosis should not be reduced to whether a test crosses a threshold. Consider the magnitude of the rate changes, their direction and persistence, the width of the uncertainty intervals, the number of periods, and whether any important operational event coincides with the pattern. A single unusual period followed by a return toward earlier rates may be random fluctuation; a sustained change that aligns with a change in standard care or eligibility warrants closer scrutiny.

The binomial standard error formula follows from the variance of a sample proportion. If \(Y_j\) is the number of events, then under the stated model \(E(Y_j)=n_jp_j\) and \(\operatorname{Var}(Y_j)=n_jp_j(1-p_j)\). Since \(\hat{p}_j=Y_j/n_j\), dividing the count variance by \(n_j^2\) gives \(\operatorname{Var}(\hat{p}_j)=p_j(1-p_j)/n_j\). This explains why equal-looking rates can have very different precision when denominators differ.

Diagnostics, Alternatives, and Limitations

A table and plot are descriptive diagnostics: they help locate and characterize differences but cannot establish their cause. Check that the plot’s time axis is correctly ordered, period boundaries are documented, denominators reconcile with the trial records, and the plotted rates use the same event definition and assessment window. Investigate sudden jumps around protocol amendments, changes in endpoint assessment, changes in follow-up, or shifts in enrollment criteria. Small counts, rates near zero or one, and many short periods can make apparent volatility especially misleading.

An omnibus comparison of rates, such as the chi-square calculation above, asks whether there is evidence that at least one period differs from the others. It does not specifically test a monotone increase or decrease, and a non-significant result may reflect limited precision. Conversely, a small p-value does not show that the difference is clinically important or explain its source. A trend model can quantify a time pattern under additional assumptions, but that is a different analysis from the initial descriptive diagnosis.

Compared with a single pooled control rate, period-specific rates preserve temporal information. Compared with a formal model, the table and plot require fewer modeling assumptions and make data irregularities easier to notice, but they do not adjust for differences in participant characteristics or provide a definitive causal explanation. Their purpose is to expose possible drift and guide careful review, not to replace design-based comparisons or a prespecified analysis.

A useful conclusion states what the data show and what remains uncertain: for example, “The observed control rate fell across the three periods, with a sizable first-to-last difference; the pattern merits review, but these summaries alone cannot distinguish a true change in control risk from changing participant mix or outcome ascertainment.” This is more informative than labeling the trend “present” or “absent” on the basis of a single p-value.