Calendar Time as a Confounder in Platform Comparisons

Why Calendar Time Can Distort a Platform Comparison

In a platform trial, treatment arms may open or close at different times. Meanwhile, standard care, eligibility rules, supportive care, and outcome measurement can also change. As a result, participants receiving a particular treatment may come from a different calendar period than the control participants used in the comparison. The observed difference can then reflect both treatment and changes over time.

Calendar time is a confounder when it is associated with which treatment participants receive and also influences the outcome. This is not an assertion that every comparison in a platform trial is confounded: concurrent randomization can protect a comparison within the same period. The concern is especially important when a treatment is compared with controls recruited in a different period, or when a pooled comparison combines periods in different proportions across arms.

Definition: Calendar time is a confounder of a treatment–outcome comparison when it is a common cause, directly or through other variables, of treatment received and the outcome. In a platform trial, time may affect treatment assignment because an arm is available only during particular periods, and affect outcomes because care, patient mix, or outcome ascertainment changes over time.

Reading the Causal Diagram

Let \(T\) denote calendar period, \(A\) indicate treatment assignment (\(A=1\) for the treatment of interest and \(A=0\) for control), and \(Y\) denote the outcome. A basic causal diagram is:

T \longrightarrow A \longrightarrow Y \qquad\text{and}\qquad T \longrightarrow Y

The path \(A \leftarrow T \rightarrow Y\) is a backdoor path: it connects treatment and outcome through a common cause rather than through the treatment effect. For example, an arm might open after a new standard-of-care option becomes available. The period then helps determine whether a participant can receive the study treatment, while the care change can affect the outcome regardless of study treatment.

Arm availability makes the structure more explicit. If \(V\) indicates whether the treatment is available at enrollment, one possible diagram is \(T \rightarrow V \rightarrow A \rightarrow Y\), together with \(T \rightarrow Y\). Availability does not itself create an outcome effect in every setting; it can link treatment assignment to time, which is related to the outcome. Other causes of time-related differences may include changing eligibility, referral patterns, supportive care, or the way outcomes are measured.

A causal diagram describes assumptions about cause and effect; it does not establish those assumptions from observed data. In a randomized comparison conducted within a period, randomization makes treatment assignment independent of prognosis within that period, apart from chance. But if treatment is compared with controls from another period, the randomization in the original trial does not make those groups exchangeable across time.

Conditions for a causal interpretation: The treatment and outcome must be well defined, and the observed outcome must correspond to the treatment received (consistency). Participants must not affect one another’s outcomes in a way that violates the analysis (no interference). For a period-specific comparison, treatment groups must be comparable within that period, as through valid randomization. To estimate a treatment effect across periods by comparing within periods, both treatment options must be represented in the relevant periods (positivity). These conditions do not guarantee that a cross-period comparison is unbiased; time-related differences still require consideration.

How Time Enters the Observed Difference

A short derivation makes the source of bias concrete. Suppose there are several periods, the treatment effect is constant across them, and \(m_t\) is the mean outcome that control participants would have in period \(t\). Let \(\tau\) be the treatment effect, so that the mean under treatment in period \(t\) is \(m_t+\tau\). Assume treatment groups are exchangeable within each period. The observed treatment mean averages the period-specific means using the period distribution among treated participants; the control mean uses the distribution among controls.

\begin{aligned} E[Y\mid A=1] - E[Y\mid A=0] &= \sum_t P(T=t\mid A=1)(m_t+\tau) - \sum_t P(T=t\mid A=0)m_t \\ &= \tau + \sum_t \left\{P(T=t\mid A=1)-P(T=t\mid A=0)\right\}m_t. \end{aligned}

The second term is a calendar-time composition term. It is zero if both groups have the same distribution across periods, or if the control mean is unchanged across periods. Otherwise, the observed difference need not equal the treatment effect \(\tau\). If effects differ by period, the treatment mean also averages different period-specific effects; the observed pooled contrast then depends on both period composition and effects within periods.

This decomposition relies on the stated assumptions, including a constant treatment effect and valid within-period comparisons. It is a way to explain the distortion, not a general recipe for correcting it. In particular, if a treatment is unavailable in a period, the data contain no within-period comparison of that treatment with control there.

Worked Example: A Nonconcurrent Control Makes Harm Look Beneficial

Hypothetical data. In an earlier period, 40 of 100 control participants experience an adverse event. In a later period, 20 of 100 control participants experience it. A new treatment is available only in the later period, where 30 of 100 treated participants experience the event. Suppose the contemporaneous treatment effect is a 10-percentage-point increase in event risk.

The event risk among treated participants is \(30/100=0.30\), or 30%. Comparing treated participants with the earlier controls gives:

\widehat{RD}_{\text{nonconcurrent}} = \frac{30}{100}-\frac{40}{100} =0.30-0.40=-0.10.

This cross-period comparison suggests a 10-percentage-point reduction in adverse-event risk. But the later-period control risk is \(20/100=0.20\). The contemporaneous contrast is:

\widehat{RD}_{\text{concurrent}} = \frac{30}{100}-\frac{20}{100} =0.30-0.20=0.10.

In the later period, treatment is associated with a 10-percentage-point increase in risk, consistent with the stated hypothetical treatment effect. The earlier controls had a higher event risk, so comparing them with later treated participants reverses the direction of the contrast. The data offer no treated-versus-control overlap in the earlier period, so this example cannot establish the treatment effect there.

Worked Example: Different Period Mixes Distort a Pooled Comparison

Hypothetical data. Consider a favorable binary outcome. In the early period, the control risk is 40%; in the late period, it is 20%. Assume treatment lowers the favorable-outcome risk by 5 percentage points in each period, giving treatment risks of 35% early and 15% late. Suppose 20% of treated participants are early and 80% are late, while 80% of controls are early and 20% are late.

The pooled treatment risk, using its actual period mix, is:

$$ 0.20(0.35)+0.80(0.15)=0.07+0.12=0.19. $$

The pooled control risk, using its different period mix, is:

$$ 0.80(0.40)+0.20(0.20)=0.32+0.04=0.36. $$

Thus the pooled risk difference is \(0.19-0.36=-0.17\), or a 17-percentage-point lower favorable-outcome risk among treated participants. Yet the treatment effect is \(-0.05\), or 5 percentage points, in each period. The additional 12-percentage-point difference in the pooled contrast arises from the unequal period mixes and the changing control risk. In this hypothetical case, the pooled comparison makes the reduction appear much larger than either period-specific contrast.

Worked Example: A Change in Care Can Explain a Cross-Period Contrast

Hypothetical data. Suppose the outcome is discharge by day 7, and a higher proportion is favorable. Before a discharge-support protocol is introduced, 40 of 100 control participants are discharged by day 7. After the protocol is introduced, 70 of 100 control participants are discharged by day 7. A new treatment is available only after the change, and 75 of 100 treated participants are discharged by day 7.

Comparing the new treatment with the earlier controls gives a risk difference of:

\frac{75}{100}-\frac{40}{100} =0.75-0.40=0.35.

That is a 35-percentage-point higher discharge proportion. In the later period, however, the control proportion is 70%, so the concurrent contrast is:

\frac{75}{100}-\frac{70}{100} =0.75-0.70=0.05.

The concurrent comparison is a 5-percentage-point difference. The change in discharge support is a plausible reason the earlier controls are not a suitable stand-in for controls in the later period. This example does not establish that the treatment caused the observed difference; a causal interpretation still depends on valid comparison within the later period and comparable outcome measurement.

Recognizing the Structure and Its Limits

The first question is whether the comparison is concurrent: were the treatment and control participants enrolled in overlapping periods when the treatment was available? If they were randomized contemporaneously, the primary comparison is protected by that randomization, subject to chance imbalance and the trial’s implementation. If the treatment group comes from a different period than the control group, calendar time is a plausible common cause to assess.

Next, consider the mechanisms that could connect time with outcome. A calendar date alone need not alter a patient’s prognosis. Changes associated with that date may matter: a new standard treatment, altered eligibility criteria, different referral pathways, changes in outcome assessment, or changes in follow-up. Ask whether those changes could affect the outcome independently of the study treatment, and whether they also coincide with treatment availability or assignment.

A practical limitation is lack of overlap. If an arm was unavailable before a certain date, there are no participants receiving that treatment in the earlier period. A causal diagram can reveal this problem, but naming calendar time does not create the missing comparison. Even when both arms are represented in multiple periods, the period-specific sample sizes may be small. Further, recording a date may not capture all changes that travel with time, such as an unrecorded shift in care.

Concurrent controls are therefore the main alternative to a cross-period control comparison when they are available: they share the same calendar context and, with valid randomization, support a direct treatment comparison. Nonconcurrent controls may add information, but a raw pooled comparison can mix treatment differences with time differences. Whether and how to use those observations requires additional assumptions and methods beyond identifying calendar time as a possible confounder.

Common Mistakes

Wrong: “The trial randomized treatment, so calendar time cannot confound the comparison.” Correction: Randomization supports a comparison among participants randomized at the same time under the trial’s allocation process. It does not make a treatment group from a later period exchangeable with controls from an earlier period.
Wrong: “The control rate changed, so the treatment effect changed.” Correction: A changing control outcome can reflect changes in prognosis or care. It does not, by itself, show that treatment effects differ over time.
Wrong: “Including the calendar period in the diagram removes the bias.” Correction: A diagram identifies a possible bias pathway; it does not remove it. A valid comparison still needs appropriate data, assumptions, and analysis.
Wrong: “A nonconcurrent control is always unusable.” Correction: It may contribute information under suitable assumptions, but it should not be treated as automatically equivalent to a concurrent control.

Key takeaway: When treatment availability and outcomes both vary over calendar time, a pooled arm-versus-control difference can combine a treatment contrast with a period contrast. Use the causal diagram to identify the pathway, check whether the comparison is concurrent, and state clearly what assumptions the data can support.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

A treatment arm opens in a later period, after a change in routine care. Name the possible backdoor path linking treatment and outcome.
In a period with concurrent randomization, what does randomization protect, and what does it not guarantee about a comparison with controls enrolled in a different period?
Control risks are 30% early and 10% late. Treated participants are 80% early and 20% late; controls are 20% early and 80% late. Assume the treatment has no effect in either period. Calculate the pooled risks and their difference.
Why can recording calendar period not solve a comparison problem when a treatment arm was unavailable in an earlier period?

Solutions. (1) The path is \(A \leftarrow T \rightarrow Y\), possibly with availability between time and assignment, such as \(T \rightarrow V \rightarrow A\). (2) Randomization supports exchangeability between assigned groups within the randomized period, apart from chance and implementation issues. It does not guarantee exchangeability between participants enrolled in different periods. (3) With no treatment effect, the treated risk is \(0.80(0.30)+0.20(0.10)=0.26\). The control risk is \(0.20(0.30)+0.80(0.10)=0.14\). The pooled difference is \(0.26-0.14=0.12\), or 12 percentage points, despite a zero effect in each period; the unequal period mixes create the difference. (4) There are no treated participants in that earlier period, so a within-period treatment comparison cannot be made there. The period label cannot supply the absent overlap.