Tutorials › Multi-Arm Multi-Stage (MAMS) Designs › Conditional Event-Risk Prediction From Intermediate Data

Time-to-Event MAMS With Intermediate Outcomes · Tutorial 316 of 499

Conditional Event-Risk Prediction From Intermediate Data

Learn to calculate response-conditioned event risks and combine them into an interpretable arm-level prediction at an interim landmark.

Advanced 11 min read

What You'll Learn

  • Define later event risk conditional on an intermediate response among patients event-free at a landmark.
  • Derive the arm-level prediction by averaging response-specific risks over the arm’s response distribution.
  • Incorporate events that occur before the intermediate assessment into risk by a fixed final horizon.
  • Calculate and interpret arm-level predicted risks from hypothetical interim data.
  • Identify the transport, measurement, and censoring assumptions behind the prediction.
  • Diagnose weak calibration, sparse response strata, and misleading pooled response-risk relationships.

Why Condition Event Risk on an Intermediate Response?

At an interim look, an intermediate outcome may be available before the final time-to-event endpoint is mature. A response recorded by a fixed landmark can provide information about the risk of a later event, but the response rate alone is not an event-risk prediction. To make that prediction, we need the event risk associated with each response category and the proportion of patients in each category.

This tutorial develops that calculation for a binary response and a fixed event horizon. The same logic extends to more response categories: calculate the later-event risk within each category, then average those risks using the category proportions in the arm. The calculation describes predicted risk under an explicit model; by itself, it does not establish that treatment caused a response or that changing the response would change event risk.

Define the Landmark Estimand

Let \(A=a\) denote assignment to arm \(a\), and let \(L\) be the time by which the intermediate response is assessed. Fix a later horizon \(\tau>L\). Define \(E=1\) if the event occurs by \(L\). Among participants with \(E=0\), let \(R\) denote the response measured at \(L\), with categories such as response and no response. Let \(Z=1\) indicate an event in the interval \((L,\tau]\). Thus, \(Z\) is defined for the landmark risk set: participants who have not had the event by \(L\).

Definition: For arm \(a\) and response category \(r\), the conditional later-event risk is \(q_{ar}=P(Z=1\mid A=a,E=0,R=r)\). The response-category proportion in the landmark risk set is \(\pi_{ar}=P(R=r\mid A=a,E=0)\). The arm-level predicted risk of an event by \(\tau\), for someone event-free at \(L\), is \(\sum_r \pi_{ar}q_{ar}\).

The landmark definition matters. A response assessed at \(L\) cannot be used to classify someone who had already experienced the event before \(L\). If the target is event risk by \(\tau\) from the original arm population, early events must also be included. Write \(e_a=P(E=1\mid A=a)\). The total predicted risk by \(\tau\) is then \(e_a+(1-e_a)\sum_r\pi_{ar}q_{ar}\). This separates observed early risk from predicted later risk in the event-free group.

Deriving the Arm-Level Prediction

Among participants event-free at \(L\), the response categories partition the risk set. The law of total probability therefore gives the later-event risk as a weighted average of the category-specific risks. For a binary response, denoted \(R=1\) for response and \(R=0\) for no response, the formula is:

$$ P(Z=1\mid A=a,E=0) = P(R=1\mid A=a,E=0)P(Z=1\mid A=a,E=0,R=1) + P(R=0\mid A=a,E=0)P(Z=1\mid A=a,E=0,R=0) = \pi_{a1}q_{a1}+(1-\pi_{a1})q_{a0}. $$

The weights are response proportions in the arm’s landmark risk set, not proportions in the original randomized group if some participants experienced an event before \(L\). Each product is the fraction of the landmark risk set expected to have a later event through that response category. Summing the products gives the predicted later-event proportion.

For risk by \(\tau\) in the original arm population, partition first by early event status and then, for those event-free at \(L\), by response category:

$$ P(\text{event by }\tau\mid A=a) = e_a+(1-e_a)\sum_r\pi_{ar}q_{ar}. $$

This expression assumes the response is defined and ascertained in the event-free group at \(L\), and that the \(q_{ar}\) values refer to the same later interval and event definition as the target prediction. If the target is only the conditional risk among patients event-free at \(L\), the early-event term is not part of that estimand.

Worked Example: Predicting Later Risk From Two Response Categories

Suppose a hypothetical arm has 100 participants who are event-free at a prespecified landmark. A response is recorded for all 100: 60 respond and 40 do not. From an applicable, sufficiently mature source of outcome data, the estimated later-event risks through the chosen horizon are 0.10 for responders and 0.30 for nonresponders. Assume these conditional risks apply to this arm’s landmark risk set.

The response proportions are \(\pi_{a1}=60/100=0.60\) and \(\pi_{a0}=40/100=0.40\). The predicted later-event risk is:

$$ (0.60)(0.10)+(0.40)(0.30)=0.06+0.12=0.18. $$

The predicted event risk from the landmark to the horizon is 18% in this arm’s event-free landmark population. Equivalently, the calculation predicts 18 later events per 100 such participants on average, not a guarantee that exactly 18 events will occur. It does not describe risk by the horizon in the full original arm unless early events are added.

Using Interim Arm Evidence

At an interim look, the response distribution \(\pi_{ar}\) may be observed in the current arm while the later-event risks \(q_{ar}\) come from mature participants in the same trial, earlier data, or a prespecified model. The calculation is useful only if the source of each component is clear. Substituting an observed response distribution into a conditional-risk model produces a plug-in prediction, not a direct observation of the arm’s final event risk.

If early events have also been observed in the original arm, the same predicted later risk can be combined with the early-event proportion. In the next example, the response-specific risks are assumed applicable in both arms. That assumption lets the calculation isolate how the different response distributions contribute to the predicted risks.

Worked Example: Comparing Two Arms at an Interim Look

Consider two hypothetical arms. In arm A, 8% of the original group experience the event by landmark \(L\). Of those event-free at \(L\), 70% respond. In arm B, the early-event proportion is 5%, and 45% of those event-free at \(L\) respond. Assume the later-event risks are 0.08 for responders and 0.28 for nonresponders in either arm, with the same event horizon and response definitions.

For arm A, the predicted later-event risk among those event-free at \(L\) is:

$$ (0.70)(0.08)+(0.30)(0.28)=0.056+0.084=0.140. $$

Including early events, the predicted risk by \(\tau\) from the original arm population is:

$$ 0.08+(1-0.08)(0.140)=0.08+(0.92)(0.140)=0.2088. $$

Thus, the plug-in prediction is 20.88% (about 20.9%) by \(\tau\) in arm A.

For arm B, the landmark risk is:

$$ (0.45)(0.08)+(0.55)(0.28)=0.036+0.154=0.190. $$

Including early events gives:

$$ 0.05+(1-0.05)(0.190)=0.05+(0.95)(0.190)=0.2305. $$

The corresponding prediction is 23.05% (about 23.1%). Under the stated common conditional-risk assumption, arm A’s predicted risk is 2.17 percentage points lower: \(0.2305-0.2088=0.0217\). This is an absolute predicted-risk contrast by the specified horizon. It is not, on its own, an estimate of a causal effect mediated by response; differences in the response distribution may have other explanations, and the shared conditional risks are an assumption.

Estimating the Conditional Risks and Checking the Calculation

A simple estimate of \(q_{ar}\) is the number of later events divided by the number of participants in response category \(r\) who have adequate follow-up through \(\tau\). For example, if 12 of 80 responders experience the event by \(\tau\), the crude estimate is \(12/80=0.15\). This ratio is appropriate only when the relevant outcomes are observed for everyone in the denominator or when censoring is handled by a method justified for the data. Treating participants censored before \(\tau\) as event-free generally understates risk.

A basic derivation shows why category-specific risks must be combined using their arm-specific proportions. In hypothetical mature data from one landmark risk set, suppose 12 of 80 responders and 24 of 120 nonresponders experience an event by \(\tau\). Then \(q_{a1}=12/80=0.15\), \(q_{a0}=24/120=0.20\), and \(\pi_{a1}=80/200=0.40\). The weighted prediction is:

$$ (0.40)(0.15)+(0.60)(0.20)=0.06+0.12=0.18. $$

There are 36 events among 200 participants, so the directly observed risk is \(36/200=0.18\), matching the weighted calculation. In general, if every response category is represented and its risk is estimated in the same complete dataset, the weighted average reproduces the overall risk. When category-specific risks are transported from another source, however, the weighted result is a prediction and need not equal the current arm’s observed risk.

Useful checks include comparing predicted with observed risks in mature data, separately by arm and response category; checking that every category used in the prediction has data; and verifying that the landmark, response definition, event definition, and follow-up horizon match across sources. Large differences in calibration across arms or time periods warn that a shared conditional-risk mapping may not transport. Very few participants or events in a category make its estimated risk unstable, even when the overall event count appears adequate.

1
Specify the target.
State the landmark, the later event interval, and whether the target is risk among those event-free at \(L\) or risk from the original arm population.
2
Obtain response proportions.
Calculate each arm’s response-category proportions among participants event-free at \(L\), using the same response definition as the conditional-risk data.
3
Obtain conditional risks.
Estimate or specify the later-event risk for each arm and response category, addressing incomplete follow-up and explaining why these risks apply to the interim population.
4
Average and interpret.
Weight the conditional risks by the observed response proportions. Add early-event risk only when predicting from the original arm population.

Assumptions, Alternatives, and Limitations

The prediction relies on several substantive and data-quality assumptions. First, response and event status must be defined consistently, and response must be measured by the stated landmark for the people included in the landmark risk set. Second, the conditional risks must apply to the current arm and time period. If conditional risks are pooled across arms, equality across arms is an extra assumption, not a consequence of observing a response. If risks come from earlier data, changes in population, care, or follow-up can undermine transportability.

Third, censoring before \(\tau\) must not be treated as absence of an event. Complete follow-up through \(\tau\) is one simple setting; otherwise, risk estimation needs a method that accounts for censoring under appropriate assumptions. Fourth, each response category used in the calculation needs enough data to estimate its risk. A response category absent from the interim arm cannot support a data-based within-category estimate for that arm without additional modeling assumptions.

A common mistake is to argue, “The response rate is higher, so the final event risk must be lower.” That conclusion does not follow from response rates alone. It requires information about later-event risk by response category and a defensible way to apply those risks to the arm. Even when a response is prognostic, its association with later events does not show that treatment’s effect on response explains treatment’s effect on the final endpoint. Predictive use is not proof of surrogacy or a causal mechanism.

This calculation is transparent and easy to audit, but it reduces the intermediate information to response categories and uses specified conditional risks. Direct analysis of sufficiently mature final outcomes is preferable when those outcomes answer the target question with adequate precision. More flexible models can incorporate covariates or time-to-event information, but they bring additional modeling assumptions and diagnostics. The introductory material on delayed intermediate outcomes distinguishes availability from maturity; here, even an observed response does not eliminate the need to assess whether later-event data are mature enough to estimate the conditional risks. The calculation also does not quantify the uncertainty in the plug-in prediction: uncertainty in response proportions, conditional risks, and their dependence should be considered before treating small arm differences as persuasive interim evidence.

Key takeaway: Conditional event-risk prediction combines an arm’s response distribution with later-event risks conditional on response. The result is interpretable only when the landmark population, time horizon, early events, and assumptions behind the conditional risks are made explicit.

Check Your Understanding

Use the definitions and calculations in the tutorial. Answers follow each question.

  1. An arm has 50% responders among participants event-free at \(L\). Later-event risk is 0.12 for responders and 0.32 for nonresponders. What is the predicted risk after \(L\)?
    Solution: \(0.50(0.12)+0.50(0.32)=0.06+0.16=0.22\), or 22%.
  2. In the preceding setting, 6% of the original arm population experienced the event by \(L\). What is the predicted risk by \(\tau\) from the original population?
    Solution: \(0.06+(1-0.06)(0.22)=0.06+0.2068=0.2668\), or 26.68%.
  3. Why can’t a higher response proportion alone establish a lower predicted event risk?
    Solution: The prediction also depends on the later-event risks in each response category. Without those risks and an assumption that they apply to the arm, the response proportion does not determine the event risk.
  4. What is wrong with counting a participant censored before \(\tau\) as event-free through \(\tau\)?
    Solution: The participant’s event status through the target horizon is unknown. Coding that participant as event-free can bias the estimated conditional risk downward; censoring must be handled under suitable assumptions.
  5. If conditional risks are pooled across treatment arms, what additional assumption is being made?
    Solution: The pooled risks are assumed to apply across arms, despite possible differences in conditional event risk by arm. This equality must be assessed or justified; it does not follow from pooling the data.
Clinical Biostats