Worked Example With Ordinal And Delayed Outcomes

Why Timing and Outcome Grades Matter

A dose may look promising early in a Phase 1/2 trial even though the evidence is incomplete. Toxicity may be graded and available soon after treatment, while efficacy is recorded later and can take several follow-up visits to classify. Treating an efficacy outcome that is still pending as a failure distorts the evidence; ignoring all participants without mature efficacy can also waste information, especially when their toxicity has already been observed.

This tutorial works through a transparent Bayesian updating rule for a hypothetical dose-finding study. Efficacy and toxicity are both ordered outcomes. The example makes the simplifying assumption that their distributions are independent within dose, conditional on dose. That assumption makes the calculations easy to audit; it is not a general claim that efficacy and toxicity are unrelated.

Model, Assumptions, and Decision Rule

For each dose and endpoint, assign an independent \(\operatorname{Dirichlet}(1,1,1)\) prior to the three category probabilities. This is a uniform prior over the probability simplex. If the observed category counts are \(n_0,n_1,n_2\), the posterior is \(\operatorname{Dirichlet}(1+n_0,1+n_1,1+n_2)\). Its posterior mean for category \(k\) is the corresponding parameter divided by the sum of all three parameters.

This update assumes outcomes for different participants are conditionally independent given dose, category definitions are fixed, and each participant contributes one classified outcome per endpoint. It also assumes the efficacy outcomes that are not yet available are administratively pending, rather than selectively missing because of their unobserved values. Pending efficacy contributes no efficacy category count yet. Toxicity already observed for that participant can still update the toxicity distribution.

To make the example’s decision rule explicit, define the efficacy score as \(0\), \(0.5\), or \(1\) for efficacy categories \(0\), \(1\), or \(2\), respectively. The posterior mean score is the posterior expected value of this prespecified score; it is not a claim that the categories are equally spaced clinically. A dose is eligible if its posterior mean probability of severe toxicity, \(T=2\), is at most \(0.25\). Among eligible doses, recommend the dose with the greatest posterior mean efficacy score. If scores tie, choose the dose with lower posterior mean severe-toxicity probability, then the lower dose if still tied. A real protocol should justify and prespecify its thresholds and score.

The formula follows directly from the Dirichlet posterior. The efficacy score is a weighted sum of posterior category probabilities; substituting their posterior means gives the numerator shown. For toxicity, the severe-category probability has posterior mean equal to its Dirichlet parameter divided by the parameter total. Each observed category adds one to its matching parameter, while the three prior parameters sum to three. Thus pending efficacy is not inserted as a zero: it does not appear among the efficacy counts at all.

Updating as Efficacy Outcomes Mature

The posterior update is sequential: when an efficacy classification becomes available, add one to that category’s count and recalculate. Previously observed toxicity counts remain in the toxicity analysis. They are not replaced by a complete-case subset made up only of participants whose efficacy outcomes have matured.

Early and Final Recommendations

Suppose all eight efficacy outcomes per dose are eventually classified. For Dose 1, the three additional outcomes are \(0,1,2\); for Dose 2 they are \(0,0,1\); and for Dose 3 they are \(2,2,2\). Toxicity counts do not change because all eight toxicity outcomes were already observed at the interim review.

Diagnostics, Alternatives, and Limitations

The recommendation is meaningful only if the data timing and model assumptions are credible. Review the number of participants with mature efficacy by dose and enrollment time. If follow-up completion differs substantially by dose or is associated with clinical status, a simple “pending contributes no efficacy information yet” analysis may be misleading. Check that toxicity grades were classified consistently and that the severe-toxicity grouping matches the protocol’s clinical concern. Posterior predictive checks and model-fit diagnostics can reveal discrepancies between observed category frequencies and model implications, but small samples will limit how much these checks can establish.

The marginal Dirichlet approach is simple and transparent, but it does not borrow information across doses, and small samples can leave posterior summaries sensitive to prior choices. Its independence assumption also cannot represent an association such as severe toxicity being more common among participants with strong efficacy responses. A joint model for the full efficacy-toxicity table can represent that association, but updating with partially observed pairs requires integrating over the missing component. An ordinal regression model is another alternative: it can share information across ordered categories and doses, but adds modeling assumptions such as a particular relationship across category thresholds. These approaches answer related but not identical questions.

A common mistake is to label every pending efficacy outcome as \(E=0\), reasoning that an outcome not yet observed is “not a response.” That confuses absence of information with an observed category and artificially lowers the efficacy score. The correction is to update efficacy only with outcomes classified at the review; if there is evidence that delay or non-observation is informative, the analysis needs an explicit model for that process. A second mistake is to treat the posterior mean threshold as a universal safety guarantee. It is only the decision rule chosen for this example: uncertainty, clinical consequences, and trial objectives may call for a different prespecified rule.