Why the Ordering Must Be Explicit
When a trial has both deaths and a clinical outcome, a single summary requires a rule for comparing unlike experiences. A participant who dies cannot simply be assigned a routine value on a symptom scale measured among survivors. Instead, the analysis can define an ordered composite: a prespecified hierarchy that says which outcome is better when two participants differ in survival status, survival time, or clinical outcome.
The hierarchy is part of the treatment question, not a technical detail added after observing the data. A rule that places every participant alive at a fixed horizon above every participant who died by that horizon gives survival status priority. A rule that also compares time to death among participants who died makes a further choice. These decisions determine which participant is judged to have the more favorable outcome in every comparison, and therefore can affect the treatment contrast.
Specify the Hierarchy Before Comparing Groups
A useful specification begins by fixing the follow-up horizon and defining the clinical endpoint, its measurement time, and its favorable direction. For example, a larger improvement score might be better, while a larger symptom-severity score might be worse. The direction must be stated rather than assumed.
One common hierarchy is survival status first: being alive at the horizon is better than dying before it. Among participants who died, longer survival time is better. Among participants alive at the horizon, the clinical endpoint determines the ordering. Under this rule, the clinical endpoint does not reverse the comparison between a participant who died and one who survived to the horizon.
Other hierarchies are possible. Investigators might treat all deaths as equally poor, or compare time to death among deceased participants, as in the rule above. They might also prioritize a specified clinical event before death timing, if that ordering is clinically justified and can be applied consistently. There is no universally correct ordering for every trial. The choice should reflect the clinical question, be justified in advance, and be understandable to the intended audience.
Worked Example: Applying a Survival-First Hierarchy
Suppose a hypothetical trial follows participants for 12 months. The prespecified rule is that anyone alive at 12 months ranks above anyone who died before 12 months. Among those who died, more months lived ranks higher. Among those alive at 12 months, greater improvement on a clinical scale ranks higher. Consider three hypothetical participants: A died at month 8; B died at month 10; C was alive at month 12 with a 3-point improvement.
Compare A and B. Both died before the horizon, so compare time to death. A ranks lower because \(8<10\): A lived fewer months. Thus B, who died at month 10, ranks higher.
Compare B and C. C was alive at the horizon and B died before it. Survival status has priority, so C ranks higher, regardless of B's 10 months lived or the size of C's clinical improvement.
Compare two survivors. If another participant survived to month 12 with a 4-point improvement, that participant ranks higher than C because \(4>3\). These comparisons follow from the stated ordering; no numerical distance between ranks is needed.
A Pairwise Treatment Contrast
A rank-based treatment contrast can be defined by comparing every treated participant with every control participant. For each pair, record a treatment win if the treated participant ranks higher, a loss if the treated participant ranks lower, and a tie if the outcomes have the same rank under the rule. Let \(P_W\), \(P_L\), and \(P_T\) denote the probabilities of a win, a loss, and a tie when one participant is drawn from each treatment group. These probabilities refer to the defined population and the specified outcome hierarchy.
The tie-adjusted win probability is 0.5 when wins and losses balance after ties count as half. Values above 0.5 favor treatment under the chosen hierarchy; values below 0.5 favor control. The net win probability is zero at balance, positive when wins are more frequent than losses, and negative when losses are more frequent. Neither quantity is a mean change in the clinical endpoint.
In a sample with \(n_T\) treated participants and \(n_C\) control participants, compare all \(n_Tn_C\) cross-group pairs. If \(W\), \(L\), and \(T\) are the resulting counts of wins, losses, and ties, then the empirical tie-adjusted win probability is \((W+\tfrac{1}{2}T)/(n_Tn_C)\), and the empirical net win probability is \((W-L)/(n_Tn_C)\). The following identity explains why the two summaries are linked.
To verify it, use \(T=n_Tn_C-W-L\). Then \(W+\tfrac{1}{2}T=\tfrac{1}{2}(W-L+n_Tn_C)\). Dividing by \(n_Tn_C\) and subtracting \(\tfrac{1}{2}\) gives the displayed identity. This is an arithmetic relationship between the summaries, not a claim that either one is free of sampling uncertainty.
Worked Example: Counting Wins, Losses, and Ties
In a hypothetical trial, three treated participants and three control participants are assessed at 12 months using the survival-first hierarchy. Among deceased participants, longer survival ranks higher; among survivors, greater improvement ranks higher. The hypothetical treated outcomes are: T1 died at month 9, T2 survived with a 4-point improvement, and T3 survived with a 1-point improvement. The control outcomes are: C1 died at month 6, C2 survived with a 2-point improvement, and C3 survived with a 1-point improvement.
There are \(3\times3=9\) treated-control pairs. T1 beats C1 because T1 lived longer, but loses to C2 and C3 because they survived to the horizon. That gives T1 one win and two losses. T2 beats all three controls: T2 survived and has greater improvement than either surviving control. That gives T2 three wins. T3 beats deceased C1, loses to C2 because \(1<2\), and ties C3 because both survived with a 1-point improvement. That gives T3 one win, one loss, and one tie.
Combining the comparisons gives \(W=5\), \(L=3\), and \(T=1\), accounting for all nine pairs. The estimated tie-adjusted win probability is
Thus, in these hypothetical data, a randomly selected treated participant has an estimated 61.1% chance of ranking better than a randomly selected control participant when a tie counts as half a win. The estimated net win probability is \((5-3)/9\approx0.2222\). These are sample summaries, not proof of a treatment effect; uncertainty should be assessed using an appropriate inferential procedure.
Ordering Choices Can Change the Contrast
A useful sensitivity analysis asks whether a reasonable alternative hierarchy changes who wins particular comparisons and, consequently, the treatment contrast. For example, comparing survival time among people who died distinguishes two deaths that a rule treating all deaths as tied would consider equal. An ordering can therefore change both the number of ties and the balance of wins and losses.
Worked Example: Comparing Two Rules for Deaths
Consider four hypothetical participants at a 12-month horizon. In the treatment group, one participant died at month 12 and one survived with a 1-point improvement. In the control group, one participant died at month 6 and one survived with a 2-point improvement. In both rules below, any survivor ranks above any participant who died. Among survivors, greater improvement ranks higher. The rules differ only in how they compare deaths.
Rule A: compare survival time among deaths. The treated death at month 12 ranks above the control death at month 6. The treated survivor loses to the control survivor because \(1<2\). The treated death loses to the control survivor, and the treated survivor beats the control death. Thus there are two wins, two losses, and no ties among four pairs. The tie-adjusted win probability is \(2/4=0.5000\).
Rule B: treat all deaths as tied. The death-versus-death comparison is now a tie instead of a treatment win. The two cross-status comparisons and the survivor-versus-survivor comparison are unchanged. There is one win, two losses, and one tie, so the tie-adjusted win probability is \((1+\tfrac{1}{2})/4=0.3750\).
The difference is not a calculation error: the rules encode different judgments about whether longer survival before death should count as a better outcome. Reporting only the numerical contrast without its hierarchy would hide that judgment.
Ranks Are Not the Same as Numerical Scores
An ordered outcome specifies which result is better, but it does not automatically specify numerical distances between results. If investigators assign scores to categories and compare group means, the score differences depend on the chosen numbers. A pairwise rank summary, by contrast, depends on the ordering and tie rule, not on arbitrary distances between rank categories.
Worked Example: The Effect of Score Spacing
Suppose, hypothetically, that the hierarchy assigns score 0 to death, 1 to alive with worsening, 2 to alive with no change, and 3 to alive with improvement. Two treated participants have scores 0 and 3; two control participants both have score 2. The treatment mean score is \((0+3)/2=1.5\), and the control mean score is \((2+2)/2=2.0\). The mean score difference, treatment minus control, is \(1.5-2.0=-0.5\) points.
For pairwise comparisons, the treated participant who died loses to both controls, while the treated participant who improved beats both. The result is two wins and two losses, so the tie-adjusted win probability is \(2/4=0.5000\). These summaries answer different questions: the mean difference uses the numerical score distances, while the pairwise summary uses the ordering alone. A negative mean score difference does not contradict a balanced pairwise summary.
Changing the score spacing, while keeping the same ordering, can change the mean difference. Therefore, a mean contrast is meaningful only when the scores and their distances have a defensible interpretation. Do not describe a rank-based probability as a mean score difference, or the other way around.
Assumptions, Diagnostics, and Limitations
The ordering can always be stated, but its treatment interpretation depends on the trial and the data. In a randomized trial with outcomes observed as defined, the group contrast can be interpreted as comparing outcomes under assignment to the two treatment strategies in the trial population, subject to the trial’s design and conduct. The hierarchy itself does not establish causation; randomization and appropriate analysis support causal interpretation of the assignment contrast.
Before calculating a contrast, check that the horizon and outcome measurements are defined consistently across groups, that the clinical endpoint direction is correct, and that every pair can be ranked by the rule. Review how many participants died, how many survived, and how many comparisons are ties. A high tie frequency may be appropriate, but it can also indicate that the endpoint is too coarsely categorized to distinguish outcomes. Examine whether the conclusions are sensitive to defensible alternatives, such as tying all deaths or ordering them by survival time.
A ranking rule cannot repair incomplete or ambiguously timed outcome data. If death status by the horizon is unknown, or the endpoint is missing for a survivor, that participant cannot automatically be placed in the hierarchy. The analysis must address the missing information using a justified method; silently assigning a rank changes the outcome definition. Follow-up beyond the horizon should not be used for one group but not the other without an explicit estimand and analysis plan.
The main alternative summaries answer different questions. A conventional composite can define a binary event such as death or clinical worsening, but discards distinctions within the event category unless they are built into its definition. A clinical endpoint analyzed among survivors focuses on a selected subset and does not by itself summarize the combined survival and clinical outcome. A numerical score can produce a mean contrast, but requires meaningful score distances. The ranked composite makes the priority ordering visible, while leaving the clinical judgment in that ordering open to scrutiny.
- Failure mode: declaring all survivors better than deaths without specifying the horizon. Correction: define exactly when survival is assessed.
- Failure mode: reversing the favorable direction of the endpoint, such as treating greater symptom severity as improvement. Correction: state the scale direction and check every comparison against it.
- Failure mode: changing the hierarchy after seeing which rule favors treatment. Correction: prespecify the primary ordering and label alternatives as sensitivity analyses.
- Failure mode: interpreting a win probability as the probability that treatment benefits a particular individual. Correction: it summarizes comparisons between randomly selected participants from the two groups, not an individual treatment effect.
Check Your Understanding
Answer each question using the definitions and rules above. Solutions follow the questions.
- Under a 12-month survival-first hierarchy, participant A died at month 8 and participant B died at month 10. Who ranks higher, and why?
- In a sample with 12 treated-control pairs, there are 6 wins, 4 losses, and 2 ties. Calculate the tie-adjusted win probability and net win probability.
- Why can a mean score difference change when the numerical spacing between ordered categories changes, even though a pairwise rank comparison does not?
- What additional specification is needed before comparing deaths and clinical endpoint values in a ranked composite?
Solutions. (1) B ranks higher because both died before the horizon and \(10>8\), so B lived longer. (2) The tie-adjusted win probability is \((6+\tfrac{1}{2}(2))/12=7/12\approx0.5833\). The net win probability is \((6-4)/12=2/12\approx0.1667\), which also equals \(2(7/12)-1\). (3) A mean uses the numerical distances between scores, so changing those distances changes the average; pairwise ranking uses only which outcome is higher or whether outcomes tie. (4) Specify the horizon and the hierarchy that determines whether survival status takes priority, how deaths are compared, and how the clinical endpoint is oriented and used among survivors.