Population-Level Summaries of Treatment Effects

Why the Summary Is Part of the Clinical Question

An endpoint tells us what is measured; a population-level summary tells us which feature of its distribution is being compared. For example, a treatment may change the average symptom score, the chance of a clinically important event, or the time until an event. These are not interchangeable descriptions of one universal treatment effect. They answer different clinical questions.

The earlier material on the five attributes of a clinical trial estimand established that the population, treatment conditions, endpoint, and population-level summary work together to define the target question. Here, the focus is the summary attribute. The endpoint and target population are held fixed when comparing summaries, so that differences in interpretation arise from the summary itself.

For notation, let \(Y^a\) denote the outcome that would be observed under treatment condition \(a\) in the target population. The distribution of \(Y^a\) is the population distribution under that condition. A treatment contrast compares a summary of the distribution under one condition with the corresponding summary under another. This notation describes the target question; it does not, by itself, say how the quantity will be estimated from trial data.

The quantities are meaningful only when the underlying definitions are sufficiently clear: who is in the target population, what the treatment conditions are, how the endpoint is measured, and when it is measured. For risks and survival summaries, the event definition and time horizon also matter. The examples below assume those features are well specified and compare treatment conditions in the same target population.

Means and Medians for Quantitative Outcomes

For a quantitative endpoint, the mean summarizes the population average. If \(Y^a\) has a finite mean, the mean contrast for treatment \(a=1\) versus \(a=0\) is \(E(Y^1)-E(Y^0)\). It retains the endpoint’s units: a mean difference in a blood-pressure change measured in mmHg is measured in mmHg.

The median is the 50th percentile of a population distribution. More precisely, a median \(m_a\) satisfies \(P(Y^a \leq m_a)\geq 0.5\) and \(P(Y^a \geq m_a)\geq 0.5\). A median contrast is \(m_1-m_0\), in the endpoint’s units. For distributions with more than one median, the convention for selecting or reporting a median should be specified.

The mean uses the magnitudes of all outcomes and can be strongly affected by extreme values. The median describes the distribution’s center by its midpoint, but it does not generally describe the average amount of change. When the distribution is skewed or has unusual values, these summaries can differ substantially.

Risks, Risk Ratios, and Odds Ratios

For a binary endpoint defined over a specified period, the risk under condition \(a\) is \(p_a=P(Y^a=1)\), where \(Y^a=1\) denotes that the event occurs during that period. Three common summaries compare the risks: the risk difference \(p_1-p_0\), the risk ratio \(p_1/p_0\), and the odds ratio.

A risk difference is an absolute contrast, often reported in percentage points. A risk ratio is a relative contrast: a value below 1 means the event risk is lower under condition 1 than under condition 0, provided the event is coded and the comparison direction are stated. An odds ratio compares event odds, not event risks. Odds are \(p/(1-p)\), so an odds ratio of 0.5 means the event odds are half as large; it does not generally mean the event risk is half as large.

The distinction between the relative measures follows directly from their definitions:

When the event risks are small, \(1-p_0\) and \(1-p_1\) are near 1, so the odds ratio can be close to the risk ratio. This approximation is not a definition and may be poor for common events. The risk difference can also be more directly useful for decisions about absolute benefit or harm, while a relative measure may help compare proportional changes across settings with different baseline risks. Neither scale is universally best: the clinical question determines which contrast is most informative.

Time-to-Event Summaries

When the endpoint is time from a defined origin to a specified event, a population-level summary can be expressed through the survival function \(S_a(t)=P(T^a>t)\), the probability of remaining event-free beyond time \(t\) under condition \(a\). Its complement \(1-S_a(t)\) is the cumulative event risk by time \(t\), assuming the event status and time origin are defined. A survival contrast at a fixed time is \(S_1(t)-S_0(t)\); the corresponding risk difference is \([1-S_1(t)]-[1-S_0(t)]\), which has the opposite sign.

Another summary is median event time, the time by which at least half the population has experienced the event, when such a time is reached and is well defined. The restricted mean survival time up to a prespecified horizon \(\tau\) is the area under the survival curve from 0 to \(\tau\):

It represents the expected event-free time accumulated during the interval from 0 to \(\tau\), and is measured in units of time. A difference in restricted mean survival time compares those expected event-free durations up to the chosen horizon. The horizon must be stated because changing it can change the contrast.

A single fixed-time survival probability can be clinically transparent but depends on the selected time point. A median summarizes the point where survival reaches 0.5, but may not exist within the observed follow-up if fewer than half experience the event. Restricted mean survival time uses the curve across a stated interval but depends on the horizon. Any chosen summary requires adequate information about the relevant part of the population distribution.

Choosing and Checking a Summary

The appropriate summary follows from the clinical question, not from whichever quantity is easiest to calculate. Before interpreting a contrast, confirm that it specifies the target population, treatment conditions, endpoint definition, follow-up time or horizon where applicable, and direction of comparison. Then check that the measure matches the desired interpretation: a mean difference in clinical units, a proportion-point risk difference, a relative risk, an odds contrast, or an event-free-time summary.

Several failure modes deserve attention. A mean may be difficult to interpret when extreme outcomes dominate it; a median may hide clinically important differences away from the center. Relative measures can sound dramatic while corresponding to a small absolute change, or can obscure an important absolute difference when baseline risks are high. An odds ratio can be mistakenly described as a risk ratio. A time-to-event contrast may conceal different curve patterns if reduced to one time point or one median. When possible, presenting a primary, clinically justified summary alongside complementary summaries can clarify rather than replace the target question.

These concerns are not resolved merely by a Bayesian analysis. Bayesian methods can represent uncertainty about a chosen population-level quantity, but the data model and prior do not determine which clinical summary should define the question. The estimand’s summary must be selected and interpreted on clinical grounds; uncertainty about its value is a separate issue.

Common Mistakes and Diagnostics

• Calling every treatment contrast “the treatment effect.” State the actual summary and scale. A mean difference and a median difference need not agree in magnitude.
• Interpreting an odds ratio as a risk ratio. Convert the risks to odds using \(p/(1-p)\), and use “odds” in the interpretation.
• Reporting a relative effect without the event risks. Include absolute risks or a risk difference when absolute clinical impact matters.
• Reporting a survival contrast without a time point or horizon. State the time for a survival probability and the horizon for restricted mean survival time.
• Assuming agreement among summaries is guaranteed. Examine the outcome distribution or survival curves; different summaries can emphasize different parts of the population.

A practical diagnostic is to ask whether the reported measure can be restated in a sentence that includes the population, treatment comparison, endpoint, scale, and time. If the sentence is ambiguous—for example, “risk fell by 40%” without specifying relative or absolute change—the summary has not been communicated precisely.