Safety Evidence in Benefit-Risk Review
A benefit-risk assessment is not a vote in which one efficacy result cancels one adverse event. Reviewers need to understand what benefits and harms were measured, which patients contributed data, how long they were observed, and how uncertain each estimate remains. A Bayesian analysis can put uncertainty about benefits and harms on a common probabilistic footing, but it cannot make weak data informative or decide which outcomes matter most without an explicit clinical rationale.
The relevant safety quantity depends on the question. For a trial with comparable fixed follow-up, the proportion of patients experiencing an event may be useful. If follow-up varies, event counts alone are not comparable: time at risk or time-to-event information may be needed. Neither a proportion nor a rate, by itself, describes severity, reversibility, timing, or clinical importance. Those features belong in the interpretation and in any explicitly justified benefit-risk framework.
Make the Safety Estimand and Data Conditions Explicit
Before fitting a model, specify the event definition, analysis population, treatment comparison, time window, and measure of interest. “Serious adverse events” may be a broad category, while a specific event of special interest may have a narrower definition. If the question is the probability of at least one event by a fixed time, state that time horizon. If the question concerns events per unit of exposure, define the exposure unit and whether recurrent events count.
For a simple comparison of event proportions, assume that each patient contributes one binary outcome over a comparable, prespecified follow-up period; outcomes are correctly classified; and patients’ outcomes are independent conditional on the model. With independent Beta priors and binomial sampling in the two randomized arms, the arm-specific posterior distributions are independent. This simple model does not account for varying follow-up, recurrent events, clustering, or informative loss to follow-up. Those features require suitable models and explicit assumptions.
For event rates, a simple alternative assumes counts follow a Poisson model conditional on exposure, with a constant event rate over the observation period. It also assumes that exposure is measured appropriately and that the rate model is reasonable for the event process. A constant-rate model may be inadequate when risk changes over time, events recur in dependent clusters, or censoring is informative. When time to first event and censoring matter, a time-to-event model may be more appropriate than a proportion or a crude rate.
Worked Example: Comparing Fixed-Follow-Up Event Risks
Hypothetical data. In a randomized trial, 18 of 300 patients assigned to treatment and 10 of 300 assigned to comparator have a specified adverse event during the same 12-week follow-up. Assume complete, comparable ascertainment; one binary outcome per patient; and independent outcomes between patients. Use independent Beta(1,1) priors for the two event probabilities, \(p_T\) and \(p_C\).
With a Beta(\(a,b\)) prior and \(y\) events among \(n\) patients, the binomial likelihood gives a Beta(\(a+y,b+n-y\)) posterior. Thus:
The posterior means are \(19/302=0.06291\) and \(11/302=0.03642\), respectively. The posterior mean difference in 12-week event risk is therefore \(8/302=0.02649\), or 2.649 percentage points higher in the treatment arm. This is an estimate of the risk difference, not a conclusion that treatment caused a harmful increase with certainty.
For a transparent approximate assessment of whether the excess exceeds 2 percentage points, the variance of a Beta(\(a,b\)) random variable is \(ab/[(a+b)^2(a+b+1)]\). The posterior variances are:
Because the arm posteriors are independent, the variance of their difference is the sum, approximately \(0.0003104\), and its posterior standard deviation is \(\sqrt{0.0003104}\approx0.01762\). A normal approximation to the posterior difference gives:
Under this approximation, the posterior probability that the treatment-arm risk exceeds the comparator risk by more than 2 percentage points is about 64.4%. The normal approximation is a convenience, not an exact posterior calculation; direct simulation from the two Beta posteriors can assess this probability without that approximation. Whether a 2-point excess is clinically important depends on event severity and the benefits at stake.
Exposure and Follow-Up Change the Interpretation
Equal patient counts do not guarantee equal opportunity to observe an event. A shorter follow-up period, early discontinuation, or differences in time at risk can make an event proportion misleading. An exposure-adjusted rate uses events per unit of person-time, but it answers a rate question and relies on assumptions about the event process. It is not automatically interchangeable with the probability of an event by a stated time.
Worked Example: Sparse Events and Person-Time
Hypothetical data. Suppose a serious event occurs 4 times during 4,000 person-years in the treatment arm and once during 3,800 person-years in the comparator arm. For illustration, let \(Y\) be the event count, \(E\) the exposure in units of 1,000 person-years, and \(\lambda\) the event rate per 1,000 person-years. Assume \(Y\mid\lambda\sim Poisson(E\lambda)\), a constant rate, and independent Gamma(0.5, 0.5) priors for the two arm-specific rates, using the shape-rate parameterization.
The observed exposure values are \(E_T=4\) and \(E_C=3.8\). The posterior for a rate with Gamma(\(a,b\)) prior is Gamma(\(a+y,b+E\)). Therefore:
Their posterior mean rates are \(4.5/4.5=1.000\) and \(1.5/4.3\approx0.3488\) events per 1,000 person-years. These estimates should not be read as stable evidence that the treatment rate is nearly three times as high: only five events were observed, and the posterior uncertainty is substantial. The prior also contributes materially, especially for the arm with one event, so its role should be examined in sensitivity analyses.
One useful calculation is the posterior predictive chance of at least one event in a further 1,000 person-years, assuming the same rate model and a comparable population. For a Gamma(\(a,b\)) posterior mixed with a Poisson count over exposure \(E_{new}\), the predictive probability of zero events is \([b/(b+E_{new})]^a\). Hence the treatment-arm predictive probability of one or more events is:
For the comparator it is \(1-(4.3/5.3)^{1.5}\approx0.269\). These are predictions for future event counts under the stated model, not probabilities that an individual patient will experience an event by a fixed follow-up time. They also do not establish that the two arms have comparable future patients or exposure patterns.
Integrating Benefit and Harm Without Hiding Values
Safety evidence is interpreted alongside benefit evidence, but the analysis should preserve the outcome-specific estimates before any summary is formed. A numerical summary can be useful when its scale and weights have a defensible clinical basis. The weights are not supplied by Bayes’ theorem: they express judgments about the relative importance of outcomes. If a review uses a utility, it should explain its units, how weights were chosen, and whether the utility captures the relevant clinical trade-offs.
Worked Example: An Explicit Additive Utility
Hypothetical data. In the same-sized randomized groups, a clinically defined response occurs in 150 of 300 treatment patients and 120 of 300 comparator patients. Use independent Beta(1,1) priors for response probabilities, as well as the safety data from the first example. For illustration only, define each patient’s utility as \(U=B-5H\), where \(B=1\) for response and \(0\) otherwise, and \(H=1\) for the specified adverse event and \(0\) otherwise. This assigns one response a value of +1 and one adverse event a value of −5.
The response posteriors are Beta(151,151) and Beta(121,181), with posterior means \(151/302=0.5000\) and \(121/302\approx0.4007\). The posterior mean response difference is \(30/302\approx0.09934\), or 9.934 percentage points. From the safety example, the posterior mean adverse-event difference is \(8/302\approx0.02649\).
By linearity of expectation, the posterior mean incremental utility is the response difference minus five times the adverse-event difference:
On this illustrative utility scale, the posterior mean incremental utility is −0.03311 utility units per patient. This does not establish that treatment has negative utility for every patient or that the weighting is clinically appropriate. The calculation uses an additive utility, so it does not require response and harm to be independent to compute this expectation. However, if their joint combinations change clinical value—for example, if a response offsets a particular harm for some patients but not others—the additive utility may be inadequate. The weights also require clinical justification and sensitivity analysis.
Justification, Diagnostics, and Alternatives
The basic updating calculations follow directly from the likelihood. For a binomial observation \(y\) in \(n\) patients, the likelihood is proportional to \(p^y(1-p)^{n-y}\). Multiplying by a Beta(\(a,b\)) prior density, proportional to \(p^{a-1}(1-p)^{b-1}\), gives a posterior density proportional to \(p^{a+y-1}(1-p)^{b+n-y-1}\). This is the kernel of a Beta(\(a+y,b+n-y\)) distribution. The conjugate calculation is simple, but its simplicity does not remove the need to justify the event definition, sampling model, prior, and follow-up assumptions.
Useful review checks include verifying event counts against source data and adjudication rules; tabulating patients, person-time, follow-up duration, and discontinuation by arm; and checking whether the endpoint and analysis population match the clinical question. Examine posterior distributions, not only posterior means, and test whether conclusions change under plausible alternative priors, event definitions, or missing-data assumptions. Prior-data conflict and sensitivity analyses are especially relevant when events are sparse. Posterior predictive checks can help identify whether a chosen model reproduces features of the observed data, but cannot validate the clinical relevance of an endpoint.
The main alternatives answer different questions. A crude proportion is easy to interpret when follow-up is comparable and fixed. A rate uses person-time but presumes that a rate is a useful summary. A time-to-event analysis can represent timing and censoring, subject to its own assumptions. A structured clinical profile that keeps benefit and harm outcomes separate avoids imposing a single utility scale; a prespecified utility can summarize trade-offs but depends on value judgments. Regulatory review may need both outcome-specific evidence and a summary, rather than treating one as a substitute for the other.
A common mistake is to say, “There were fewer events in the comparator arm, so treatment is unsafe,” without considering denominators, time at risk, uncertainty, or event severity. The correction is to specify the estimand, compare like with like, quantify posterior uncertainty, and then interpret the result in clinical context. Conversely, a low estimated event rate or a high posterior probability of benefit is not proof that important harms are absent. Limited sample size, short follow-up, rare events, incomplete ascertainment, model misspecification, and uncertainty about long-term effects all constrain what a trial can establish.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- A trial observes 6 events over 600 person-years in one arm and 4 over 200 person-years in another. Calculate the crude rates per 1,000 person-years. Why is comparing event counts alone inadequate?
- With a Beta(1,1) prior, 7 events among 100 patients produce what posterior distribution for the event probability?
- If a posterior mean benefit difference is 0.08 and the harm difference is 0.01, what is the posterior mean incremental utility under \(U=B-4H\)? State its units.
- Name two assumptions that make a simple binomial comparison inappropriate or questionable when follow-up differs across patients.
Solutions. (1) The rates are \(6/600\times1,000=10\) and \(4/200\times1,000=20\) events per 1,000 person-years. The second arm has fewer events but twice the crude rate; counts omit exposure. (2) The posterior is Beta(8,94), because there are 8 posterior event counts and \(1+(100-7)=94\) posterior non-event counts. (3) The mean incremental utility is \(0.08-4(0.01)=0.04\) utility units per patient, under the stated additive scale. (4) Examples include the assumption that each patient has comparable fixed follow-up and a single binary outcome, and the assumption that event ascertainment is complete and outcomes are independent. Unequal time at risk can violate comparability of the binomial outcome window; informative loss to follow-up can also bias the observed event proportions.