Why Standardize a Treatment Effect?
A treatment effect measured in the outcome’s original units is often easiest to interpret clinically: for example, a mean reduction of 4 points on a symptom scale. A standardized mean difference expresses that effect relative to the variability of the outcome. This can help compare effects across scales, but it changes the interpretation: the result is a dimensionless ratio, not a difference in clinical units.
The key statistical issue is that the treatment effect and the standard deviation may both be uncertain. Consequently, the posterior distribution of their ratio cannot generally be obtained by dividing one posterior summary by another. The appropriate calculation transforms each joint posterior draw.
The denominator must match the estimand. In a two-arm trial with a shared outcome variance, \(\sigma\) is the common within-arm standard deviation. In a paired design, standardizing the mean paired change by the standard deviation of paired changes answers a different question from standardizing by the marginal standard deviation of measurements. State which standard deviation is used before interpreting \(\delta\).
Model and Posterior Transformation
For a basic parallel-group analysis, suppose participants are independent within and between arms, and each arm’s outcomes are normally distributed with means \(\mu_T\) and \(\mu_C\) and a shared variance \(\sigma^2\). The common-variance assumption means the two arms have the same within-arm outcome standard deviation. A Bayesian analysis assigns a prior to the unknown parameters and updates it using the trial data. The resulting posterior represents uncertainty about \(\Delta\) and \(\sigma\) jointly.
A standard conjugate setup places a prior on the arm means conditional on \(\sigma^2\) and a prior on \(\sigma^2\); other suitable prior specifications are possible. The exact prior and likelihood determine the posterior draws. Once draws from the joint posterior of \((\Delta,\sigma)\) are available, the transformation itself is direct.
For each draw \(m\), retain a paired value \((\Delta^{(m)},\sigma^{(m)})\) from the posterior. The two values must come from the same draw so their posterior dependence is preserved.
Calculate \(\delta^{(m)}=\Delta^{(m)}/\sigma^{(m)}\). Because \(\sigma^{(m)}>0\), the sign of each standardized draw is the sign of its corresponding mean difference.
Use the \(\delta^{(m)}\) values to estimate posterior means, medians, credible intervals, or probabilities such as \(P(\delta>0.5\mid y)\), where \(y\) denotes the observed trial data.
The reason this procedure is necessary follows from a change of variables. Write the joint posterior density of \((\Delta,\sigma)\) as \(p(\Delta,\sigma\mid y)\). Since \(\Delta=\delta\sigma\), the absolute Jacobian determinant for the transformation from \((\delta,\sigma)\) to \((\Delta,\sigma)\) is \(\sigma\). Therefore,
This marginal posterior combines uncertainty in the numerator with uncertainty in the denominator. Equivalently, conditional on a particular \(\sigma\), scaling \(\Delta\) by \(1/\sigma\) also scales its conditional standard deviation by \(1/\sigma\). Averaging over posterior uncertainty in \(\sigma\) produces a mixture of these scaled distributions. In general, that mixture is not summarized correctly by dividing the posterior mean of \(\Delta\) by the posterior mean of \(\sigma\).
Calculating Posterior Summaries
Worked Example: Standardizing a Mean Difference in a Trial
Suppose a hypothetical randomized trial compares a treatment with control on a symptom score, where a higher score is worse. There are 30 participants in each arm; the observed means are 18 and 22 points, respectively, and the within-arm standard deviations are about 5 points. Thus the observed treatment-minus-control difference is \(-4\) points. A Bayesian analysis under a shared-variance normal model produces the following eight illustrative joint posterior draws. The small set is used to show the arithmetic, not to claim precise Monte Carlo accuracy.
| Draw | \(\Delta^{(m)}\) (points) | \(\sigma^{(m)}\) (points) | \(\delta^{(m)}=\Delta^{(m)}/\sigma^{(m)}\) |
|---|---|---|---|
| 1 | -2.8 | 4.8 | -0.5833 |
| 2 | -3.5 | 5.0 | -0.7000 |
| 3 | -4.1 | 4.6 | -0.8913 |
| 4 | -2.4 | 5.2 | -0.4615 |
| 5 | -5.0 | 5.1 | -0.9804 |
| 6 | -3.8 | 4.4 | -0.8636 |
| 7 | -4.5 | 5.3 | -0.8491 |
| 8 | -3.1 | 4.7 | -0.6596 |
For example, draw 1 gives \(-2.8/4.8=-0.5833\). The mean of the eight standardized draws is approximately \(-5.9888/8=-0.7486\). Seven of the eight draws are below \(-0.5\), so their empirical posterior probability of \(\delta<-0.5\) is \(7/8=0.875\). The negative direction corresponds to lower symptom scores in the treatment arm. With a real analysis, use enough well-behaved posterior draws to estimate summaries and probabilities accurately.
The result is expressed in standard deviation units, not symptom-score points. Whether a magnitude of 0.5 is clinically important depends on the endpoint and clinical context; it is not a universal threshold.
Worked Example: Why a Ratio of Posterior Means Is Different
Consider three equally weighted hypothetical joint posterior draws for a positive mean difference and standard deviation: \((\Delta,\sigma)=(2,2),(4,4),(6,8)\), with both quantities measured in the same outcome units. The draw-wise standardized effects are
Their posterior mean is \((1+1+0.75)/3=0.9167\). In contrast, the ratio of the posterior means is
These are different summaries of different calculations. The posterior mean of the ratio is obtained by averaging the ratios; the ratio of posterior means is obtained by averaging each component first. In general, neither uncertainty nor posterior dependence disappears just because a ratio is reported. Use the draw-wise ratios to summarize the posterior of \(\delta\).
Credible Intervals, Probabilities, and Interpretation
A credible interval for \(\delta\) is calculated from quantiles of the transformed posterior draws. For instance, the endpoints of an equal-tailed 95% credible interval are the 2.5th and 97.5th percentiles of those draws. Under the specified model and prior, the posterior probability that \(\delta\) lies in that interval is 0.95. This is a statement about the posterior distribution, not a guarantee that the interval contains a fixed parameter in repeated samples.
Posterior probabilities for clinically defined thresholds are calculated in the same way. If a positive standardized difference is beneficial and the chosen threshold is \(\delta_0\), estimate \(P(\delta>\delta_0\mid y)\) by the proportion of transformed draws exceeding \(\delta_0\). The direction and threshold should be chosen to match the endpoint’s clinical meaning before interpreting that probability.
Worked Example: Threshold Probability and Unit Conversion
Suppose a hypothetical trial measures a biomarker in milligrams per litre. Six joint posterior draws for the treatment-minus-control mean difference and its standardizing standard deviation are \((12,20),(18,24),(8,16),(20,25),(10,20),(16,20)\), all in milligrams per litre. The standardized draws are \(0.600,0.750,0.500,0.800,0.500,0.800\). If the prespecified threshold is \(\delta>0.5\), four draws exceed it, giving the illustrative estimate \(P(\delta>0.5\mid y)\approx4/6=0.667\). Draws equal to 0.5 do not count as exceeding 0.5.
If the biomarker is instead expressed in grams per litre, both \(\Delta\) and \(\sigma\) are divided by 1,000 in every draw. For example, \(12/1{,}000\) divided by \(20/1{,}000\) is still 0.600. Thus the standardized effect is unchanged by a common positive change of measurement units. This invariance does not make the standardized effect more clinically informative than the original-unit difference; it answers a different question.
Assumptions, Diagnostics, and Alternatives
The transformation requires a positive standardizing standard deviation and joint posterior draws for the chosen estimand. The shared-variance normal model additionally relies on independent participants, a suitable normal outcome model, and a common within-arm variance. These assumptions should be considered in light of the trial design and outcome. For paired data, retain the pairing in the likelihood and clearly identify whether the target denominator is the standard deviation of within-person differences or another scientifically justified scale.
Inspect posterior draws and model fit before relying on a standardized summary. Check that sampling diagnostics indicate adequate exploration of the joint posterior; in particular, poor mixing or high autocorrelation can make quantiles and threshold probabilities inaccurate. Compare observed outcomes with quantities simulated from the fitted model to identify important departures, such as skewness or influential outliers. Examine the posterior of \(\sigma\): if substantial mass is near zero, division can produce extreme standardized draws and an unstable or very long-tailed posterior for \(\delta\). Also assess whether the equal-variance assumption is credible. If it is not, a common standard deviation may not represent either arm adequately, and the estimand and model should be reconsidered rather than applying the same formula automatically.
The main alternative is to report the raw mean difference \(\Delta\), which retains the endpoint’s clinical units and may be more directly useful for decisions. Standardizing can help compare effects across measurement scales, but the result depends on the selected denominator and can obscure absolute benefit. Using separate arm standard deviations or a pooled sample standard deviation defines other standardized quantities; it does not automatically yield the same posterior estimand as \(\Delta/\sigma\) with a shared population standard deviation.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- If the treatment-minus-control difference is \(-3\) points and the common standard deviation is 6 points, what is the standardized mean difference?
- Why should paired draws of \(\Delta\) and \(\sigma\) be used when calculating posterior standardized effects?
- In one posterior draw, \(\Delta=5\) and \(\sigma=10\). Does that draw exceed a threshold of \(\delta>0.5\)?
- What happens to \(\delta\) if the outcome is converted from one unit to another by multiplying all measurements by a positive constant?
Solutions. (1) \(\delta=-3/6=-0.5\), a difference of one-half of a standard deviation in the negative direction. (2) The quantities may be posterior-dependent; pairing preserves that dependence and gives the correct draw-wise ratio distribution. (3) No. The ratio is \(5/10=0.5\), which equals but does not exceed the threshold. (4) Both \(\Delta\) and \(\sigma\) are multiplied by the same constant, so their ratio is unchanged.