Why Standardize a Difference in Means?
In a propensity score analysis, balance assessment asks whether the treatment groups are similar on measured covariates before drawing conclusions about treatment effects. As established in the earlier material on Covariate Balance as a Design Diagnostic, balance is a design question: compare the covariate distributions in the groups being analyzed, rather than treating a nonsignificant hypothesis test as evidence that the groups are balanced.
For a continuous covariate, the difference between group means is easy to calculate, but its size depends on the units. A mean difference of 5 could be small for annual income measured in thousands of dollars, or large for age measured in years. A standardized mean difference (SMD) expresses the mean difference relative to a standard deviation, so it is unit-free and easier to compare across covariates measured on different scales.
The numerator and denominator must refer to the same covariate, measured on the same scale, in the two groups being compared. The reference standard deviation must be positive; if it is zero, this formula is undefined. When a problem supplies one common reference standard deviation, use that value directly. The particular reference-SD convention matters, so calculations should state the denominator rather than reporting an unexplained number.
Calculation and Interpretation
Calculate the treated-minus-control difference first, then divide by the reference standard deviation. A positive SMD means the treated group has the larger mean; a negative SMD means the control group has the larger mean. Reversing the group order changes the sign but not the absolute magnitude. For example, an SMD of \(0.40\) means that the treated-group mean is 0.40 reference standard deviations above the control-group mean. It does not mean a 40% difference.
Standardization changes the scale of the mean difference, not the underlying data. The SMD is dimensionless: if \(X\) is converted from centimeters to meters, both the numerator and denominator are divided by 100, so the ratio is unchanged. This makes the magnitude useful for comparing mean differences across covariates. It does not guarantee that the covariates have similar distributions in every respect.
Worked Example: Comparing Baseline Age Means
Suppose a hypothetical analysis compares 120 treated patients with 150 controls. Mean baseline age is 54 years in the treated group and 50 years in the control group. The group standard deviations are 10 and 8 years. Calculate the treated-minus-control SMD.
First calculate the reference standard deviation: $$ s_{\mathrm{ref}}=\sqrt{\frac{10^2+8^2}{2}} =\sqrt{\frac{100+64}{2}} =\sqrt{82} \approx 9.055\text{ years}. $$
The mean difference is \(54-50=4\) years. Therefore, $$ \mathrm{SMD}_{\mathrm{age}}=\frac{54-50}{9.055} =\frac{4}{9.055} \approx 0.442. $$
The positive sign indicates that the treated group is older on average. The difference in means is about 0.44 reference standard deviations. This is a description of mean separation, not a statement that treatment caused an age difference or that the groups differ by 44%.
A Short Justification for Standardization
The reason the SMD is unit-free follows directly from changing units. Suppose the covariate is transformed by \(X^*=a+bX\), where \(b>0\). Each group mean transforms as \(\bar{x}_g^*=a+b\bar{x}_g\), and each group standard deviation transforms as \(s_g^*=b s_g\). Thus the transformed mean difference is $$ \bar{x}_1^*-\bar{x}_0^* =(a+b\bar{x}_1)-(a+b\bar{x}_0) =b(\bar{x}_1-\bar{x}_0). $$ The reference standard deviation also transforms by \(b\): $$ s_{\mathrm{ref}}^* =\sqrt{\frac{(b s_1)^2+(b s_0)^2}{2}} =b\sqrt{\frac{s_1^2+s_0^2}{2}} =b s_{\mathrm{ref}}. $$ Consequently, $$ \mathrm{SMD}_X^* =\frac{b(\bar{x}_1-\bar{x}_0)}{b s_{\mathrm{ref}}} =\mathrm{SMD}_X. $$ The additive shift \(a\) cancels in the difference, and a positive change of scale cancels between numerator and denominator. If \(b<0\), the sign reverses because the covariate ordering reverses, but the absolute SMD is unchanged. This justification concerns a linear change of measurement units; it does not establish balance of distributional features other than the mean.
Worked Applications
Worked Example: A Negative Standardized Difference
In hypothetical data, a continuous baseline laboratory measure has mean 18 units in the treated group and 22 units in the control group. Both groups have a standard deviation of 8 units. Calculate and interpret the SMD.
Since the two standard deviations are both 8 units, the reference standard deviation is $$ s_{\mathrm{ref}}=\sqrt{\frac{8^2+8^2}{2}}=\sqrt{64}=8\text{ units}. $$ The treated-minus-control mean difference is \(18-22=-4\) units, so $$ \mathrm{SMD}=\frac{18-22}{8}=\frac{-4}{8}=-0.500. $$
The negative sign indicates that the treated-group mean is lower. Its magnitude, 0.500, says that the separation between the means is half of the reference standard deviation. Reporting only \(0.500\) would lose the direction; reporting only \(-4\) units would not express the standardized magnitude.
Worked Example: Using a Supplied Reference Standard Deviation
A hypothetical balance table reports a treated-group mean of 72 and a control-group mean of 69 for a continuous score. It specifies a reference standard deviation of 12 score points. Calculate the SMD using that stated denominator.
The treated-minus-control difference is \(72-69=3\) points. Therefore, $$ \mathrm{SMD}=\frac{72-69}{12}=\frac{3}{12}=0.250. $$
The treated-group mean is 0.25 reference standard deviations higher than the control-group mean. The given denominator should be used as specified; silently replacing it with another standard deviation would produce a different calculation and make the reported diagnostic difficult to reproduce.
Worked Example: Rechecking the Sign After Reordering Groups
For a hypothetical continuous covariate, the treated mean is 36 and the control mean is 40. Their standard deviations are 6 and 8. Find the treated-minus-control SMD, then show what happens if the group order is reversed.
The reference standard deviation is $$ s_{\mathrm{ref}}=\sqrt{\frac{6^2+8^2}{2}} =\sqrt{\frac{36+64}{2}} =\sqrt{50} \approx 7.071. $$ The treated-minus-control calculation is $$ \mathrm{SMD}_{1-0}=\frac{36-40}{7.071} =\frac{-4}{7.071} \approx -0.566. $$
With the order reversed, the numerator is \(40-36=4\), giving $$ \mathrm{SMD}_{0-1}=\frac{40-36}{7.071} \approx 0.566. $$
The sign changes because the comparison order changed; the absolute magnitude remains approximately 0.566. A balance table should make its group order clear so readers can interpret the sign consistently.
What the SMD Does and Does Not Diagnose
A continuous-covariate SMD summarizes the difference between two group means relative to a reference spread. It is not a complete comparison of the covariate distributions. Groups can have equal means and different variances or shapes; for example, one group may have observations concentrated near its mean while the other has heavier tails. In that situation the mean-based SMD can be zero even though the distributions differ.
The SMD also does not incorporate sample size in its definition. Its purpose here is to quantify a standardized difference, not to test whether a population mean difference is statistically distinguishable from zero. A hypothesis test can be highly sensitive to sample size, while a balance diagnostic is used to assess the design and the groups actually being compared. Do not use a large p-value as a substitute for checking covariate balance.
A practical diagnostic review should check the arithmetic and the data being summarized. Confirm that the two means are calculated for the intended treatment groups and analysis sample, that the covariate has the same coding and units in each group, and that the denominator is the stated reference standard deviation. Recalculate an unexpected sign from the group means. If the SMD seems small but distributional differences are plausible, inspect additional summaries or plots appropriate to the covariate rather than treating this one number as conclusive.
| Feature | What the SMD conveys | What it does not establish |
|---|---|---|
| Direction | Which group has the larger mean, given the stated group order | That treatment caused the observed difference |
| Magnitude | Mean separation in units of the chosen reference SD | A percentage difference or a universal pass/fail judgment |
| Comparability | A unit-free summary of mean differences across continuous covariates | Equal variances, tails, or full distributions |
| Inference | A descriptive balance diagnostic for the analyzed groups | A p-value, confidence interval, or test of a causal effect |
Common Mistakes
- Reversing the sign: Using control minus treated while describing the result as treated minus control gives the wrong directional interpretation. Write the group order beside the calculation.
- Using the wrong scale: Dividing by a variance instead of a standard deviation gives a quantity with units and is not this standardized mean difference.
- Calling an SMD a percentage: An SMD of 0.25 means a quarter of the reference standard deviation, not a 25% difference in the covariate.
- Assuming a small mean difference proves full balance: The SMD summarizes means, not variances, tails, or every feature of the distributions.
- Changing the denominator without saying so: The standardized result depends on the reference SD convention. State and use the denominator consistently.
- Interpreting the SMD as a causal effect: It describes a covariate difference between groups; it does not estimate the effect of treatment on that covariate.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- A hypothetical covariate has treated mean 30, control mean 27, and reference standard deviation 6. Calculate the treated-minus-control SMD and interpret its sign.
- A hypothetical variable has treated mean 5 and control mean 9. Its group standard deviations are both 4. Calculate the SMD and state its absolute magnitude.
- Why is the SMD unchanged when a continuous covariate is converted from centimeters to meters?
- Can an SMD of zero establish that the two covariate distributions are identical? Explain.
Solutions. (1) \((30-27)/6=0.500\). The treated-group mean is 0.5 reference standard deviations higher. (2) The reference SD is 4, so \((5-9)/4=-1.000\); the treated-group mean is lower, and the absolute magnitude is 1.000. (3) Both the mean difference and standard deviation are divided by 100, so their ratio is unchanged. (4) No. It establishes equality of the sample means under this calculation, but the groups can still differ in variance, tails, or distributional shape.