Choosing a Denominator for Standardized Mean Differences

Why the Denominator Matters

A standardized mean difference expresses a difference in covariate means relative to a standard deviation. The numerator is the same whichever denominator is chosen, but the scale can change substantially when treatment and control groups have different spreads. As a result, two reports can describe the same difference in means but show different standardized values. A balance diagnostic is interpretable only when its denominator convention is clear and applied consistently.

For a continuous covariate \(X\), let \(T=1\) denote the treatment group and \(T=0\) the control group. Write \(n_1\) and \(n_0\) for the group sizes, \(\bar X_1\) and \(\bar X_0\) for the group means, and \(s_1\) and \(s_0\) for the sample standard deviations. The signed mean difference is \(\bar X_1-\bar X_0\). A positive standardized difference therefore means the treatment-group mean is higher.

The denominator is a scale, not a correction to the mean difference. It does not make the groups more or less balanced; it changes how the same difference is expressed. There is no single denominator that is mandatory for every balance analysis. Common choices include the standard deviation pooled within the two groups, the treatment-group standard deviation, and the control-group standard deviation. The exact formula matters because the word “pooled” is used for more than one calculation.

Three Standard Deviation Choices

The treatment-group and control-group choices are direct: set \(D=s_1\) or \(D=s_0\), respectively. Each measures the difference using the spread in one named group. The pooled choice combines information about within-group variation. A degrees-of-freedom-weighted pooled standard deviation is

This formula weights each group variance by its degrees of freedom, then divides by the total degrees of freedom. It is the familiar pooled estimate of a common variance in a two-group model. As a descriptive scale for an SMD, however, it can be calculated without assuming that the two population variances are equal. Equal variances are needed for the usual interpretation of this pooled variance as an estimate of one common population variance, not for the arithmetic or descriptive use of the resulting denominator.

Another convention gives the groups equal weight in the variance calculation:

This is also often called a pooled standard deviation. It differs from the degrees-of-freedom-weighted version when group sizes differ. A report that says only “pooled SD” leaves this distinction unresolved, so the formula should be stated.

Why the Pooled Formula Has This Form

The degrees-of-freedom-weighted formula comes from combining the within-group sums of squared deviations. In group \(g\), the sample variance satisfies \((n_g-1)s_g^2=\sum_i(X_{ig}-\bar X_g)^2\). Thus, adding the two groups’ within-group sums of squares gives the numerator below. Dividing by the total within-group degrees of freedom gives the pooled variance; taking its square root returns to the units of \(X\).

The calculation deliberately centers each group around its own mean. It describes within-group spread, not the total spread of all observations around one overall mean. The variance of all observations combined also includes a between-group component caused by the difference in group means. That overall variance is a different denominator, and it should not be substituted silently for a pooled within-group standard deviation.

Denominator Choice Can Change Comparisons

When several covariates are being assessed, a denominator convention can affect not only the displayed magnitudes but also the ordering of covariates by absolute SMD. A larger absolute SMD under one convention does not necessarily remain larger under another if the covariates’ group spreads differ. This is why comparisons across diagnostic tables or analyses require matching definitions, not just matching labels.

The control-group denominator can be useful when the control-group spread is the intended reference scale; the treatment-group denominator has the analogous interpretation. The pooled denominator summarizes within-group variation from both groups. These are choices of reference, not tests of a causal model. Whichever is selected, use the same convention for all covariates and all diagnostics that will be compared.

Assumptions, Diagnostics, and Common Pitfalls

For these calculations, the main requirements are that the covariate is quantitatively measured, the group means and SDs are defined for the observations being compared, and the chosen denominator is positive. If a group has zero observed variance, a group-specific denominator is zero and its SMD is undefined. A pooled denominator may still be positive if the other group varies, but that does not make the zero-variance group’s spread informative; inspect the data and explain the convention rather than treating the resulting number as routine.

A denominator also does not summarize every feature of the covariate distributions. Equal means relative to a selected SD do not establish that the distributions have the same shape, tails, or subgroups. An SMD is one descriptive balance diagnostic. It should be read alongside the design context and other relevant diagnostics, rather than treated as a complete account of distributional balance.

• Ambiguous “pooled SD.” The degrees-of-freedom-weighted and equal-group formulas need not agree. State which one was used.
• Confusing within-group pooling with the overall SD. The overall SD includes between-group mean differences; the pooled within-group formula does not. Do not label one as the other.
• Comparing incompatible reports. Different denominators can change magnitudes and covariate rankings. Recalculate using a common rule before comparing.
• Assuming pooling proves equal variances. The pooled formula can be used as a descriptive scale, but its use alone does not show that the groups have equal population variances.
• Dropping the sign without explanation. Absolute SMDs show magnitude but lose direction. Preserve the signed value when the direction of the difference matters.