Standardized Differences After Weighting

The denominator is calculated before applying the analysis weights; the numerator uses the weighted means. This separates the change in covariate balance due to weighting from a change in the scale used to measure that balance. Another convention is to divide by a pooled standard deviation calculated after weighting. That convention is not interchangeable with the fixed-reference version: the weighted variance itself can change when the weights change. Always identify which denominator is being used, especially when comparing reported SMDs.

For a binary covariate, use the reference standard deviation of a Bernoulli variable. If its unweighted prevalence in the reference sample is \(p_{\mathrm{ref}}\), then \(S_{\mathrm{ref}}=\sqrt{p_{\mathrm{ref}}(1-p_{\mathrm{ref}})}\). For a time-varying covariate, define the reference sample as the eligible risk set at the relevant time, and calculate its scale there before weighting.

Binary and Time-Varying Covariates

The same weighted-mean formula applies to a binary covariate coded 0 and 1: its weighted mean is its weighted proportion of 1s. The denominator must be based on a defined reference prevalence, not silently substituted with a weighted prevalence from one treatment group. For longitudinal data, the calculation is repeated at each time using the appropriate covariate measured before the treatment decision at that time and the analysis weights intended for that comparison.

At time \(t\), let \(L_t\) denote a covariate measured before the treatment \(A_t\), and consider the eligible individuals in the risk set for that decision. Comparing the weighted means of \(L_t\) between \(A_t=1\) and \(A_t=0\) gives a time-specific mean-balance diagnostic. Use the weight appropriate to the longitudinal treatment contrast and state whether it includes the weight contribution for treatment at time \(t\). Mixing weights from different time points can produce a comparison that does not correspond to the intended diagnostic.

Two Properties of the Statistic

These properties explain why normalization choices and covariate units matter.

Proof. The rescaled mean in group \(a\) is $$ \frac{\sum_{i:A_i=a}c_aW_iX_i}{\sum_{i:A_i=a}c_aW_i} = \frac{c_a\sum_{i:A_i=a}W_iX_i}{c_a\sum_{i:A_i=a}W_i} =\bar X_{w,a}. $$ The group means are unchanged, so their difference is unchanged. The reference standard deviation is fixed, hence the SMD is unchanged as well. This is why normalizing weights to sum to a common total within each treatment group does not alter this diagnostic. \(\square\)

Proof. By linearity of the weighted mean, \(\bar Y_{w,a}=b\bar X_{w,a}+d\). Thus the weighted mean difference is \(b(\bar X_{w,1}-\bar X_{w,0})\). A standard deviation transforms as \(S_{\mathrm{ref},Y}=\lvert b\rvert S_{\mathrm{ref},X}\), because adding \(d\) does not change a standard deviation and multiplication by \(b\) multiplies it by \(\lvert b\rvert\). Therefore $$ \operatorname{SMD}_w(Y) =\frac{b}{\lvert b\rvert}\operatorname{SMD}_w(X). $$ Taking absolute values proves the magnitude claim, and the sign claim follows from \(b/\lvert b\rvert\). \(\square\)

Reading and Comparing Results

An SMD is dimensionless. Its sign gives direction, while its absolute value gives separation relative to the chosen reference scale. A table should report the covariate, time point if relevant, signed or absolute SMD, reference-scale definition, and the weighting scheme. If several treatment-model specifications are being compared, keep the analysis sample and reference scales fixed; otherwise a change in SMD can reflect a changed denominator or sample as well as a changed weighted difference.

Rules such as treating an absolute SMD below 0.1 as reassuring are conventions, not universal significance tests or guarantees. No threshold proves that a causal effect is unbiased. A small SMD says that the two weighted means are close relative to the stated scale for that covariate. It does not establish equality of variances, tails, or the full covariate distributions, and it does not address unmeasured confounding.

Common Pitfalls and Diagnostics

• Switching denominators without saying so. Two SMDs can differ even when their weighted mean difference is identical. Report the denominator convention and use it consistently for comparisons.
• Comparing unlike risk sets. A time-specific SMD should use the individuals eligible for that treatment decision and the covariate value measured before it. State the time and risk set.
• Reporting only an absolute value. The absolute SMD hides which group has the higher mean. Keep the signed value when direction is informative, even if a table also reports magnitude.
• Treating a small SMD as proof of balance in every sense. It concerns a difference in means only. Other distributional features and unmeasured variables are not assessed by this statistic.
• Using the statistic without checking its ingredients. Confirm that weights are finite and nonnegative, each compared group has positive total weight, observations are aligned with the intended time point, and \(S_{\mathrm{ref}}>0\). A zero reference standard deviation makes the SMD undefined; a very small one can make it unstable or uninformative.

Compared with an unstandardized weighted difference, the SMD improves scale comparability but adds a denominator convention that must be defended. Compared with a post-weighting standardization, a fixed pre-weighting reference makes comparisons across weighting choices more stable. Neither approach turns a mean comparison into a complete assessment of covariate distributions. Extreme or concentrated weights can also make weighted means depend heavily on a few observations, so interpret SMDs alongside the weight diagnostics and effective sample size rather than in isolation.