Why a Balance Table Needs More Than a Standardized Difference
A balance table compresses a large amount of design information into a small space. Its purpose is not merely to display a number for each covariate: readers need to know which groups were summarized, which observations or weights were used, what each summary means, and how the standardized difference was calculated. Without those details, identical-looking entries can describe different analyses.
A before-and-after table places summaries from the original, unadjusted groups beside summaries after matching or weighting. This makes it possible to see both the starting imbalance and how the adjustment changed it. The table is a descriptive design diagnostic; it does not establish that treatment groups are exchangeable, that unmeasured confounding is absent, or that a treatment effect is unbiased.
For a continuous covariate, the summaries are usually group means and standard deviations. For a binary covariate, they are proportions. A multi-level categorical covariate can be represented by the proportion in each level, with a standardized difference for each level indicator. The standardized mean difference (SMD) is a signed, unitless difference scaled by a standard deviation or a binary-variable analogue. Its sign indicates which group has the larger mean or proportion under the chosen subtraction order.
Specify the Table Before Calculating It
Before assembling entries, make the design represented by the table explicit. State which group is the treated group and which is the comparison group; define the covariates and their units or coding; and identify the observations used before adjustment. Then describe the adjustment and the observations or weights used afterward. A matched analysis should say whether its summaries use matched units and how any unmatched units were handled. A weighted analysis should identify the weights and the population or group summaries they represent.
For clarity, the examples below subtract the comparison-group summary from the treated-group summary. They use a fixed reference denominator computed from the original, unadjusted groups. This makes before-and-after SMDs comparable on the same scale. Other denominator conventions can be appropriate, but a table must state its convention; readers should not be expected to infer it.
Worked Example: Continuous and Binary Covariates Before and After Matching
Suppose a hypothetical observational data set has 100 treated and 200 comparison units before matching. Matching retains 80 treated units and 80 comparison units. Age is measured in years; diabetes is coded 1 for present and 0 for absent. For this table, use the original groups' pooled standard-deviation analogue as a fixed denominator for both stages. The following summaries are hypothetical.
| Covariate | Before: treated | Before: comparison | Before SMD | After: treated | After: comparison | After SMD |
|---|---|---|---|---|---|---|
| Age (years), mean (SD) | 52 (10) | 49 (12) | 0.272 | 51 (9) | 50 (10) | 0.0905 |
| Diabetes, proportion | 0.400 | 0.250 | 0.324 | 0.350 | 0.300 | 0.108 |
For age, the fixed reference denominator is
The before-matching SMD is \((52-49)/11.045=3/11.045=0.2716\), displayed as 0.272. The after-matching SMD uses the same denominator: \((51-50)/11.045=1/11.045=0.09054\). Thus the treated group had a higher mean age at both stages, while the standardized difference was smaller after matching. The mean difference is in years; the SMD is unitless.
For diabetes, the original-group proportions are \(p_T=40/100=0.400\) and \(p_C=50/200=0.250\). The fixed binary reference denominator is
The before SMD is \((0.400-0.250)/0.4623=0.3244\). After matching, the proportions are \(28/80=0.350\) and \(24/80=0.300\), so the after SMD is \((0.350-0.300)/0.4623=0.1081\). Those unrounded fractions are used in the calculation; the displayed summaries are rounded. The table should be accompanied by a note such as: “Before, \(n_T=100\), \(n_C=200\); after matching, \(n_T=n_C=80\); entries are group means (SDs) or proportions; SMD is treated minus comparison, using the original-group denominator.” That note makes the table interpretable without guessing.
Why Use a Consistent Denominator?
A before-and-after comparison is easiest to read when both SMDs for a covariate use the same reference scale. Let \(D_{\mathrm{before}}\) and \(D_{\mathrm{after}}\) be the treated-minus-comparison differences in a covariate summary, and let \(s_{\mathrm{ref}}\) be one fixed positive denominator. Then
Because the divisor is unchanged, a change in the displayed SMD reflects a change in the group difference, not a simultaneous change in its scale. If instead separate denominators are used, the SMD can change because the numerator changed, because the denominator changed, or both. That approach is not automatically invalid, but the table must name the conventions at each stage and readers must interpret the entries accordingly.
This is a reporting decision, not a claim that one denominator suits every analysis. The earlier material on choosing a denominator for standardized mean differences explains why conventions can differ. In a table, clarity and consistency matter: specify the denominator, keep full precision during calculation, and avoid comparing entries as though they shared a scale when they do not.
Worked Example: Reconstructing a Binary-Covariate Entry
In a hypothetical weighted analysis, the original treated and comparison proportions for a binary covariate are 0.20 and 0.30. After weighting, the corresponding proportions are 0.333 and 0.317. The table uses the original proportions to define a fixed binary denominator and subtracts comparison from treated. Find both SMDs and explain the entries.
The reference denominator is
Before weighting, the difference is \(0.20-0.30=-0.10\), so the SMD is \(-0.10/0.4301=-0.2325\). The negative sign means the treated group had the lower proportion. After weighting, using the displayed proportions, the difference is \(0.333-0.317=0.016\), and the SMD is \(0.016/0.4301=0.0372\). If the proportions are rounded displays of more precise weighted estimates, the actual SMD should be calculated from those unrounded estimates and may differ slightly. The table note must say that the after summaries are weighted proportions, identify the weight convention, and make clear that the denominator remains based on the original groups.
This example also shows why arithmetic and rounding need to agree. A reported difference of 0.016 divided by 0.4301 is about 0.0372, not a value computed from a different, unshown difference. When limited display precision makes the arithmetic appear inconsistent, say that calculations use unrounded values rather than implying that rounded entries reproduce the result exactly.
Reporting Categorical Covariates
A categorical variable with more than two levels does not have a single natural mean. A practical table reports the proportion in each level and treats each level as an indicator for calculating a standardized difference. The rows should identify the levels and make clear that proportions within a categorical variable sum to one. Since the rows are related, they are not independent pieces of evidence; they provide a readable description of where group composition differs.
Worked Example: Three Levels of a Categorical Covariate
Consider hypothetical three-level employment status, coded as A, B, or C. Before adjustment, the treated proportions are 0.20, 0.30, and 0.50, while the comparison proportions are 0.25, 0.35, and 0.40. After matching, the proportions are 0.250, 0.3125, and 0.4375 for treated units, and 0.275, 0.3625, and 0.3625 for comparison units. Use each original level-specific binary denominator as the fixed reference for that level.
For level A, the original denominator is \(\sqrt{[0.20(0.80)+0.25(0.75)]/2}=\sqrt{0.17375}=0.4168\). Its before SMD is \((0.20-0.25)/0.4168=-0.1200\); its after SMD is \((0.250-0.275)/0.4168=-0.0600\).
For level B, the denominator is \(\sqrt{[0.30(0.70)+0.35(0.65)]/2}=\sqrt{0.21875}=0.4677\). The before SMD is \((0.30-0.35)/0.4677=-0.1069\), and the after SMD is \((0.3125-0.3625)/0.4677=-0.1069\).
For level C, the denominator is \(\sqrt{[0.50(0.50)+0.40(0.60)]/2}=\sqrt{0.245}=0.4950\). The before SMD is \((0.50-0.40)/0.4950=0.2020\); the after SMD is \((0.4375-0.3625)/0.4950=0.1515\). These entries show how each level's treated-versus-comparison difference changed on its own fixed scale. A table should identify the levels and the variable, not merely show three unexplained numbers.
What to Check and What the Table Cannot Say
A useful balance table is internally consistent and faithful to the design. Check that the before and after columns use the intended samples; that weighted summaries use the stated weights; and that matched summaries are based on the retained matched units rather than silently reverting to the full original sample. Confirm that proportions use the same event coding in both groups and stages, and that the direction of subtraction is unchanged. For categorical variables, verify that level proportions sum to one apart from rounding.
Also check whether reported SMDs can be reproduced from the unrounded summaries and the stated denominator. In practice, weights may be normalized in different ways, and tables may use effective sample sizes or raw counts for different purposes. Identify what each number represents instead of reporting an ambiguous single “\(n\).” For a matched design, distinguish the number of matched units from the number of pairs if both are relevant.
Balance tables do not summarize every feature of a distribution. Equal means or proportions do not guarantee similar spread, tails, or joint covariate patterns. Nor does a small SMD prove that all confounding has been removed: it describes measured covariates and depends on their definitions and the observed data. The table is best used alongside other design diagnostics appropriate to the analysis, rather than as a certificate of causal validity.
A common mistake is to read smaller after-adjustment SMDs as proof that the adjustment has succeeded in every important respect. The correction is to treat the table as evidence about the listed measured covariates and the reported summaries only. Another mistake is to compare before and after SMDs calculated with different denominators without noting the change. In that case, a changed SMD may reflect a changed scale rather than an improved group difference. State and apply the convention consistently, and inspect the summaries themselves as well as the standardized values.
Check Your Understanding
Answer each question, then compare with the brief solutions.
- Why should a table state the sample sizes before and after matching?
- A continuous covariate has a treated-minus-comparison mean difference of 2 units and a stated fixed denominator of 8 units. What is its SMD, and what does its sign indicate?
- For a binary covariate, the unadjusted proportions are 0.10 and 0.20. Calculate the fixed reference denominator using the binary formula.
- Why can an after-adjustment SMD be difficult to compare with its before-adjustment value if the denominator changed?
- What is one limitation of reporting only the SMDs for the levels of a categorical covariate?
Solutions. (1) The sample sizes identify which units contribute to each summary and make the adjustment's retained sample visible. (2) The SMD is \(2/8=0.25\); its positive sign means the treated mean is higher under the stated subtraction order. (3) The denominator is \(\sqrt{[0.10(0.90)+0.20(0.80)]/2}=\sqrt{0.125}=0.3536\). (4) The change can reflect both a changed group difference and a changed scale, so the two entries are not directly comparable as if the divisor were fixed. (5) Level-specific SMDs describe marginal proportions but do not describe every joint pattern or other distributional feature of the covariate data.