Variance Ratios as Balance Diagnostics

Why Compare Variances?

A covariate can have the same average in treated and control groups yet differ substantially in how its values are spread out. For example, both groups might have the same mean age, while ages in one group are tightly clustered and ages in the other range widely. A mean comparison alone would not reveal that difference. A variance ratio summarizes the relative spread of a continuous covariate in the two groups.

In propensity-score matching, balance is assessed by comparing covariate distributions between treated and control units in the matched sample. Standardized mean differences describe differences in location, while variance ratios provide a complementary summary of spread. Neither quantity alone establishes that the full distributions are alike.

Definition: For a continuous covariate $X$, the treated-to-control variance ratio is the sample variance among treated units divided by the sample variance among control units: $$ VR_X=\frac{s_T^2}{s_C^2}. $$ Here $s_T^2$ and $s_C^2$ are the usual within-group sample variances, calculated with denominator $n_T-1$ and $n_C-1$, respectively. The ratio is dimensionless. A value of 1 means the two observed sample variances are equal; a value greater than 1 means the treated group has greater sample variance, and a value less than 1 means it has less.

The sample variance for either group is calculated from that group’s own mean. For example, in the treated group,

s_T^2=\frac{\sum_{i:A_i=1}(X_i-\bar{X}_T)^2}{n_T-1},

where $A_i=1$ denotes a treated unit, $n_T$ is the number of treated units, and $\bar{X}_T$ is their sample mean. The control variance is calculated in the same way using the control units. These are marginal, group-specific variances: they are not the variance of treated-control differences within matched pairs.

Calculation and Interpretation

Calculate each group’s variance first, then divide the treated variance by the control variance. The direction of the ratio is important: reversing the order gives its reciprocal. A ratio of 2 means the treated sample variance is twice the control sample variance; it does not mean the treated standard deviation is twice as large. Since standard deviation is the square root of variance, that standard-deviation ratio would be $\sqrt{2}$.

The variance ratio is unchanged by a common change of measurement units. If $X$ is converted from years to months, both sample variances are multiplied by $12^2$, so their ratio is unchanged. This follows directly:

\frac{s_{T,\text{months}}^2}{s_{C,\text{months}}^2} = \frac{12^2s_{T,\text{years}}^2}{12^2s_{C,\text{years}}^2} = \frac{s_{T,\text{years}}^2}{s_{C,\text{years}}^2}.

This unit invariance makes ratios easy to compare across measurement scales, but it does not make them a complete measure of balance. The ratio describes relative variance, not the location of the groups, and it does not show whether their distributions differ in other ways.

Worked Example: Equal Means, Different Variances

Suppose a hypothetical matched sample has five treated and five control units. Their values of a continuous covariate are:

Treated: 2, 4, 6, 8, 10. Control: 0, 4, 6, 8, 12.

Both groups have mean 6. For the treated group, the squared deviations from 6 sum to $16+4+0+4+16=40$, so $s_T^2=40/(5-1)=10$. For the control group, the squared deviations sum to $36+4+0+4+36=80$, so $s_C^2=80/(5-1)=20$.

The treated-to-control variance ratio is

VR_X=\frac{10}{20}=0.5000.

The group means are exactly equal, but the treated sample variance is half the control sample variance. The means therefore indicate no difference in average covariate value, while the variance ratio reveals a difference in spread. Both summaries are needed to describe these aspects of balance.

Reading the Ratio Alongside Mean Differences

A variance ratio and a standardized mean difference answer different questions. The variance ratio compares spread; the standardized mean difference compares group means relative to a scale based on variability. A ratio near 1 does not guarantee similar means, and a small standardized mean difference does not guarantee similar variances. Reading them together gives a more informative, though still partial, assessment of balance.

Worked Example: Two Diagnostics for One Covariate

Consider another hypothetical matched sample, with covariate values measured in years:

Treated: 52, 54, 56, 58. Control: 50, 52, 56, 58.

The treated mean is $(52+54+56+58)/4=55$ years, and the control mean is $(50+52+56+58)/4=54$ years. For the treated group, the squared deviations from 55 sum to $9+1+1+9=20$, giving $s_T^2=20/3=6.667$ years squared. For controls, the squared deviations from 54 sum to $16+4+4+16=40$, giving $s_C^2=40/3=13.33$ years squared.

Thus,

VR_X=\frac{20/3}{40/3}=0.5000.

The treated mean exceeds the control mean by 1 year, and the treated sample variance is half the control sample variance. For context, using the pooled within-group standard deviation $\sqrt{(s_T^2+s_C^2)/2}=\sqrt{10}=3.162$ years, the standardized mean difference is $1/3.162=0.3162$. The mean difference and variance ratio together describe a location difference and a spread difference; neither should be substituted for the other.

In practice, compute these summaries for each continuous covariate before and after matching, using clearly identified treated and control samples. Compare like with like: a post-match treated variance should be divided by the post-match control variance for the same covariate, with the same coding and variance convention. A before-and-after table can show whether matching changed the relative spread as well as the mean difference.

Worked Example: Comparing Ratios Across Covariates

Suppose a hypothetical matched sample contains four units in each group. Two continuous covariates are recorded as follows:

Covariate	Control values	Treated values
X	2, 4, 6, 8	1, 5, 5, 9
Y	10, 12, 14, 16	11, 13, 15, 17

For $X$, both group means are 5. The control squared deviations sum to $9+1+1+9=20$, so $s_C^2=20/3=6.667$. The treated squared deviations sum to $16+0+0+16=32$, so $s_T^2=32/3=10.67$. Therefore $VR_X=(32/3)/(20/3)=1.600$. The treated variance is 1.6 times the control variance, despite identical means.

For $Y$, the control mean is 13 and the treated mean is 14, so the treated-control mean difference is 1 unit. In each group the squared deviations from that group’s own mean sum to 20, giving $s_T^2=s_C^2=20/3$ and $VR_Y=1.000$. The spread is equal in this sample, but the means differ. These two covariates illustrate why a balance table should report location and spread summaries separately.

Assumptions, Diagnostics, and Limitations

A variance ratio is a descriptive summary rather than an estimate of a causal effect. Its calculation requires that the covariate be measured comparably in treated and control units, that the group membership and analysis sample be specified, and that each group have enough non-identical observed values for a meaningful variance calculation. The sample variances should be computed using the same convention in both groups. In weighted analyses, use a variance summary consistent with the weighting design rather than automatically treating weighted observations as an ordinary unweighted sample.

Check the underlying calculations and sample sizes as well as the ratio. A control variance of zero makes the treated-to-control ratio undefined. If both variances are zero, the ratio is $0/0$, which is also undefined; the covariate is constant in both groups, so this ratio cannot describe their relative spread. If only the treated variance is zero and the control variance is positive, the ratio is zero. Very small group sizes can also make sample variances unstable, so a striking ratio from a few observations should be interpreted cautiously.

Ratios below 1 indicate less treated-group variance; ratios above 1 indicate more. Some applied guidance treats values between 0.5 and 2 as a rough range for review, but this is a heuristic, not a universal threshold or a formal pass/fail rule. The appropriate assessment depends on the covariate, sample size, analysis goal, and other balance evidence. Do not treat a ratio inside a chosen range as proof of balance.

A variance ratio can be 1 even when group means differ, as the second worked example illustrates. It can also be 1 when the groups have different distributional shapes but the same variance. Thus it does not assess all features of a distribution. Use it with mean-difference summaries and other appropriate diagnostics; do not infer that matching succeeded for a covariate merely because its variance ratio is close to 1.

Common Mistakes

Using the ratio as a location measure. A ratio of 1 says the sample variances are equal; it says nothing by itself about whether the group means are equal.
Reversing the numerator and denominator without noticing. The defined ratio is treated variance divided by control variance. Reversing it changes, for example, 0.5 to 2.
Calling the variance ratio a standard-deviation ratio. A variance ratio of 4 corresponds to a standard-deviation ratio of 2, not 4.
Treating a heuristic range as a guarantee. Being within a suggested range does not establish overall covariate balance or eliminate the need for other diagnostics.
Ignoring a zero denominator. When the control variance is zero, the ratio is undefined; it is not a finite measure of relative spread.

Key takeaway: The treated-to-control variance ratio is a dimensionless summary of relative covariate spread. Interpret it with its direction clear, alongside mean differences, and as one diagnostic rather than a complete verdict on balance.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions below.

The treated variance of a covariate is 18 and its control variance is 12. Calculate the treated-to-control variance ratio and interpret it.
Two groups have equal covariate means, but their variances are 9 and 36 in the treated and control groups, respectively. What is the ratio, and what does it reveal?
A treated-to-control variance ratio is 1. What can and cannot be concluded from that value?
The control variance is zero and the treated variance is positive. Is the ratio defined? Explain.

Solutions. (1) $VR_X=18/12=1.500$, so the treated sample variance is 1.5 times the control sample variance. (2) $VR_X=9/36=0.2500$; despite equal means, the treated sample variance is one quarter of the control sample variance. (3) The two sample variances are equal, but the group means and other distributional features may still differ. (4) No. Division by a zero control variance is undefined, so there is no finite treated-to-control variance ratio.