Covariate Balance as a Weight Diagnostic

Why Balance Is a Weight Diagnostic

Inverse probability weights are intended to remove the measured covariate differences that guided treatment assignment. If, after weighting, patients receiving different treatments still differ systematically in measured confounders, the weights have not produced the intended pseudo-population for those covariates. Balance is therefore an important diagnostic of weighting.

In longitudinal analyses, the comparison is time-specific. Just before treatment at time $t$, the relevant measured covariates are $L_t$, and treatment history through the previous time is $\bar A_{t-1}$. The balance target is that, within a given prior treatment history, the weighted distribution of measured $L_t$ does not depend on the current treatment $A_t$. This qualification matters: stabilized weights may preserve differences between prior treatment histories, so an unconditional comparison of current treatment groups need not express the intended balance property.

Definition: Let $H_t=(\bar A_{t-1},\bar L_t)$ denote the observed treatment and covariate history available just before $A_t$, where $\bar L_t=(L_0,\ldots,L_t)$. Write $p_t(a\mid H_t)=P(A_t=a\mid H_t)$ for the true treatment probability and $q_t(a\mid\bar A_{t-1})$ for a chosen numerator probability. The stabilized treatment weight through time $t$ is $$ W_t=\prod_{k=0}^{t}\frac{q_k(A_k\mid\bar A_{k-1})}{p_k(A_k\mid H_k)}. $$ For a function $g$ of the measured covariates at time $t$, time-specific weighted balance means that, within each prior treatment history with positive probability, the weighted mean of $g(L_t)$ is the same across possible values of $A_t$. Considering all such functions describes equality of the weighted distributions; checking a limited set of functions provides only evidence about balance.

The Population Balance Identity

The identity behind this diagnostic follows from the weight factor for the current treatment. Assume that $p_t$ is the true conditional treatment probability, that $p_t(a\mid H_t)>0$ wherever the relevant histories occur, and that $q_t(a\mid\bar A_{t-1})>0$ for treatment values being compared. Assume the weighted expectations below are finite. No causal exchangeability assumption is needed for this balance identity: it is a probability calculation about treatment assignment and weights.

Result: For any integrable $g(L_t)$, any prior history $\bar a_{t-1}$ with positive weighted mass, and any treatment value $a$ with positive $q_t(a\mid\bar a_{t-1})$, $$ \frac{E\!\left[W_t I(\bar A_{t-1}=\bar a_{t-1}) I(A_t=a)g(L_t)\right]} {E\!\left[W_t I(\bar A_{t-1}=\bar a_{t-1}) I(A_t=a)\right]} = \frac{E\!\left[W_{t-1} I(\bar A_{t-1}=\bar a_{t-1})g(L_t)\right]} {E\!\left[W_{t-1} I(\bar A_{t-1}=\bar a_{t-1})\right]}. $$ The right-hand side does not depend on $a$, so the weighted mean is equal across current treatment values within that prior history.

To prove the identity, condition on $H_t$. The factor $W_{t-1}$, the indicators for the prior treatment history, and $g(L_t)$ are all determined by $H_t$. The remaining factor involving current treatment has conditional expectation $$ E\!\left[\left.\frac{q_t(A_t\mid\bar A_{t-1})}{p_t(A_t\mid H_t)}I(A_t=a)\right|H_t\right] = \frac{q_t(a\mid\bar A_{t-1})}{p_t(a\mid H_t)}p_t(a\mid H_t) = q_t(a\mid\bar A_{t-1}). $$ On the event $\bar A_{t-1}=\bar a_{t-1}$, this last quantity is the same constant $q_t(a\mid\bar a_{t-1})$ in numerator and denominator. Therefore the numerator of the left-hand side is that constant times the numerator of the right-hand side, and its denominator is that constant times the denominator of the right-hand side. Cancelling it proves the identity. $\square$

This is a population result, not a promise that every finite weighted sample will be exactly balanced. Estimated treatment probabilities, random variation, sparse histories, and extreme weights can all leave visible discrepancies. The identity also says nothing by itself about covariates that were not measured or included in the treatment history.

Worked Balance Calculations

Worked Example: Balancing a Binary Confounder at One Time

Let $L$ be binary, with $P(L=1)=P(L=0)=0.5$. Suppose $P(A=1\mid L=1)=0.8$ and $P(A=1\mid L=0)=0.2$. Use unstabilized inverse probability weights and calculate the weighted prevalence of $L=1$ separately in each treatment group.

Among treated individuals, the joint probabilities for $L=1$ and $L=0$ are $0.5(0.8)=0.4$ and $0.5(0.2)=0.1$. Their weights are $1/0.8=1.25$ and $1/0.2=5$. The weighted masses are $0.4(1.25)=0.5$ and $0.1(5)=0.5$. Thus the weighted prevalence of $L=1$ among treated individuals is $0.5/(0.5+0.5)=0.5$.

Among untreated individuals, the joint probabilities for $L=1$ and $L=0$ are $0.5(0.2)=0.1$ and $0.5(0.8)=0.4$. Their weights are $1/0.2=5$ and $1/0.8=1.25$. The weighted masses are $0.1(5)=0.5$ and $0.4(1.25)=0.5$, again giving prevalence $0.5/(0.5+0.5)=0.5$. Weighting has removed the difference in this measured covariate in the population calculation.

Worked Example: Balance Within a Prior Treatment History

Within one fixed prior treatment history, suppose $P(L_t=1)=0.5$, $P(A_t=1\mid L_t=1)=0.75$, and $P(A_t=1\mid L_t=0)=0.25$. Let the stabilized numerator probability for current treatment be $q_t(1\mid\bar A_{t-1})=0.4$, so $q_t(0\mid\bar A_{t-1})=0.6$. Calculate the weighted covariate prevalence for each current treatment group.

For $A_t=1$, the joint probabilities for $L_t=1$ and $L_t=0$ are $0.5(0.75)=0.375$ and $0.5(0.25)=0.125$. The corresponding stabilized weight factors are $0.4/0.75$ and $0.4/0.25$. The weighted masses are $0.375(0.4/0.75)=0.2$ and $0.125(0.4/0.25)=0.2$. The weighted prevalence of $L_t=1$ is $0.2/(0.2+0.2)=0.5$.

For $A_t=0$, the joint probabilities are $0.5(0.25)=0.125$ for $L_t=1$ and $0.5(0.75)=0.375$ for $L_t=0$. The weight factors are $0.6/0.25$ and $0.6/0.75$, giving weighted masses $0.125(0.6/0.25)=0.3$ and $0.375(0.6/0.75)=0.3$. The weighted prevalence is $0.3/(0.3+0.3)=0.5$. The overall weighted masses differ between treatment groups, as the numerator probabilities allow; the covariate prevalence within this history is equal.

Balance Is Evidence, Not Identification

The balance calculation depends on having the correct treatment probabilities for the measured history and adequate positivity for the comparisons. A causal interpretation additionally relies on assumptions such as consistency, no interference, and sequential exchangeability: conditional on the observed history, treatment at each time must be independent of the relevant future potential outcomes. Positivity must also hold for the treatment strategies whose effects are being estimated. Balance on measured covariates does not test these assumptions, especially exchangeability.

Worked Example: Perfect Measured Balance With Residual Confounding

Let an unmeasured binary variable $U$ have probability $0.5$ of being one. Suppose $P(A=1\mid U=1)=0.8$ and $P(A=1\mid U=0)=0.2$. Let the measured covariate $X$ be constant for everyone, and let the outcome satisfy $Y=U$, regardless of treatment. Thus the individual causal effect is zero for everyone.

The treatment probability is $P(A=1)=0.5(0.8)+0.5(0.2)=0.5$. Among treated individuals, $P(U=1\mid A=1)=0.5(0.8)/0.5=0.8$, so $E[Y\mid A=1]=0.8$. Among untreated individuals, $P(U=1\mid A=0)=0.5(0.2)/0.5=0.2$, so $E[Y\mid A=0]=0.2$. The unadjusted difference is $0.8-0.2=0.6$, despite the true causal effect being zero.

Because $X$ is constant, its distribution is identical in the treatment groups before and after weighting. That balance is genuine but uninformative about the imbalance in $U$. Even if measured-covariate balance were perfect, it could not establish that all confounding had been controlled.

A practical reading of balance is therefore conditional. Poor balance in an important measured confounder is a warning: the fitted weights have not adequately reproduced the intended measured-covariate comparison. Good balance is reassuring about that specific measured comparison, but it neither verifies the causal assumptions nor proves that the treatment model is correct in every respect.

Diagnostics, Alternatives, and Limits

Assess balance using the covariates that inform treatment assignment and the substantive causal question. In longitudinal data, examine the covariates available before treatment at each time and compare current treatment groups within relevant prior treatment histories. A comparison that ignores prior history can mix two different issues: changes in the composition of treatment histories and current-time balance within a history.

When substantial differences remain, first check that the treatment probabilities and weights were constructed for the intended time and history. Then investigate whether a measured covariate or history component was omitted, whether the fitted treatment probabilities adequately represent the data, and whether sparse histories or extreme weights are making the comparison unstable. The earlier material on positivity diagnostics is relevant here: lack of support can make the desired weighted comparison unreliable or impossible. A small discrepancy is not automatically harmless, and a large discrepancy should not be hidden by a single overall summary.

Balance is complementary to, not interchangeable with, other approaches. Outcome regression adjusts for covariates through a model for the outcome; weighting instead constructs a treatment-comparable pseudo-population. Their failure modes differ, and either approach can be misspecified. Comparing weighted covariate summaries is more directly tied to the weighting target than judging treatment probabilities by how well they predict treatment. In finite samples, however, observed balance is only a diagnostic summary: it cannot reveal unmeasured confounding, repair structural lack of positivity, or guarantee a correctly specified causal model.

Key takeaway: Correct longitudinal treatment weights imply balance of measured pre-treatment covariates across current treatment values within prior treatment histories in the population. Observed balance supports the weighting procedure for those covariates; it is not proof of causal identification.

Check Your Understanding

Answer each question, then compare with the solutions.

In the balance identity, why is the comparison conditioned on prior treatment history?
In the first worked example, what weighted prevalence of $L=1$ is obtained among each treatment group?
Does the population balance identity require sequential exchangeability? What does it require about the treatment probabilities?
Can perfect balance on recorded covariates prove that there is no unmeasured confounding?

Solutions. (1) Stabilized numerator probabilities can depend on prior treatment. The identity guarantees current-time balance within a fixed prior history, not necessarily after mixing histories with different compositions. (2) It is $0.5$ in both groups: the treated weighted masses are $0.5$ and $0.5$, and the untreated weighted masses are also $0.5$ and $0.5$. (3) No. The identity requires the denominator probabilities to be the true conditional treatment probabilities, relevant probabilities to be positive, and the expectations to be finite. Sequential exchangeability is an additional causal-identification assumption. (4) No. Balance only concerns recorded covariates; an unmeasured common cause can remain imbalanced and confound the treatment-outcome association.