Covariate Balance After Weighting in a Diabetes Cohort

Why Balance Is a Diagnostic, Not a Verdict

In a diabetes cohort, treatment decisions may respond to recent glycemic control and renal function. A patient with rising HbA1c or declining estimated glomerular filtration rate (eGFR) may be more likely to start or intensify a particular therapy. Those histories can also predict later outcomes. In a marginal structural analysis, weighting aims to create a pseudo-population in which measured treatment predictors are balanced across treatment choices at each decision time.

Balance assessment asks whether the observed covariate histories are more alike after weighting. It does not establish that all confounding has been removed, that the weights are correctly modeled, or that a causal effect is identified. It is one important diagnostic alongside the design assumptions and the outcome analysis.

What to Compare at Each Decision Time

Let \(A_t\) denote treatment at time \(t\), and let \(L_t\) contain covariates measured by that time, including HbA1c and renal function. Write \(H_t=(L_0,A_0,L_1,\ldots,A_{t-1},L_t)\) for the observed history just before \(A_t\). A treatment decision can depend on several aspects of that history: the current HbA1c and eGFR, their earlier levels, and their recent changes.

For binary treatment, let \(w_i\) be the analysis weight for person \(i\) at the decision being assessed. In treatment group \(a\), the weighted mean of a covariate \(X\) is

The corresponding weighted variance uses the same normalized weights:

A signed standardized mean difference (SMD), defined here as group 1 minus group 0, is

The sign indicates the direction of the difference; its absolute value describes its size in pooled standard-deviation units. For categorical covariates, compare weighted proportions for each category. For a longitudinal history, assess more than one current-time mean: include prior measurements, changes or slopes, and clinically meaningful nonlinear transformations or interactions.

Why Inverse-Probability Weights Balance Measured Histories

The population identity below explains the diagnostic target. Let \(p_t(a\mid H_t)=P(A_t=a\mid H_t)\). For this result, use the unstabilized treatment weight \(W_t=W_{t-1}/p_t(A_t\mid H_t)\), where \(W_{t-1}\) is nonnegative and measurable with respect to \(H_t\). Assume treatment has a finite number of categories, \(p_t(a\mid H_t)>0\) on the histories considered, and \(0<E[W_{t-1}]<\infty\).

Proof. Conditional on \(H_t\), the probability of \(A_t=a\) is \(p_t(a\mid H_t)\), and on that event \(W_t=W_{t-1}/p_t(a\mid H_t)\). Thus,

The stated integrability condition makes the right-hand side integrable and the equality of finite expectations valid. Taking expectations proves the identity. In particular, setting \(g=1\) shows that every treatment category has the same weighted mass, \(E[W_{t-1}]\). Dividing the identity by this mass shows that the weighted distribution of \(H_t\) within each category has the same expectations for all such \(g\). Taking \(g\) to be indicators of measurable history sets gives equality of those weighted distributions. \(\square\)

This is a population result under the true treatment probabilities. In data, probabilities are estimated, weights are finite-sample quantities, and exact equality is not guaranteed. Residual imbalance can therefore be informative about finite-sample variability, limited overlap, or how the weighting procedure behaves.

What Stabilization Changes

A stabilized weight replaces the numerator 1 with a treatment probability depending on past treatment. Write \(q_t(a\mid\bar A_{t-1})\) for this numerator probability and define \(W_t^s=W_{t-1}^s q_t(A_t\mid\bar A_{t-1})/p_t(A_t\mid H_t)\). The comparison it motivates is conditional on a fixed past treatment history.

Derivation. Conditional on \(H_t\), the numerator \(q_t(a\mid b)\) is fixed on the event \(\bar A_{t-1}=b\). The conditional expectation of \(I(A_t=a)/p_t(a\mid H_t)\) is 1. Multiplying by the past weight and \(q_t(a\mid b)\), then taking expectations, gives the displayed identity. Taking \(g=1\) gives the weighted mass in that treatment-history stratum. Dividing the identity by that mass shows that the normalized weighted history distribution is the same across current treatment categories within the stratum. The integrability condition ensures these expectations are finite. \(\square\)

The numerator factor can make marginal comparisons across different past treatment histories look different, by design. Consequently, a diagnostic should state whether it compares current treatment groups within past-treatment strata or reports a marginal summary across those strata. The former more directly checks balance at the current decision.

Diagnostics, Alternatives, and Limitations

A useful assessment considers the full pattern, not a single cutoff. Inspect balance before and after weighting at every decision time; summarize current and lagged HbA1c and eGFR, clinically relevant transformations, and categorical predictors. Weighted empirical distributions or plots can reveal differences in spread, skewness, or tails even when means agree. Also inspect the weight distribution and effective sample size,

For equal weights, ESS equals the number of observations. Concentration in a few large weights reduces it, so apparently close weighted means may be unstable. Check that each treatment-history stratum has adequate representation and report how any truncation changes both weights and balance; truncation can reduce extreme weights while reintroducing imbalance.

Balance is not a test of the treatment model and there is no universal SMD threshold that guarantees validity. Good measured balance can coexist with unmeasured confounding, measurement error, or incorrect time ordering. Poor balance can flag inadequate overlap, model limitations, or finite-sample instability, but a diagnostic alone does not identify which cause is responsible. The causal interpretation additionally relies on consistency, sequential exchangeability conditional on the measured histories, positivity, and appropriate handling of loss to follow-up when relevant.

Matching can target similarity among selected individuals but may discard observations and does not automatically address changing treatment and confounder histories. Outcome regression instead models the outcome conditional on covariates; it can adjust measured differences under a suitable model but does not itself create a weighted balanced population. Weighting is useful for marginal contrasts, but balance diagnostics should be read with the estimand and assumptions in view, not as a certificate of causal validity.

Common Pitfalls

• Reporting only current covariate means. Similar current HbA1c does not imply similar prior HbA1c or eGFR trajectories. Add lagged values and relevant history summaries.
• Ignoring the sign convention. If SMD is group 1 minus group 0, keep that order in the numerator. Taking an absolute value is valid only when explicitly reporting an absolute SMD.
• Using ordinary integrability for weighted identities. For the finite-expectation identity above, require \(E[W_{t-1}|g(H_t)|]<\infty\); integrability of \(g\) alone does not ensure weighted expectations exist.
• Treating good balance as proof. Balance applies to measured variables and the observed weighted sample. It cannot verify the absence of unmeasured confounding or establish positivity across unsupported treatment choices.
• Pooling past treatment histories without explanation. Stabilized weights can retain associations with past treatment through their numerator. State whether current-treatment balance was assessed within past-treatment strata.