Diagnostics For Treatment Variation And Positivity

Why Treatment Variation Matters

G-estimation uses how treatment varies across observed histories to separate treatment effects from the treatment-free outcome. If people with a particular history almost always receive the same treatment, the data provide little direct comparison of treatment choices for that history. The resulting problem is not merely that a fitted model may be imprecise: some treatment contrasts may be weakly supported by the observed data.

The relevant question is therefore not simply whether both treatments occur somewhere in the study. It is whether treatment varies among people with comparable histories, and whether that variation is sufficient for the blip parameters being estimated. Diagnostics can reveal warning signs, but no finite-sample summary proves that the causal assumptions hold.

For a continuous treatment, the corresponding requirement concerns support: the conditional distribution of treatment given history must have density over treatment values needed for the contrast. A continuous-treatment density that is very small in a relevant region signals weak support there. A binary propensity threshold is not a general diagnostic for continuous treatment.

In longitudinal settings, each treatment decision is assessed conditional on its own pre-treatment history. A favorable overall treated-versus-untreated count can conceal a history with almost no treated individuals, or a history with almost no untreated individuals. Diagnostics should therefore be organized by time and by meaningful features of \(H_t\), not only by marginal treatment frequencies.

Treatment Probabilities and Residual Variation

For binary treatment, the treatment residual at a history \(H_t\) is \(A_t-p_t(H_t)\). Conditional on that history, its mean is zero and its variance is

This variance is largest at \(p_t(H_t)=0.5\), where it equals \(0.25\), and approaches zero as the probability approaches either zero or one. Thus, near-deterministic treatment assignment leaves little residual treatment variation for a conditional moment equation to use.

In a simple one-time additive structural nested model, suppose the blip is \(\psi(H;\beta)\), and define the blip-down outcome \(Y^0(\beta)=Y-A\psi(H;\beta)\). Under consistency, sequential exchangeability given \(H\), and a correctly specified blip model, the treatment-free outcome at the true parameter is conditionally mean-independent of treatment given \(H\). If \(s(H)\) is a vector of functions used to identify the blip parameters, a corresponding population moment is

In practice, the treatment probability is usually estimated, and the moment is solved using observed data and a blip-down outcome calculated at candidate parameter values. In a longitudinal analysis, analogous moments are used at successive treatment times with the relevant history and recursively blipped-down outcome. The diagnostic implication is local: weak variation at a time and history can weaken the moment information for parameters that depend on that treatment decision.

Why Weak Variation Reduces Information

A short calculation shows the connection. For a scalar parameter with blip \(\psi(H;\beta)=\beta s(H)\), the blip-down outcome is \(Y-A\beta s(H)\). Treating the true treatment probability as known for this calculation, the derivative of the population moment with respect to \(\beta\) is

Conditional on \(H\), \(E[(A-p(H))A\mid H]=p(H)\{1-p(H)\}\). Therefore,

This is a measure of how strongly the moment changes with the parameter. When treatment probabilities are near zero or one for histories where \(s(H)\) is nonzero, this quantity can be small. For a vector of parameters, the corresponding local information involves a matrix such as \(E[p(H)\{1-p(H)\}s(H)s(H)^\mathsf{T}]\). If that matrix is nearly singular, some parameter combinations are weakly identified by treatment variation. This calculation does not by itself establish a full sampling variance formula; it explains why limited variation can lead to unstable estimates and wide uncertainty intervals.

Diagnostics for Model-Relevant Variation

Marginal probabilities and stratum counts are a starting point. They should be accompanied by diagnostics that reflect which histories and blip parameters contribute to the estimating equation. In a simple model with \(s(H)=1\), the amount of treatment-residual variation is summarized by \(p(H)\{1-p(H)\}\). With effect modification, variation must also occur in the histories that distinguish the corresponding components of \(s(H)\). A large sample can still provide weak information about a parameter if the histories that define that parameter have little treatment overlap.

For a practical analysis, inspect probability distributions by treatment time, key history strata, and the covariates used in the blip functions. Look for empty or nearly empty treatment cells, extreme fitted probabilities, and poor overlap in histories important to the target contrast. When the model has several parameters, inspect whether the treatment-residual-weighted design is close to rank deficient; a single minimum and maximum propensity cannot reveal that structure.

These checks depend on the fitted treatment model. A misspecified probability model can hide or create apparent extremes, and fitted probabilities are estimated rather than known. Report how the probabilities were obtained, the history variables included, relevant counts, and any limitations of the model. In longitudinal analyses, repeat the assessment at each treatment time: a problem at one decision point can limit what is learned about blips involving that time.

Weight Summaries as a Complementary Warning

Inverse-probability weights provide another view of limited treatment support. For binary treatment at one time, an unstabilized weight for an individual is

Thus, a treated person receives weight \(1/p(H_i)\), and an untreated person receives weight \(1/\{1-p(H_i)\}\). A very small fitted probability for the treatment actually received can produce a large weight. Although ordinary g-estimation does not require this weighting formula as its estimator, weight summaries can flag histories in which observed treatment choices are unusual under the fitted assignment model.

One common summary is the effective sample size of nonnegative weights:

It equals the actual sample size when all weights are equal and becomes smaller as weight is concentrated in a few observations. It is a descriptive summary, not a replacement for the actual sample size or a complete measure of information for g-estimation.

Interpreting the Diagnostics

Positivity is a condition about the data-generating treatment probabilities and the target population. Estimated probabilities, counts, residual variances, and weights are finite-sample diagnostics for possible problems with that condition. No observed data set can establish that the true conditional probabilities are bounded away from zero and one for every relevant history. Conversely, a few extreme fitted probabilities do not by themselves prove that identification has failed; they call for careful examination of the histories, treatment model, and target contrast.

The practical consequence of weak variation is usually limited information and potentially unstable estimation. In a structural nested model, the effect for a history region with almost no observed treatment alternatives may rely heavily on model structure and extrapolation rather than direct comparisons. A reported estimate can still be computed, but its numerical existence should not be confused with strong empirical support for every component of the blip.

Diagnostics should be interpreted alongside the assumptions needed for causal interpretation, including consistency, sequential exchangeability, and an adequately specified treatment model for the moment equation being used. Positivity cannot compensate for unmeasured confounding, and a good-looking propensity distribution does not verify exchangeability. Likewise, treatment-variation diagnostics address support and information, not whether the chosen blip model correctly represents the treatment effect.

Common Mistake

Another error is to treat a conventional propensity cutoff as a theorem. A value such as 0.05 can be a useful flag, but no universal cutoff separates valid from invalid analyses. The consequences depend on sample size, history structure, the target population, and which contrasts are being estimated. Report the pattern and its implications rather than relying on a single threshold.