Functional-Form Diagnostics for Treatment Models

Why Functional Form Needs Its Own Diagnostics

A treatment model can rank people well and still describe the effect of a continuous covariate poorly. For example, a model with a straight-line effect of age on the log odds of treatment may predict probabilities that are systematically too high in one age range and too low in another. A single overall fit summary can obscure that pattern. In a marginal structural model (MSM), such errors matter because estimated treatment probabilities determine the weights used to create a pseudo-population.

The target here is the functional form for a continuous covariate, not the full set of model choices. Residual patterns, calibration within covariate ranges, and post-weighting balance provide complementary evidence. None proves that a treatment model is correct: these are diagnostics, and their interpretation depends on support, sample size, and the history included in the model.

For an interpretable diagnostic, the fitted probability should be evaluated on the same treatment occasion and history as the observed \(A_t\). The model must at least use a well-defined pre-treatment history, and the probabilities used in inverse-probability weights must be positive for the treatment choices represented in the target population. If the histories have little support in a region of \(X_t\), apparent patterns there can reflect sparse data or extrapolation rather than a clearly identified shape error.

Residual Patterns and Their Direction

A positive residual means the observed treatment was 1 while the fitted probability was below 1, or that the model assigned a probability smaller than the observed treatment indicator. Averaging residuals over many observations makes the comparison more useful: within a covariate range, the average residual is the observed treatment proportion minus the average fitted probability. A positive average therefore signals underprediction in that range; a negative average signals overprediction.

A plot of residuals against \(X_t\), or a smooth of their conditional average, can reveal a trend or curvature that a single average conceals. For a correctly specified conditional probability, the population residual has mean zero conditional on the full history. In practice, a binned or smoothed residual plot is a noisy, lower-dimensional check: it asks whether the model has systematic errors associated with the continuous covariate, not whether every detail of the conditional model is correct.

Calibration Within Covariate Ranges

Calibration compares observed treatment frequencies with fitted probabilities. Overall calibration checks whether the average observed treatment frequency is close to the average predicted probability. For functional form, the more revealing comparison is often made within ranges of \(X_t\), or using a smooth calibration curve against \(X_t\). A model can match the overall frequency while missing the way treatment probability changes across the covariate.

To verify this identity, condition first on the full history. Because \(E[A_t\mid H_t]=q_t(H_t)\) and \(\hat q_t(H_t)\) is determined by that history, $$ E[R_t\mid H_t]=E[A_t-\hat q_t(H_t)\mid H_t]=q_t(H_t)-\hat q_t(H_t). $$ Taking conditional expectations of both sides given \(B\), and using that \(B\) is determined by \(H_t\), gives the result. In data, the group mean residual estimates the left side; it is not the unknown true discrepancy itself. \(\square\)

Weighted Balance as a Functional-Form Check

Another diagnostic asks whether weighting by the fitted treatment probabilities makes covariates comparable across treatment groups. For one treatment occasion, define the unstabilized inverse-probability weights \(A_t/\hat q_t(H_t)\) for treated observations and \((1-A_t)/(1-\hat q_t(H_t))\) for untreated observations. If the true probability \(q_t(H_t)\) is used, weighting balances any integrable function of the pre-treatment history in expectation.

For the treated identity, condition on \(H_t\). Since \(h(X_t)\) and \(q_t(H_t)\) are determined by \(H_t\), $$ E\left[\left.\frac{A_t h(X_t)}{q_t(H_t)}\right|H_t\right] =\frac{h(X_t)}{q_t(H_t)}E[A_t\mid H_t] =h(X_t). $$ Taking expectations gives the first equality. For the untreated identity, \(E[1-A_t\mid H_t]=1-q_t(H_t)\), so $$ E\left[\left.\frac{(1-A_t)h(X_t)}{1-q_t(H_t)}\right|H_t\right] =\frac{h(X_t)}{1-q_t(H_t)}(1-q_t(H_t)) =h(X_t). $$ Taking expectations gives the second equality. The integrability condition ensures that these expectations exist. \(\square\)

With fitted probabilities, substantial imbalance in \(X_t\) or in functions of \(X_t\) can be evidence that the model's covariate effect is inadequate. A balance check should not stop at the mean: \(h(X_t)=X_t\) checks the mean, while choices such as \(X_t^2\) or indicators for covariate ranges probe other features. This is a diagnostic principle, not a guarantee that all relevant confounding has been controlled.

Choosing and Comparing Functional Forms

A practical comparison might begin with a linear term for \(X_t\) in the treatment model and then consider a scientifically plausible alternative, such as a quadratic term or a prespecified transformation. Each candidate should be judged using the same target population and treatment history. Compare residual or calibration patterns across \(X_t\), and inspect weighted balance for \(X_t\) and relevant functions of it. A small improvement in likelihood or discrimination alone does not demonstrate improved balance.

Diagnostics are most informative when the covariate has enough observations across its range. Displaying range-specific sample sizes, treatment counts, and uncertainty helps distinguish a persistent pattern from noise. Extreme fitted probabilities also deserve attention: they can create unstable weights, so a model that appears to improve balance by assigning extreme probabilities may worsen the reliability of the weighted analysis.

No single diagnostic dominates. Residual plots and calibration examine probability errors directly; weighted balance examines a consequence of those errors for the weighted pseudo-population. Flexible forms can reduce visible discrepancies, but they may overfit sparse regions. A model should not be chosen solely because it makes one balance statistic look favorable, and passing these checks does not establish exchangeability, positivity, or correct specification of every covariate effect.

Common Pitfalls and Limitations

• Reversing residual signs. Since \(R_t=A_t-\hat q_t(H_t)\), a positive group average means underprediction, while a negative average means overprediction.
• Relying on the overall average. Opposing errors across covariate ranges can cancel, leaving an overall residual mean near zero.
• Treating balance as proof. A small difference in the weighted mean of \(X_t\) does not guarantee balance in its distribution, in other history variables, or in unmeasured confounders.
• Overinterpreting noisy curves. Sparse support and random variation can produce apparent bends. Conversely, in-sample fitting can make residual summaries look reassuring, especially when model fitting directly targets related averages.
• Confusing fit with the causal target. A better-fitting treatment model is not automatically a better causal analysis; probabilities must be appropriate for the weighting estimand and the observed history.

These diagnostics flag reasons to investigate and compare specifications; they do not select a unique true model from data. If plausible forms lead to meaningfully different weighting behavior, report that sensitivity rather than hiding it behind one preferred fit.