Sensitivity to Treatment-Model Specification

Why Refit Weights Under Alternative Specifications?

A weighted marginal structural model relies on estimated treatment probabilities. If a treatment model omits a relevant interaction, uses an inadequate time structure, or imposes an unsuitable functional form, its weights may leave covariates imbalanced or place excessive influence on a few observations. A single fitted model cannot reveal how dependent the conclusions are on those choices. Sensitivity analysis makes that dependence visible by refitting weights under a small set of substantively plausible specifications.

The comparison is informative only when its ingredients are held steady. The candidate models should address the same treatment decisions using the same observed histories and target the same causal contrast. Keep the sample, weight numerator, truncation rule if used, outcome model, and effect scale fixed. Change the treatment-model specification itself, for reasons grounded in subject-matter knowledge or the earlier diagnostics on functional form, interactions, and time structure. The purpose is not to search until one specification produces a preferred effect.

Definition: A treatment-model specification sensitivity analysis refits the treatment probabilities and corresponding inverse-probability weights under multiple substantively defensible models, then compares the resulting estimates of the same target effect and diagnostics of the resulting weights. The candidate specifications must preserve the estimand and analysis protocol; they are not different causal questions.

What Is Being Compared?

For a longitudinal study, let $A_t$ be treatment at time $t$, $L_t$ the measured covariates available by that time, and $H_t=(\bar L_t,\bar A_{t-1})$ the observed history before treatment $A_t$. A candidate specification $m$ estimates the probability of the observed treatment at each time, conditional on the history:

\widehat p_{it}^{(m)} = \widehat P_m(A_{it}\mid H_{it}).

For binary treatment, one common stabilized weight for person $i$ through time $K$ is

\widehat W_i^{(m)} = \prod_{t=0}^{K} \frac{\widehat P(A_{it}\mid \bar A_{i,t-1})} {\widehat P_m(A_{it}\mid H_{it})}.

In this comparison, use the same numerator model for every candidate $m$. The denominator is the part being varied. A one-time treatment analysis is the special case $K=0$. Once each candidate’s weights are refitted, compare three connected features: the estimated target effect, balance of the covariate history after weighting, and the distribution of the weights. Balance diagnostics should address covariates and time points relevant to the treatment assignment process, not just a convenient subset.

For example, the weighted mean of a baseline covariate $L$ in treatment group $a$ is

\bar L_{a,w}^{(m)} = \frac{\sum_i I(A_i=a)\widehat W_i^{(m)}L_i} {\sum_i I(A_i=a)\widehat W_i^{(m)}}.

A standardized difference or a distributional diagnostic can summarize the gap between treatment groups. Weight-tail summaries include the maximum, selected quantiles, and effective sample size,

\operatorname{ESS}^{(m)} = \frac{\left(\sum_i \widehat W_i^{(m)}\right)^2} {\sum_i \left(\widehat W_i^{(m)}\right)^2}.

ESS is a descriptive measure of weight concentration, not a substitute for a variance estimate. In a longitudinal analysis, inspect cumulative weights: moderate contributions at several times can multiply into an extreme final weight.

Why Balance and Effect Estimates Belong Together

The following population identity explains why balance is relevant. It is stated for a single binary treatment and a correctly specified propensity score $e(L)=P(A=1\mid L)$, with $0<e(L)<1$ almost surely. Let $p=P(A=1)$, and suppose $g(L)$ is integrable. The stabilized weights for treated and untreated individuals are $p/e(L)$ and $(1-p)/(1-e(L))$, respectively.

Result: Under the stated conditions, weighting by the true stabilized propensity weights gives the same weighted expectation of $g(L)$ in each treatment group: $$ \frac{E\!\left[A\frac{p}{e(L)}g(L)\right]}{E\!\left[A\frac{p}{e(L)}\right]} = E[g(L)] = \frac{E\!\left[(1-A)\frac{1-p}{1-e(L)}g(L)\right]}{E\!\left[(1-A)\frac{1-p}{1-e(L)}\right]}. $$

Proof. By the definition of the propensity score, $E[A\mid L]=e(L)$. Iterated expectation therefore gives

E\!\left[A\frac{p}{e(L)}g(L)\right] = E\!\left[E[A\mid L]\frac{p}{e(L)}g(L)\right] = pE[g(L)].

Taking $g(L)=1$ in the same calculation gives $E[A p/e(L)]=p$. The normalized treated weighted expectation is thus $pE[g(L)]/p=E[g(L)]$. For controls, $E[1-A\mid L]=1-e(L)$, so

E\!\left[(1-A)\frac{1-p}{1-e(L)}g(L)\right] =(1-p)E[g(L)], \qquad E\!\left[(1-A)\frac{1-p}{1-e(L)}\right]=1-p.

Their ratio is also $E[g(L)]$. This proves the identity. $\square$

This population calculation does not promise exact balance in a finite sample with estimated probabilities. It shows why persistent imbalance under a candidate specification is a warning: its weights may not adequately reproduce the covariate distribution relevant to treatment assignment. For longitudinal weights, the corresponding rationale is sequential: conditional treatment probabilities at each decision are used to remove dependence on the measured history at that decision. The identity does not establish that unmeasured confounding is absent or that the causal assumptions hold.

Worked Comparisons

Worked Example: Balance Changes the Interpretation of an Effect

Consider a one-time treatment study with a binary covariate $L$. There are four people with $L=0$: three treated, with outcomes 8, 10, and 12, and one untreated, with outcome 2. There are four with $L=1$: one treated, with outcome 14, and three untreated, with outcomes 4, 6, and 8. Compare a model that assigns every person propensity $0.5$ with a model saturated in $L$, which assigns $e(0)=0.75$ and $e(1)=0.25$. Both use $p=0.5$ in the numerator.

Under the constant-propensity model, every stabilized weight is $0.5/0.5=1$. The treated mean outcome is $(8+10+12+14)/4=11$, and the untreated mean is $(2+4+6+8)/4=5$. The weighted contrast is $11-5=6$. But the treated proportion with $L=1$ is $1/4=0.25$, while the untreated proportion is $3/4=0.75$.

Under the model saturated in $L$, treated weights are $0.5/0.75=2/3$ for $L=0$ and $0.5/0.25=2$ for $L=1$. Control weights are $0.5/(1-0.75)=2$ for $L=0$ and $0.5/(1-0.25)=2/3$ for $L=1$. Each treatment group has total weight 4. In each group, the weighted total for $L=1$ is 2, so the weighted proportion is $2/4=0.5$.

The treated weighted outcome mean is $\{(8+10+12)(2/3)+14(2)\}/4=(20+28)/4=12$. The control weighted mean is $\{2(2)+(4+6+8)(2/3)\}/4=(4+12)/4=4$. The contrast is $12-4=8$. The estimates differ by 2, and the candidate with better balance also changes the weighted contrast. This example illustrates sensitivity; by itself, it does not prove that the saturated model is correct or that 8 is the true causal effect.

Worked Example: A Product of Weights Can Create a Tail

Suppose four observed treatment histories each have two treatment decisions. For clarity, use a common numerator probability of $0.5$ at each decision, so each numerator product is $0.25$. Candidate model A assigns an observed-treatment probability of $0.5$ at both decisions for all four people. Its weight for each is $0.25/(0.5\times0.5)=1$.

Candidate model B assigns observed-treatment probabilities $(0.8,0.8)$, $(0.8,0.2)$, $(0.2,0.8)$, and $(0.2,0.2)$ to the four histories. The respective denominator products are $0.64, 0.16, 0.16,$ and $0.04$. The weights are therefore $0.25/0.64=0.390625$, $0.25/0.16=1.5625$, $1.5625$, and $0.25/0.04=6.25$.

For model A, the maximum weight is 1 and ESS is $4^2/4=4$. For model B, the sum of weights is $9.765625$, and the sum of squared weights is approximately $44.0979$. Thus ESS is $9.765625^2/44.0979\approx2.16$, out of four people. Model B’s smallest assigned probabilities create a much more concentrated weight distribution. That is a reason to investigate the specification and support of the data; it is not, alone, a reason to prefer model A. The example makes no balance or effect claim, since those require covariates and outcomes for the histories.

Worked Example: Read a Three-Model Sensitivity Summary

Suppose three plausible denominator specifications are fitted on the same study sample: a linear predictor, a prespecified nonlinear term, and an interaction justified by the treatment process. Each uses the same numerator and weighted outcome analysis. The fitted summaries are as follows.

Specification	Effect estimate	Largest absolute standardized difference	Maximum weight	ESS
Linear predictor	7.8	0.08	9.0	420
Nonlinear term	8.4	0.05	12.0	360
Interaction	7.5	0.16	28.0	190

The effect-estimate range is $8.4-7.5=0.9$. The nonlinear model has the smallest reported maximum imbalance, $0.05$, but also a lower ESS than the linear model. The interaction model has the largest reported imbalance, the largest maximum weight, and the smallest ESS. These summaries suggest that the interaction specification deserves closer diagnostic review rather than automatic selection or rejection. Check balance across the relevant covariates and histories, and inspect whether the high weights are supported by the observed treatment probabilities.

The estimate range is not a confidence interval: it records variation across specifications, not sampling uncertainty. Nor does the smallest imbalance guarantee a correct causal model. The comparison is useful because it places changes in the target-effect estimate beside evidence about balance and weight concentration.

How to Conduct and Interpret the Sensitivity Analysis

Specify plausible alternatives.
State the substantive reason for each form, interaction, or time-structure choice. Do not include arbitrary alternatives solely because they move the effect estimate.

Keep the target and analysis fixed.
Use the same study population, treatment histories, numerator, outcome analysis, and effect scale. Refit the denominator treatment model for each candidate.

Check balance and tails.
Compare standardized differences and, where appropriate, distributional or history-specific balance. Report weight quantiles, extremes, and ESS; do not rely on one summary alone.

Interpret the joint pattern.
Ask whether effect changes accompany improved or worsened balance, or a more concentrated weight distribution. Treat disagreement as a sensitivity finding, not a model-selection rule.

A common mistake is to choose whichever model gives the most plausible-looking treatment effect, then describe its balance as confirmation. This uses the outcome to select the weighting model and can obscure the very sensitivity the analysis is meant to disclose. A better approach is to define the plausible specifications and diagnostic criteria before interpreting the effect estimates, then report the full pattern.

Another mistake is to treat good post-weighting balance as proof of correct specification. Balance is an observed-data diagnostic: it cannot rule out unmeasured confounding, incorrect temporal ordering, or violations of other causal assumptions. Conversely, poor balance under every candidate may point to inadequate support, omitted history, or a need to revisit the candidate set; it does not establish that one more complex model will solve the problem. The earlier material on post-weighting balance provides the diagnostic measures; here the key is to compare them across defensible refits.

Sensitivity analysis has limits. It depends on which alternatives are considered plausible, and similar results across a narrow set do not guarantee robustness to every form of misspecification. Different candidates may also trade balance against weight concentration. Report the candidates, the shared analysis choices, and the diagnostics alongside the effect estimates. Sampling uncertainty remains a separate issue: a small difference between point estimates should not be declared negligible without considering uncertainty, while a large difference should prompt investigation rather than selective reporting.

Check Your Understanding

Answer each question, then compare with the brief solutions.

When comparing two treatment-model specifications, name three analysis choices that should ordinarily be held fixed.
A candidate model produces excellent balance but an ESS far below the sample size. What does that combination tell you, and what does it not tell you?
With the true propensity $e(L)$, explain why the treated stabilized weighted expectation of $g(L)$ equals $E[g(L)]$.
Why is the range of effect estimates across candidate specifications not a confidence interval?

Solutions. (1) Keep the target population and effect scale, numerator model, and outcome analysis fixed; also use the same sample and any prespecified truncation rule. (2) The observed covariate balance is good under that weighting, but the weights are concentrated and may yield unstable estimates. ESS is a concentration diagnostic, not proof that the model is correct or that the effect is biased. (3) Since $E[A\mid L]=e(L)$, iterated expectation gives $E[A\{p/e(L)\}g(L)]=pE[g(L)]$, while the denominator is $E[A\{p/e(L)\}]=p$; their ratio is $E[g(L)]$. (4) The range describes variation across chosen specifications. It does not quantify sampling uncertainty under a specified estimator and sampling distribution.