Perturbing Treatment Models to Reveal Instability

Why Perturb a Treatment Model?

A treatment model can look plausible while its resulting weights depend strongly on a small modeling choice. A covariate term, an interaction, or a time-specific coefficient may change a few fitted probabilities substantially; because longitudinal weights multiply contributions across visits, those changes can accumulate. Perturbation analysis makes this dependence explicit by changing a specified part of the model and measuring the resulting probability and weight changes on the same observed histories.

The aim is not to find a model that produces unchanged weights under every conceivable modification. Rather, it is to identify whether reasonable, controlled changes reveal instability that matters for the weighted analysis. The earlier material on sensitivity to treatment-model specification motivates comparing plausible specifications; here the emphasis is on quantifying a perturbation and tracing its effect through the weight.

Set Up a Controlled Perturbation

Let $A_t\in\{0,1\}$ denote treatment at visit $t$, and let $H_t$ denote the observed history available immediately before that treatment. Write the fitted denominator probability as $q_t(H_t)=P(A_t=1\mid H_t)$, with linear predictor $\eta_t(H_t)$, so that $q_t(H_t)=\operatorname{expit}\{\eta_t(H_t)\}$, where $\operatorname{expit}(u)=1/(1+e^{-u})$. A perturbation changes the predictor to $\widetilde{\eta}_t(H_t)=\eta_t(H_t)+\delta_t(H_t)$.

Definition: A treatment-model perturbation is a prespecified change $\delta_t(H_t)$ to the fitted linear predictor, or equivalently a prespecified change to the model terms or coefficients that produces such a change. The reference and perturbed models are evaluated on the same individuals and observed histories. Unless stated otherwise, the numerator model in a stabilized weight is held fixed, so measured changes arise from the denominator model.

For a logistic model with design vector $x_t(H_t)$ and coefficient vector $\beta$, one simple perturbation is $\widetilde{\beta}=\beta+\epsilon v$. Then $\delta_t(H_t)=\epsilon x_t(H_t)^\mathsf{T}v$. A perturbation can instead add a term, such as an interaction between a covariate and time, or replace a common time coefficient with time-specific coefficients. The change should be defined clearly enough that its implications for each history can be calculated.

For observed treatment $A_t$, the denominator probability contribution is $q_t(H_t)^{A_t}\{1-q_t(H_t)\}^{1-A_t}$. Over visits $t=0,\ldots,K$, define the denominator-only inverse-probability weight

W=\prod_{t=0}^{K}\frac{1}{q_t(H_t)^{A_t}\{1-q_t(H_t)\}^{1-A_t}}.

For a stabilized weight, the numerator factors multiply this expression. If those factors are held fixed, they cancel from the ratio of perturbed to reference weights. Thus the change in the denominator model can be studied without confusing it with a change in stabilization.

How Predictor Changes Affect Weights

Theorem: Suppose the denominator model is logistic and all fitted probabilities for the observed histories lie strictly between zero and one. Let $\widetilde{\eta}_t=\eta_t+\delta_t$, and keep any numerator factors fixed. Then $$ \log\left(\frac{\widetilde W}{W}\right) =-\sum_{t=0}^{K}\left[A_t\delta_t-\{b(\eta_t+\delta_t)-b(\eta_t)\}\right], \qquad b(u)=\log(1+e^u). $$ In particular, $$ \left|\log\left(\frac{\widetilde W}{W}\right)\right| \leq \sum_{t=0}^{K}|\delta_t|. $$

Proof. For one observed binary treatment, the log probability assigned by the logistic model is $A_t\eta_t-b(\eta_t)$. This follows directly from $\log q_t=\eta_t-b(\eta_t)$ when $A_t=1$, and $\log(1-q_t)=-b(\eta_t)$ when $A_t=0$. The log weight is the negative sum of these observed log probabilities, plus any fixed numerator contribution. Subtracting the reference log weight from the perturbed log weight gives the displayed identity.

To establish the bound, note that $b'(u)=\operatorname{expit}(u)\in(0,1)$. By the mean value theorem, $|b(\eta_t+\delta_t)-b(\eta_t)|\leq|\delta_t|$. Also, the derivative of the observed log probability $A_tu-b(u)$ is $A_t-\operatorname{expit}(u)$, whose absolute value is at most one. Applying the mean value theorem to that log probability gives an absolute change no greater than $|\delta_t|$ at visit $t$. Summing the visit-specific bounds and using the triangle inequality proves the result. $\square$

This bound is a screening tool, not a prediction that the weight change will equal the bound. It shows why modest predictor changes can add up across many visits, while the exact response also depends on the observed treatments and fitted probabilities.

Result: If $\eta_t(\epsilon)=\eta_t+\epsilon g_t$, then at $\epsilon=0$, $$ \left.\frac{d q_t(\epsilon)}{d\epsilon}\right|_{\epsilon=0} =q_t(1-q_t)g_t, \qquad \left.\frac{d\log W(\epsilon)}{d\epsilon}\right|_{\epsilon=0} =-\sum_{t=0}^{K}(A_t-q_t)g_t. $$

Derivation. The chain rule applied to $q_t(\epsilon)=\operatorname{expit}\{\eta_t+\epsilon g_t\}$ gives $dq_t/d\epsilon=q_t(\epsilon)\{1-q_t(\epsilon)\}g_t$; evaluating at zero gives the first expression. The log weight is the negative sum of $A_t\eta_t(\epsilon)-b\{\eta_t(\epsilon)\}$, apart from fixed terms. Differentiating gives $-\sum_t[A_t-b'\{\eta_t(\epsilon)\}]g_t$. Since $b'(\eta_t)=q_t$, evaluation at zero gives the second expression. These are local derivatives: for a finite perturbation, use the exact weight ratio rather than treating the derivative as the full change.

Worked Perturbations

Worked Example: Changes in Probabilities at Two Visits

Suppose the reference treatment probability is $0.30$ at each of two visits. A covariate-term perturbation changes it to $0.20$ at visit 0 and $0.45$ at visit 1. The observed treatments are $A_0=1$ and $A_1=0$. Calculate the change in the denominator-only weight and compare it with the logit-change bound.

At visit 0, the observed treatment is 1, so the weight contribution changes from $1/0.30$ to $1/0.20$. Its ratio is $0.30/0.20=1.5$. At visit 1, the observed treatment is 0, so the contribution changes from $1/0.70$ to $1/0.55$. Its ratio is $0.70/0.55\approx1.2727$. Therefore,

\frac{\widetilde W}{W} =\frac{0.30}{0.20}\frac{0.70}{0.55} \approx1.9091, \qquad \log(\widetilde W/W)\approx0.6466.

The predictor changes are the logit differences. At visit 0, $\operatorname{logit}(0.20)-\operatorname{logit}(0.30)\approx-0.5390$. At visit 1, $\operatorname{logit}(0.45)-\operatorname{logit}(0.30)\approx0.6466$. Their absolute values sum to approximately $1.1856$, so the bound holds: $0.6466\leq1.1856$. The actual log-weight change is smaller than the bound because the bound allows each visit to contribute as much as its full absolute predictor change.

Worked Example: A Covariate-Coefficient Perturbation

For one observed treatment, let the reference linear predictor be $0.2$. A continuous covariate has value $2$, and its coefficient is increased by $0.4$, with all other terms unchanged. The perturbed predictor is therefore $0.2+(0.4)(2)=1.0$. If the observed treatment is $A=1$, calculate the fitted probabilities and the ratio of inverse-probability contributions.

The reference probability is $\operatorname{expit}(0.2)\approx0.549834$; the perturbed probability is $\operatorname{expit}(1.0)\approx0.731059$. For an observed treatment of 1, the denominator contribution to the weight is $1/q$, so

\frac{\text{perturbed contribution}}{\text{reference contribution}} =\frac{1/0.731059}{1/0.549834} =\frac{0.549834}{0.731059} \approx0.752.

Thus a $0.4$ coefficient change produces a $0.8$ predictor change for this covariate value and reduces this weight contribution by about $24.8\%$. The effect depends on the covariate value: for a different value, the same coefficient perturbation would produce a different predictor change.

Worked Example: Adding a Covariate-by-Time Interaction

Suppose a pooled logistic model has predictor $\eta_t=-1+0.5L+0.4t$, where $L$ is a binary covariate and $t\in\{0,1\}$. Consider adding an interaction $0.6Lt$. For an individual with $L=1$, suppose $A_0=1$ and $A_1=0$. Compare the visit probabilities and the two-visit denominator weights.

At $t=0$, the interaction is zero. The predictor is $-1+0.5=-0.5$ in both models, giving $q_0=\operatorname{expit}(-0.5)\approx0.377541$. At $t=1$, the reference predictor is $-1+0.5+0.4=-0.1$, giving $q_1\approx0.475021$. With the interaction, it becomes $-0.1+0.6=0.5$, giving $\widetilde q_1\approx0.622459$.

The visit-0 treatment is 1 and its probability contribution is unchanged. At visit 1 the treatment is 0, so the reference and perturbed probability contributions are $1-0.475021=0.524979$ and $1-0.622459=0.377541$, respectively. Hence

\frac{\widetilde W}{W} =\frac{0.524979}{0.377541} \approx1.3905.

The added interaction changes the two-visit weight by about $39.1\%$ for this history. It has no effect for this covariate at $t=0$, illustrating why an interaction perturbation should be checked across times and covariate patterns rather than summarized by a single coefficient alone.

Choose Perturbations and Summarize Their Effects

A useful perturbation has a clear interpretation and is applied to the same observed histories under both models. Common choices include increasing or decreasing a covariate coefficient by a prespecified amount; adding a nonlinear basis term or a covariate interaction; allowing a covariate effect to vary with time; and replacing a common time pattern with separate time-specific terms. The scale of a coefficient change is meaningful only relative to the covariate scale, so report whether continuous covariates were centered or standardized.

For each perturbation, inspect both the predicted probabilities and the weights. Suitable summaries include the maximum and a high percentile of $|\widetilde q_t-q_t|$, the distribution of $\log(\widetilde W/W)$, and the number of observed histories for which the weight ratio is large. Looking at log ratios is convenient because multiplicative changes in weights become additive and ratios above and below one are treated symmetrically on the log scale. These summaries complement balance diagnostics: a small average probability change can conceal substantial changes for a subset of histories.

Specify the change.
State exactly which term, coefficient, or time structure changes, its scale, and which model components remain fixed.

Recompute matched predictions.
Evaluate both models for the same observed histories and record visit-specific probability differences.

Recompute weights.
Use the observed treatment at each visit to form the denominator contributions, then compare weight ratios or log-weight differences.

Locate consequential changes.
Check whether large changes occur for particular covariate values, times, or treatment histories, and compare with the intended diagnostic tolerances.

Interpretation, Alternatives, and Failure Modes

Perturbation analysis is a deterministic sensitivity diagnostic conditional on the fitted data and chosen changes. It differs from comparing model fit statistics: a model can fit treatment assignments well while producing materially different probabilities or weights under a plausible structural change. It also differs from assessing post-weighting balance, which evaluates whether weighting achieved its balancing purpose. Neither kind of diagnostic replaces the other.

A large response is a warning that weighted results may depend on the treatment-model specification, especially if the change affects many observed histories or creates very large weights. It is not, by itself, proof that the reference model is wrong. Conversely, small changes under a narrow set of perturbations do not prove correct specification: an unconsidered functional form or interaction could still matter. Perturbations should therefore reflect scientifically plausible alternatives, not arbitrary coefficient changes chosen after seeing a desired result.

Changing several things at once. If a covariate effect, interaction, and time pattern all change together, it becomes difficult to identify which choice drives the response. Start with interpretable one-at-a-time changes, then examine a small set of joint alternatives when justified.
Reporting only average probability changes. An average can hide a substantial shift for a small subgroup. Inspect ranges or upper quantiles and locate the histories producing the largest changes.
Confusing predictor change with probability change. The same $\delta_t$ need not imply the same probability difference at every baseline probability. The local probability derivative is $q_t(1-q_t)\delta_t$, which varies with $q_t$.
Ignoring the observed treatment. A probability change affects the weight differently for $A_t=1$ and $A_t=0$. Always use the probability assigned to the treatment actually observed when calculating that visit’s contribution.
Treating stability as validity. Similar weights across selected perturbations do not establish exchangeability, correct measurement, or positivity. Perturbation diagnostics address sensitivity to the changes examined, not every assumption behind a causal interpretation.

A large change in weights can arise because a model term is influential, because some fitted probabilities are near zero or one, or both. Examine the visit-level probabilities and the histories involved to distinguish these patterns. The appropriate response is to investigate specification and support, not to claim that a perturbation diagnostic alone identifies the source of failure.

Check Your Understanding

Work each problem using the observed treatment probability in the weight contribution. Complete solutions follow the questions.

A one-visit model changes $q$ from $0.40$ to $0.50$. Find the ratio of perturbed to reference weight contributions when $A=1$, and when $A=0$.
A predictor changes by $\delta=0.3$ at each of four visits. What upper bound does the log-weight result give for the absolute log weight ratio?
A coefficient on a covariate is perturbed by $0.2$. What predictor change results for a history with covariate value $3$? Does this alone determine the probability change?
Why can a small mean absolute probability change fail to reveal an important weight instability?

Solutions. (1) For $A=1$, the ratio is $0.40/0.50=0.8$. For $A=0$, it is $(1-0.40)/(1-0.50)=0.60/0.50=1.2$. (2) The bound is $\sum_t|\delta_t|=4(0.3)=1.2$, so $|\log(\widetilde W/W)|\leq1.2$. (3) The predictor change is $0.2(3)=0.6$. It does not alone determine the probability change; that also depends on the reference predictor, since the expit function is nonlinear. (4) A small subgroup may have large probability shifts that barely affect the mean but substantially change observed-treatment contributions and hence some weights. Inspecting upper-tail changes and the corresponding histories can reveal this pattern.