Why interactions matter in treatment models
A treatment model can include each covariate as a separate main effect and still fail to describe how treatment is assigned. For example, the association between a patient’s health status and treatment may differ across age groups. A model that omits the age-by-health interaction forces the health association to be the same at every age. If that restriction is wrong, the fitted treatment probabilities can be inaccurate for particular covariate combinations even when the model appears plausible overall.
In a marginal structural model, treatment probabilities are used to form weights intended to create a weighted population in which measured treatment predictors are balanced. Interaction diagnostics therefore ask a focused question: does a selected interaction capture treatment-assignment structure that the main-effects model misses? The relevant evidence is not only a change in model fit. It is also whether conditional calibration and weighted balance improve without producing unacceptable instability.
For a logistic model, a main-effects specification might be \(\operatorname{logit} q_t(H_t)=\beta_0+\beta_X X+\beta_Z Z\). An interaction-augmented candidate adds \(\beta_{XZ}XZ\). The interaction should be chosen for substantive or diagnostic reasons, not added indiscriminately. The comparison is meaningful only where the relevant covariate combinations are observed with both treatment values. If positivity fails, a fitted interaction cannot supply information about an unobserved treatment contrast.
What correct probabilities imply for weighted balance
The reason to examine balance is expressed by a basic identity. Consider one treatment decision and suppress the time subscript. Let \(H\) be the measured history, \(A\) a binary treatment, \(p(H)=P(A=1\mid H)\), and \(h(H)\) any measurable function with \(E[|h(H)|]<\infty\). Assume \(0<p(H)<1\) almost surely. Then inverse-probability weighting by the true probabilities gives the same expected total of \(h\) in the treated and untreated weighted groups.
Proof. Conditional on \(H\), \(h(H)\) and \(p(H)\) are fixed and \(E[A\mid H]=p(H)\). Therefore $$ E\left[\left.\frac{A h(H)}{p(H)}\right|H\right] = \frac{h(H)}{p(H)}E[A\mid H] = h(H). $$ Taking expectations and applying the law of iterated expectations gives \(E[A h(H)/p(H)]=E[h(H)]\). Also \(E[1-A\mid H]=1-p(H)\), so $$ E\left[\left.\frac{(1-A)h(H)}{1-p(H)}\right|H\right] = \frac{h(H)}{1-p(H)}\{1-p(H)\} = h(H). $$ Taking expectations proves the second equality. The integrability condition ensures these expectations are defined. \(\square\)
With a misspecified probability \(q(H)\), the corresponding difference in weighted totals is $$ E\left[\frac{A h(H)}{q(H)}-\frac{(1-A)h(H)}{1-q(H)}\right] = E\left[\frac{h(H)\{p(H)-q(H)\}}{q(H)\{1-q(H)\}}\right], $$ provided the terms are integrable and \(0<q(H)<1\) almost surely. This follows by conditioning on \(H\): the two conditional expectations are \(h(H)p(H)/q(H)\) and \(h(H)\{1-p(H)\}/\{1-q(H)\}\), whose difference simplifies to the expression shown. Thus a probability error that varies with an interaction can leave a corresponding interaction function unbalanced.
These are population identities, not guarantees that every finite-sample balance statistic will equal zero. Estimated probabilities also need not produce exact balance just because they came from logistic regression. In particular, the likelihood score equations do not generally set the inverse-probability-weighted treated and untreated totals equal for every included covariate function.
Worked examples
Worked Example: An Omitted Interaction Creates Cell-Specific Error
Suppose \(X\) and \(Z\) are binary, with treatment probabilities \(p_{00}=0.20\), \(p_{10}=0.30\), \(p_{01}=0.30\), and \(p_{11}=0.80\). The subscripts give the values of \((X,Z)\). Does a main-effects logistic model represent these probabilities?
The logits for the first three cells are \(\operatorname{logit}(0.20)=-1.3863\), \(\operatorname{logit}(0.30)=-0.8473\), and \(\operatorname{logit}(0.30)=-0.8473\). If a main-effects model matches these three logits, its intercept is \(-1.3863\), and each main-effect coefficient is \(-0.8473-(-1.3863)=0.5390\). It must therefore predict a logit of \(-1.3863+0.5390+0.5390=-0.3083\) in cell \((1,1)\), or a probability of about \(0.4235\), not \(0.80\).
The four-cell interaction contrast on the logit scale is \(1.3863-(-0.8473)-(-0.8473)+(-1.3863)=1.6946\), which is not zero. A no-interaction logistic model requires this contrast to be zero, so these probabilities cannot all be represented by that model.
For 100 people in cell \((1,1)\), suppose 80 receive treatment and 20 do not, consistent with the stated probability. Using the misspecified prediction \(q=0.4235\), the treated weighted total is \(80/0.4235\approx188.9\), and the untreated weighted total is \(20/(1-0.4235)\approx34.7\). Using the true \(p=0.80\), those totals are \(80/0.80=100\) and \(20/0.20=100\). The discrepancy shows why balance should be checked in the covariate combinations implicated by an interaction, rather than only in aggregate.
Worked Example: Overall Calibration Can Hide Interaction Errors
Let \(X,Z\) each have four equally sized cells. A main-effects logistic candidate with \(\operatorname{logit}q=-1.3863+0.9808X+0.9808Z\) predicts probabilities approximately \(0.20,0.40,0.40,0.64\) in cells \((0,0),(1,0),(0,1),(1,1)\). Suppose 100 people are observed in each cell, with treatment counts \(5,55,25,79\), respectively.
The overall observed treatment fraction is \((5+55+25+79)/400=164/400=0.41\). The model’s average predicted probability is \((0.20+0.40+0.40+0.64)/4=0.41\) as well. Thus aggregate calibration is exact in this example.
The cell-specific observed rates are \(0.05,0.55,0.25,0.79\), however, while the predictions are \(0.20,0.40,0.40,0.64\). The errors are \(-0.15,+0.15,-0.15,+0.15\). The positive and negative errors cancel in the overall average. A calibration plot or summary that pools all four cells could therefore look satisfactory while missing substantial conditional miscalibration. A diagnostic should inspect predictions across the combinations relevant to the proposed interaction, while also checking that those combinations contain enough observations.
Worked Example: Why an Interaction Is Not Always an Improvement
Suppose the true probabilities follow \(\operatorname{logit}p=-1+0.5X+0.5Z\). The four logits are \(-1,-0.5,-0.5,0\), so the probabilities are approximately \(0.269,0.378,0.378,0.500\). Their interaction contrast is \(0-(-0.5)-(-0.5)+(-1)=0\). The data-generating relationship has no logit-scale interaction.
If all four cells are well represented, a main-effects model expresses this pattern directly. Adding an unnecessary interaction can increase sampling variability; in a small sample, its fitted coefficient may produce very high or low probabilities in a cell with few observations. Those probabilities can create extreme inverse-probability weights. The diagnostic goal is not to prefer the most complicated model, but to see whether a selected interaction resolves a meaningful pattern of conditional miscalibration or imbalance without undermining support and weight stability.
A focused diagnostic comparison
For each treatment decision under study, compare the current specification with a candidate that adds a prespecified interaction. Keep the observations and other model choices fixed so the comparison isolates the interaction. Then examine both the treatment-probability predictions and the balance induced by the resulting weights.
Choose covariates whose joint pattern could affect treatment assignment, such as disease severity by age group. Define the interaction before inspecting a large number of alternatives.
Tabulate treatment and non-treatment counts across the covariate combinations. Empty or very small cells make conditional comparisons unreliable and can lead to near-zero or near-one predicted probabilities.
Within meaningful strata or plotted groupings, compare observed treatment frequencies with predicted probabilities under each model. Overall calibration alone can conceal opposite errors that cancel across cells.
Assess whether the interaction-augmented weights reduce imbalance for the interaction and its component covariates, including relevant distributions rather than only their means.
Review predicted probabilities and weights for instability. Treat improvement as evidence about this model diagnostic, not proof that all treatment assignment has been captured.
There is an exact finite-sample special case that clarifies the role of cell-specific checks. Suppose a model assigns each covariate cell \(c\) its observed treatment fraction \(\hat p_c=n_{1c}/n_c\), where \(n_{1c}\) and \(n_{0c}\) are the treated and untreated counts and both are positive. For the indicator \(h_c\) of that cell, $$ \sum_i\frac{A_i h_c(H_i)}{\hat p_c} = \frac{n_{1c}}{n_{1c}/n_c} = n_c, \qquad \sum_i\frac{(1-A_i)h_c(H_i)}{1-\hat p_c} = \frac{n_{0c}}{n_{0c}/n_c} = n_c. $$ Thus saturated cell probabilities produce exact balance of cell indicators. This is a mathematical property of fitting the observed cell fractions, not evidence of good generalization: with many sparse cells, those fractions can be highly variable, and no estimate exists for an empty treatment group.
Interpreting the evidence and avoiding common errors
Calibration and balance answer related but different questions. Calibration asks whether predicted probabilities agree with observed treatment frequencies in specified groups. Balance asks whether the weights make measured covariates comparable across treatment groups. A model can look calibrated in broad groups but leave a particular interaction unbalanced; conversely, good balance on selected summaries does not establish calibration throughout the history space.
A common mistake is to treat a high overall calibration score, good discrimination, or a small change in likelihood as sufficient evidence that interactions are unnecessary. Those summaries do not guarantee balance in the covariate combinations that determine treatment assignment. The correction is to inspect conditional calibration and weighted balance for the selected interaction, alongside probability and weight stability. Another mistake is to add many interactions until balance looks good. That can exploit random variation, especially in sparse cells, and produce unstable weights. Interpret the comparison with the support and positivity diagnostics in mind.
These diagnostics are also limited by what is observed. An interaction check cannot detect an important unmeasured predictor, certify the causal assumptions, or make an unsupported treatment contrast estimable. The comparison is a specification diagnostic, not a test that proves the treatment model correct. In longitudinal analyses, the treatment probabilities and weights concern histories at each treatment decision; evidence at one decision does not by itself establish adequate specification at all others.
Check Your Understanding
Answer each question, then compare with the solutions below.
- What does a no-interaction logistic model imply about the four logits for binary \(X\) and \(Z\)?
- Why does agreement between the overall observed and predicted treatment fractions not rule out interaction misspecification?
- For a history value with true treatment probability \(p\) and fitted probability \(q\), what is the conditional contribution to the difference between treated and untreated weighted totals for \(h(H)\)?
- Why can an interaction-augmented model worsen an analysis when some covariate combinations are sparse?
Solutions. (1) The interaction contrast \(\operatorname{logit}p_{11}-\operatorname{logit}p_{10}-\operatorname{logit}p_{01}+\operatorname{logit}p_{00}\) must equal zero. (2) Errors in different cells can cancel when averaged; Example 2 has equal overall observed and predicted fractions despite errors of 0.15 in each cell. (3) Conditioning on \(H\) gives \(h(H)\{p/q-(1-p)/(1-q)\}=h(H)\{p-q\}/\{q(1-q)\}\). (4) Sparse cells provide little information for estimating the added coefficient and may yield extreme fitted probabilities, hence unstable inverse-probability weights.