Nonlinear Terms and Interactions in High-Dimensional Scores

Why Add Nonlinear Terms and Interactions?

A propensity score model can include many measured covariates and still represent treatment assignment poorly if its functional form is too restrictive. For example, a model that includes age only as a linear term assumes that age has a constant change in log odds of treatment per year. The actual relationship could be curved, or age could matter differently across patient groups. A model that omits such patterns may estimate scores that appear useful for prediction while leaving important features imbalanced between treatment groups.

The remedy is not to add every imaginable term. In high-dimensional settings, indiscriminate feature expansion can create sparse cells, strong collinearity, separation, and highly variable scores. Instead, construct a limited set of plausible transformations and interactions, then check whether they improve balance and whether the resulting scores remain usable. This is a confounding-control task: treatment prediction alone does not establish that covariates are balanced.

A common starting model is logistic regression applied to a designed feature set:

Here \(f_j\) may include a linear term and selected nonlinear terms, while \(h_{jk}\) is a selected interaction feature and \(\mathcal I\) is the set of interactions included. In a high-dimensional model, the coefficients may be estimated with a penalty, but the feature definitions still determine which relationships the model can represent. For an unpenalized logistic model, the coefficients are estimated by maximum likelihood, provided the likelihood has a finite estimate.

Constructing Features Deliberately

For a continuous covariate, centering and scaling before forming powers can make the expanded model easier to fit and interpret. Define \(Z_j=(X_j-c_j)/s_j\), where \(c_j\) is a reference value such as the sample mean and \(s_j\) is a positive scale such as the sample standard deviation. A quadratic specification can then use \(Z_j\) and \(Z_j^2\). The coefficient of \(Z_j^2\) describes curvature in log odds while the linear coefficient describes the slope at \(Z_j=0\); it is not generally a constant effect across the covariate range.

A square is one simple option, not a universal shape. Splines can represent smoother local curvature, but every additional basis function consumes information and may be poorly supported in sparse regions. Choose knots or transformations using subject-matter knowledge and the observed covariate range, and avoid extrapolating a complex curve where few people are observed. For categorical covariates, use indicator variables for levels relative to a stated reference level. Interactions can then allow, for example, a treatment-assignment association with severity to differ by sex or age group.

In penalized models, scale features so the penalty does not treat a large numerical range as evidence that a term is more important. Also decide how to handle hierarchy: if an interaction is included, retaining its constituent main effects is often a sensible default. This convention avoids a model that includes a joint pattern while omitting the terms needed to represent the component relationships. It is a modeling choice, not a guarantee of causal validity.

Interactions for Categorical and Continuous Features

For a binary indicator \(G\) and a continuous feature \(Z\), the product \(GZ\) permits the slope for \(Z\) to differ between \(G=0\) and \(G=1\). For two binary indicators, their product identifies the joint category. In a logistic model, an interaction coefficient is a contrast on the log-odds scale, not a difference in treatment probabilities. Consequently, it is important to inspect both the fitted probabilities and balance in the underlying features rather than interpreting a coefficient as a direct effect.

Why Correct Functional Form Matters

The balancing property explains why misspecification is consequential. If the true propensity score is \(e(X)=P(A=1\mid X)\), then conditioning on \(e(X)\) balances the distribution of the measured covariates in expectation. In particular, by iterated expectation:

This identity says that among people with the same true score, the treatment probability is the same regardless of their covariate values. An estimated score from a restrictive model need not equal the true score. If the actual assignment mechanism is curved in age or depends on an interaction, but the fitted model omits that feature, the balancing identity does not automatically hold for the fitted score. Adding a relevant term can improve balance, but it cannot guarantee balance in every covariate or correct unmeasured confounding.

For causal interpretation, the usual conditions remain necessary: treatment and outcome are consistently defined; conditional exchangeability holds given the included pretreatment covariates; and positivity holds, so treatment probabilities are strictly between zero and one for covariate patterns relevant to the target population. A more elaborate score model does not make these assumptions true. It only changes how measured covariates are represented in the estimated treatment probability.

Assessing Improvement Without Destabilizing Scores

Evaluate candidate expansions by their effect on covariate balance and score behavior, not by the treatment-model fit statistic alone. For continuous features, inspect standardized mean differences and distributional summaries; for binary and categorical features, inspect level-specific proportions and sparse cells. Check transformed features as well as the original covariates: balance in mean age does not imply balance in squared age, and balance in two marginal indicators does not imply balance in their joint category.

Compare the model before and after adding a small, prespecified set of terms. Examine score distributions separately for treated and control observations, looking for estimates near zero or one, little common support, and abrupt changes driven by a few rare combinations. In penalized models, also check whether small changes in tuning or sample composition produce large changes in selected terms or fitted scores. If balance improves only through extreme fitted scores or by discarding many observations in matching, the added complexity may be counterproductive for the intended analysis.

A simple linear logistic specification is easier to interpret and usually less demanding of the data, but can miss curvature and effect modification. A targeted expansion makes those patterns representable while preserving a manageable model. More flexible learners can represent broader patterns, but introduce different tuning and diagnostic issues; feature expansion is not automatically preferable to them. In every approach, the causal analysis must be judged by the target population, overlap, and covariate balance—not by a high treatment-prediction score.

Common Pitfalls and Limitations

• Adding powers without checking their meaning. A quadratic term can represent curvature but may behave erratically at the extremes. Center and scale continuous variables, and examine whether the observed data support the shape being fitted.
• Adding every interaction. With many features, the number of pairwise products grows rapidly. This can generate sparse cells and separation. Select interactions selectively and use regularization where appropriate.
• Concluding that prediction proves control of confounding. A model can classify treatment well while leaving a prognostically important feature imbalanced. Assess balance after the intended matching or weighting operation.
• Checking only main effects. Similar marginal means or proportions do not establish balance in nonlinear transformations or joint categories. Check the features whose inclusion motivated the expansion.
• Treating a better balance statistic as proof of validity. Balance on measured features is diagnostic evidence, not proof of exchangeability, positivity, correct outcome measurement, or absence of unmeasured confounding.

The practical goal is a score model that represents important treatment-assignment patterns well enough to support balance, while retaining adequate overlap and stable estimation. Feature expansion is a targeted modeling decision, not a substitute for causal reasoning or diagnostic checks.