Polynomial Terms and Transformations in Score Models

Why Use Polynomial Terms and Transformations?

A logistic propensity model must represent how treatment probability changes with measured baseline covariates. A straight-line term for a continuous covariate assumes a constant change in log odds for every one-unit increase. That can be too restrictive: the association may bend, flatten, or change direction. Adding a polynomial term or transforming the covariate allows a different shape while retaining a logistic model.

The choice of shape matters because the propensity score is the conditional probability of treatment given the covariates, not merely a device for predicting treatment labels. The fitted model should support balance in the covariates used for design. A flexible formula alone does not establish balance or make a causal effect identifiable: identification still depends on assumptions such as conditional exchangeability and positivity, and balance must be assessed in the resulting design.

Definition: Let $A$ indicate treatment, with $A=1$ for treated and $A=0$ for untreated, and let $X$ denote measured baseline covariates. The propensity score is $e(X)=P(A=1\mid X)$. A logistic propensity model specifies $\operatorname{logit}\{e(X)\}=\log\{e(X)/(1-e(X))\}=\eta(X)$, so the fitted score is $\hat e(X)=1/\{1+\exp[-\hat\eta(X)]\}$. Polynomial terms use powers such as $X^2$; transformed terms replace $X$ with a function such as $\log(X)$.

Quadratic Terms and Their Combined Effect

A quadratic specification for a continuous covariate $X$ can be written using a centered and scaled version $z=(X-c)/s$, where $c$ is a reference value and $s>0$ is a scale chosen for interpretability. For example, $c$ might be the sample median and $s$ a convenient unit such as 10 years. The logistic model may then include both $z$ and $z^2$:

\eta(X)=\beta_0+\beta_1z+\beta_2z^2,\qquad e(X)=\frac{1}{1+\exp\{-\eta(X)\}}.

The coefficient $\beta_1$ is not generally the effect of a one-unit increase in $z$ across the entire range: the squared term contributes too. At a particular value $z$, the local rate of change in log odds per one-unit increase in $z$ is $\beta_1+2\beta_2z$. A contrast between two values $z_a$ and $z_b$ is often easier to interpret directly:

\log\left\{\frac{\operatorname{odds}(z_b)}{\operatorname{odds}(z_a)}\right\} =\beta_1(z_b-z_a)+\beta_2(z_b^2-z_a^2), \qquad \frac{\operatorname{odds}(z_b)}{\operatorname{odds}(z_a)} =\exp\{\beta_1(z_b-z_a)+\beta_2(z_b^2-z_a^2)\}.

This contrast is conditional on any other covariates in the model being held fixed. If $\beta_2>0$, the logit curve is convex; if $\beta_2<0$, it is concave. The fitted probability also changes monotonically with the logit, but its rate of change is not constant because the logistic function is nonlinear.

Worked Example: Reading a Quadratic Age Term

Suppose a hypothetical study includes ages from 30 to 70 years and fits the age contribution $ -1.20+0.040z+0.030z^2$, with $z=(\text{age}-50)/10$. Hold all other covariates fixed. Calculate fitted propensity scores at ages 40, 50, and 60, and describe the curve.

At age 40, $z=-1$. The combined age contribution to the logit is $\eta=-1.20+0.040(-1)+0.030(-1)^2=-1.210$. Thus $\hat e=1/\{1+\exp(1.210)\}\approx0.2297$.

At age 50, $z=0$, so $\eta=-1.20$ and $\hat e=1/\{1+\exp(1.20)\}\approx0.2315$. At age 60, $z=1$, so $\eta=-1.20+0.040+0.030=-1.130$, giving $\hat e=1/\{1+\exp(1.130)\}\approx0.2442$. These are fitted treatment probabilities at the specified covariate values, not causal effects of age.

The slope of the age contribution to the logit per one-unit increase in $z$ is $0.040+0.060z$. It is zero at $z=-2/3$, corresponding to age $50+10(-2/3)\approx43.33$ years. Since the quadratic coefficient is positive, the logit has a minimum there; the fitted probability also has a minimum there. This illustrates why $\beta_1$ alone cannot describe the age association over the whole range.

Transformations and Interpretable Contrasts

A transformation changes the scale on which a predictor enters the model. For a positive covariate such as income, a model can use its natural logarithm. With $\log(X)$ in the logit, equal multiplicative changes in $X$ correspond to equal changes in the logit. The units and domain must be respected: a logarithm requires a positive argument, and its input should be made dimensionless when possible by taking a ratio to a meaningful reference value.

Worked Example: A Log-Transformed Income Predictor

In a hypothetical model, income is measured in thousands of dollars and is positive. The fitted logit contribution is $-2+0.70\log(\text{income}/40)$, where $\log$ is the natural logarithm. Compare incomes of $40,000 and $80,000, holding other covariates fixed.

At $40,000, the income ratio is $40/40=1$, so $\log(1)=0$, $\eta=-2$, and the fitted score is $1/\{1+\exp(2)\}\approx0.1192$. At $80,000, the ratio is $80/40=2$, so $\eta=-2+0.70\log(2)=-2+0.70(0.6931)\approx-1.5148$. The fitted score is $1/\{1+\exp(1.5148)\}\approx0.1802$.

The log-odds contrast for doubling income is $0.70\log(2)\approx0.4852$, so the corresponding treatment-odds ratio is $\exp\{0.70\log(2)\}\approx1.625$. In this model, doubling income multiplies the conditional odds of treatment by about 1.625, holding other covariates fixed. It does not multiply the probability by that amount, and it does not mean that income causes treatment.

Other transformations can be chosen when justified by the covariate’s meaning and the observed pattern. For example, a square-root transformation is defined for nonnegative values and often changes rapidly at small values but more slowly at large values. A reciprocal transformation requires a nonzero predictor and can become extreme near zero. Transformations are not interchangeable: each imposes a distinct shape on the logit and can have a restricted domain.

Why Centering, Scaling, and Shape Checks Matter

Centering makes the intercept interpretable at a meaningful covariate value. Scaling makes polynomial coefficients operate on manageable numbers: if raw age is squared, values such as $50^2$ can make coefficients hard to read and can create numerical difficulties. Re-expressing a quadratic with centered or scaled units changes its coefficients, but it does not change the fitted curve when the model contains the same polynomial terms and is fit to the same data. Centering can also reduce the correlation between a linear term and its square, though it does not guarantee that all coefficient estimates are precise.

A quadratic has a single turning point at $z=-\beta_1/(2\beta_2)$ when $\beta_2\ne0$. Checking whether that point lies within the observed data range is a useful diagnostic. A turning point far outside the data may make the fitted curve appear linear over the observed region, while a turning point inside the range should be compared with the data and the substantive setting. The observed range and the number of observations across it matter: a curve supported by very few observations at an endpoint is uncertain there.

Worked Example: Recognizing Unsupported Extrapolation

A hypothetical data set contains ages from 30 through 70 years. A fitted quadratic uses $z=(\text{age}-50)/10$ and has $\eta=-1+0.020z+0.080z^2$. Calculate the fitted score at age 70, then inspect what the same formula predicts at age 90 and explain the limitation.

At age 70, $z=2$, so $\eta=-1+0.020(2)+0.080(2^2)=-1+0.040+0.320=-0.640$. Therefore $\hat e=1/\{1+\exp(0.640)\}\approx0.3452$. This is at the upper edge of the observed age range, so its credibility depends on the data available near that edge and on the adequacy of the model there.

At age 90, $z=4$, so the formula gives $\eta=-1+0.020(4)+0.080(4^2)=-1+0.080+1.280=0.360$, and $\hat e=1/\{1+\exp(-0.360)\}\approx0.5890$. But age 90 lies outside the observed range. The sharp rise is a mathematical consequence of the positive squared term, not evidence that the treatment probability truly rises in this way at older ages. Without data or a defensible substantive model beyond age 70, this prediction is unsupported extrapolation.

Fitting, Diagnostics, and Alternatives

A polynomial or transformed term is included as part of the logistic propensity model. The model is fit using the observed treatment indicator and baseline covariates; after fitting, predicted probabilities are computed for the observations. The construction assumes that treatment status and the relevant covariates are measured appropriately, that the selected functional form is an adequate approximation over the region used, and that there is enough information to estimate the parameters. The causal interpretation of downstream estimates additionally requires the relevant causal assumptions; the functional form cannot supply them.

Diagnostics should examine both model behavior and the purpose of the score model. Plot fitted logits or probabilities against each continuous covariate, overlaid with observed treatment patterns, and check whether curvature is supported across the range rather than driven by a few extreme observations. Inspect fitted scores for values close to zero or one and identify covariate regions with little or no overlap between treated and untreated units. After matching or another design step, assess covariate balance directly. A high treatment-prediction rate is not evidence that covariates are balanced.

A linear term is simpler and may be adequate when the logit relationship is approximately straight across the relevant range. A quadratic is compact and can represent one bend, but it imposes a global parabolic shape that may behave poorly at the boundaries. Transformations such as $\log(X)$ can encode a scientifically meaningful scale and often yield interpretable multiplicative contrasts, but their domains and shape assumptions need checking. Restricted cubic splines, covered in the earlier material on restricted cubic splines, allow more local flexibility while controlling the behavior at the boundaries; they also require choices about knots. The appropriate comparison is not a contest for the best treatment classifier. It is a choice of plausible specification followed by balance assessment and attention to overlap.

Common pitfall: Treating $\beta_1$ in a quadratic model as the effect of a one-unit increase everywhere is incorrect because the squared term changes the contrast with $z$. Interpret the combined expression $\beta_1(z_b-z_a)+\beta_2(z_b^2-z_a^2)$ for a specified comparison. Also, do not interpret predictions outside the observed covariate support as if the fitted curve had been validated there.

Practical Specification Strategy

Set the covariate scale.
Center at a meaningful value and scale in useful units; verify that a transformation is defined for the observed values.

Specify a plausible shape.
Choose a linear, polynomial, or transformed term based on substantive understanding and the observed range, rather than treatment classification alone.

Interpret contrasts.
For polynomial terms, substitute both covariate values into the full logit contribution before calculating an odds ratio or fitted score.

Check support and balance.
Look for sparse regions, implausible curvature, extreme scores, and extrapolation; then evaluate balance in the design that uses the estimated scores.

These models remain approximations. A quadratic may miss multiple bends or abrupt changes, and a transformation may impose a shape that is convenient algebraically but wrong substantively. Estimates in poorly supported regions can be unstable even when the fitted curve looks smooth. No single coefficient, goodness-of-fit summary, or predicted treatment probability establishes that the propensity model is adequate for causal design.

Check Your Understanding

Answer each question, then compare with the brief solutions.

A logit contains $0.10z-0.04z^2$. What is the change in log odds from $z=0$ to $z=2$, holding other covariates fixed?
Why is the linear coefficient in a quadratic model not a constant effect across the covariate range?
A model includes $\log(X)$. What domain condition must hold, and what kind of covariate contrast is especially interpretable?
A quadratic model predicts scores for values of $X$ far beyond every observed value. What is the central concern?
What should be checked after fitting a propensity model, beyond how accurately it predicts treatment?

Solutions. (1) The log-odds change is $0.10(2)-0.04(2^2)=0.20-0.16=0.040$, corresponding to an odds ratio $\exp(0.040)\approx1.041$. (2) The local slope is $\beta_1+2\beta_2z$, so it depends on $z$. (3) $X$ must be positive; multiplicative changes such as doubling are natural contrasts because they produce a fixed change in $\log(X)$. (4) The prediction is extrapolation: the curve’s behavior there is not supported by observed covariate data and may be implausible. (5) Check the fitted shape, overlap and extreme scores, and covariate balance after applying the scores in the intended design.