From Covariates to Treatment Probabilities
A propensity score model describes how treatment assignment varies with observed pretreatment covariates. Its output is a predicted probability of receiving treatment, not a predicted outcome and not, by itself, an estimate of a treatment effect. Logistic regression is a useful specification when the treatment indicator is binary, but choosing the terms in its linear predictor matters: additive terms and nonlinear terms can imply different treatment probabilities for the same covariate patterns.
Let \(A_i=1\) indicate that unit \(i\) received treatment and \(A_i=0\) indicate control. Let \(X_i\) denote that unit's measured pretreatment covariates. The propensity score is \(e(X_i)=P(A_i=1\mid X_i)\). In a logistic specification, the conditional treatment probability is linked to a linear predictor by the logistic function.
An additive specification includes separate covariate terms but no products or nonlinear transformations. A nonlinear specification changes that structure, for example by adding an interaction between covariates or a squared continuous covariate. “Nonlinear” here refers to the covariates' contribution to the log odds; the logistic model remains linear in its estimated coefficients.
Specify the Terms Before Fitting
The coding determines what each coefficient represents. A binary covariate can be coded 0 or 1, with 0 as the reference category. A continuous covariate can be centered or scaled; doing so changes the coefficient's reference point or units but does not, by itself, make the fitted relationship nonlinear. If a continuous covariate is entered quadratically, include both its linear and squared terms unless there is a substantive reason to constrain the linear term.
For example, with a binary covariate \(S_i\) and a centered continuous covariate \(C_i\), an additive model and one with an interaction are:
In the interaction model, the association of \(C_i\) with treatment log odds can differ by \(S_i\). In a quadratic model, the predictor might instead include \(\beta_1C_i+\beta_2C_i^2\), allowing curvature in log odds. Terms should be chosen using subject-matter knowledge and diagnostic evidence; adding terms simply because they improve apparent fit can make estimates unstable.
Worked Example: Coding Covariates and Calculating a Score
Suppose a hypothetical fitted model uses smoking status \(S_i\), coded 1 for current smoker and 0 otherwise, and centered age \(C_i\), measured in decades from age 50. Its fitted predictor is \(\hat\eta_i=-1.2+0.7S_i+0.4C_i-0.3S_iC_i\). Calculate the score for a 60-year-old current smoker.
For this person, \(S_i=1\) and \(C_i=(60-50)/10=1\). The product term is \(S_iC_i=1\), so:
Transforming the fitted log odds to a probability gives:
Under this fitted model, the estimated treatment probability for a unit with these covariates is about 0.401, or 40.1%. This is a conditional model prediction, not a statement that this particular person has a 40.1% realized “fraction” of treatment, nor evidence that treatment causes an outcome.
Likelihood Fitting and a Model Comparison
Conditional on the covariates, logistic maximum likelihood treats each \(A_i\) as a Bernoulli observation with probability \(e(X_i)\). For independent units, the likelihood and log-likelihood are:
The maximum-likelihood estimate chooses coefficients that maximize this log-likelihood. For a grouped dataset with \(y_j\) treated units among \(n_j\) units sharing covariate pattern \(j\), the log-likelihood, omitting binomial coefficients that do not depend on the model parameters, is \(\sum_j[y_j\log(p_j)+(n_j-y_j)\log(1-p_j)]\). This makes small examples hand-checkable.
Worked Example: Additive Versus Interaction Model
Consider 40 hypothetical independent units, with two binary pretreatment covariates \(X\) and \(Z\). Each covariate is coded 0 or 1. There are 10 units in each of the four covariate patterns; the table records the number treated.
| \(X\) | \(Z\) | Units \(n_j\) | Treated \(y_j\) | Observed treatment proportion |
|---|---|---|---|---|
| 0 | 0 | 10 | 2 | 0.2 |
| 1 | 0 | 10 | 8 | 0.8 |
| 0 | 1 | 10 | 8 | 0.8 |
| 1 | 1 | 10 | 2 | 0.2 |
First fit the additive model \(\eta_i=\beta_0+\beta_X X_i+\beta_Z Z_i\). At \(\beta_0=\beta_X=\beta_Z=0\), every fitted probability is 0.5. The score equations for logistic regression require the residuals \(A_i-\hat e(X_i)\) to sum to zero overall and within each covariate column. There are 20 treated units overall, matching the 40 fitted probabilities of 0.5; there are 10 treated units among the 20 units with \(X=1\), and likewise 10 among the 20 with \(Z=1\). Thus all three score equations are zero. The design has full rank, so the log-likelihood is strictly concave in these coefficients and this stationary point is the unique maximum.
The additive model's maximized log-likelihood, omitting the same binomial constants, is:
Now add an interaction, \(\eta_i=\beta_0+\beta_X X_i+\beta_Z Z_i+\beta_{XZ}X_iZ_i\). This model assigns a separate logit to each of the four patterns. For one pattern with \(y_j\) treated among \(n_j\), differentiating \(y_j\log(p_j)+(n_j-y_j)\log(1-p_j)\) with respect to \(p_j\) gives \(y_j/p_j-(n_j-y_j)/(1-p_j)\). Setting this equal to zero yields \(p_j=y_j/n_j\), so the fitted probabilities are 0.2, 0.8, 0.8, and 0.2.
The corresponding logits are \(-\log(4),\log(4),\log(4),-\log(4)\). Solving for the coefficients gives \(\hat\beta_0=-\log(4)\approx-1.386\), \(\hat\beta_X=2\log(4)\approx2.773\), \(\hat\beta_Z=2\log(4)\approx2.773\), and \(\hat\beta_{XZ}=-4\log(4)\approx-5.545\). Its maximized log-likelihood is:
The interaction model improves the log-likelihood by about \(17.7179\). Under the null hypothesis that the interaction coefficient is zero, the likelihood-ratio statistic is \(2(\ell_{\mathrm{int}}-\ell_{\mathrm{add}})=35.44\). With one added parameter, its reference distribution under the usual large-sample regularity conditions is chi-squared with 1 degree of freedom; the corresponding p-value is about \(2.6\times10^{-9}\). In this hypothetical dataset, the treatment assignments vary sharply across joint covariate patterns in a way the additive model misses. The test assesses that model restriction, not whether either model is adequate for causal analysis.
Curvature, Interpretation, and Checks
Worked Example: A Quadratic Age Term
Suppose a logistic fit on a separate hypothetical dataset uses age centered at 50 years, \(C_i=(\text{age}_i-50)/10\), and returns the fitted predictor \(\hat\eta_i=-1.0+0.5C_i-0.2C_i^2\). Calculate fitted probabilities at ages 40, 50, and 60.
At age 40, \(C_i=-1\), so \(\hat\eta_i=-1.0+0.5(-1)-0.2(1)=-1.7\), giving \(\hat e=1/(1+\exp(1.7))\approx0.1545\). At age 50, \(C_i=0\), so \(\hat\eta_i=-1.0\) and \(\hat e=1/(1+\exp(1))\approx0.2689\). At age 60, \(C_i=1\), so \(\hat\eta_i=-1.0+0.5-0.2=-0.7\), and \(\hat e=1/(1+\exp(0.7))\approx0.3318\). The fitted treatment probability rises across these three ages, but the negative squared-term coefficient means the log-odds curve bends downward; it does not imply a constant increase per decade.
A likelihood comparison is informative only when the models are fitted to the same observations and outcome, and one model is nested in the other for the usual likelihood-ratio reference test. The regular chi-squared approximation also relies on conditions such as an identifiable model and a sufficiently informative sample; small or sparse cells can make the approximation unreliable. A large likelihood improvement does not guarantee stable estimated probabilities or good covariate balance.
Check fitted probabilities and residual patterns across relevant covariate ranges and combinations. Look for implausibly extreme scores, sparse patterns, and sensitivity to reasonable changes in coding or specification. Then assess covariate balance using the intended design procedure: treatment prediction accuracy or a likelihood test is not a substitute for checking whether treated and control units are comparable on observed covariates after that procedure. A model may distinguish treatment assignment well yet yield poor balance, or improve balance without maximizing classification accuracy.
The treatment-assignment likelihood assumes independent observations conditional on the specified covariates and a correctly specified conditional probability model for the fitted likelihood to represent the data-generating assignment probabilities. If scores are to support causal comparisons, further conditions are needed: consistency, conditional exchangeability given measured covariates, and positivity over the covariate patterns relevant to the target population. Estimating a logistic model cannot verify those causal assumptions, and no specification repairs unmeasured confounding or a lack of treatment overlap.
The additive model is parsimonious and can be more stable when data are limited, but it imposes no interaction or curvature on the log odds. Adding interactions or polynomial terms can represent important structure but uses more parameters and can produce unstable estimates, especially with rare treatment or sparse covariate patterns. For an interaction model, interpret the fitted probability for a joint covariate pattern rather than reading either main-effect coefficient as a universal effect. Flexible estimation approaches offer other ways to represent complex relationships, but no method removes the need for diagnostics and balance assessment.
Common Pitfall
Check Your Understanding
Answer each question, then compare your reasoning with the brief solutions below.
- In a logistic treatment model, what does \(\hat e(X_i)=0.30\) mean, and what does it not mean?
- For \(C=2\), how does a quadratic predictor \(\eta=\beta_0+\beta_1C+\beta_2C^2\) differ from the additive predictor \(\eta=\beta_0+\beta_1C\)?
- In the four-cell example, why is the additive-model estimate \((\hat\beta_0,\hat\beta_X,\hat\beta_Z)=(0,0,0)\) a maximum rather than merely a possible coefficient choice?
- What is the likelihood-ratio statistic for comparing the example's interaction and additive models, and what restriction is being tested?
- Name one reason that a better treatment-assignment likelihood does not establish the validity of a causal comparison.
Solutions. (1) It is the model-estimated conditional probability of treatment for units with those measured covariates; it is not an individual outcome probability or proof of a causal effect. (2) The quadratic predictor includes \(\beta_2(2^2)=4\beta_2\), allowing curvature in log odds; the additive predictor has no such term. (3) All intercept and covariate score equations are zero there, and the full-rank logistic model has a strictly concave log-likelihood, making that point the unique maximum. (4) It is \(2[-10.0081-(-27.7259)]=35.44\), testing whether the interaction coefficient is zero. (5) For example, model fit does not establish covariate balance, positivity, or absence of unmeasured confounding.