Deriving Logistic Propensity Estimation From the Bernoulli Likelihood

From Treatment Probabilities to a Likelihood

A propensity score is the conditional probability of receiving treatment given measured covariates. In observed data, treatment is binary, so a natural starting point is a probability model for each person’s treatment indicator. Logistic estimation is not a separate definition of the propensity score: it is a way to estimate that conditional probability by specifying a model and choosing the parameters that make the observed treatment assignments most plausible under it.

Let \(A_i\) equal 1 if individual \(i\) receives treatment and 0 otherwise, and let \(X_i\) denote the covariate value used in a one-covariate model. Write \(e(X_i)=P(A_i=1\mid X_i)\). The logistic model is

Here \(\beta_0\) is the intercept and \(\beta_1\) is the slope. The model places the linear expression \(\beta_0+\beta_1X_i\) on the logit scale, where \(\operatorname{logit}(e)=\log\{e/(1-e)\}\). The fitted propensity score for an individual with covariate value \(X_i\) is the model’s estimated treatment probability at that value.

Assumptions and the Bernoulli Likelihood

The likelihood construction uses a sampling model for treatment assignment. Conditional on the observed covariates, assume the \(A_i\) are independent Bernoulli random variables, with \(P(A_i=1\mid X_i)=e(X_i)\). Also assume that this conditional probability follows the specified logistic form. The covariates are treated as given when forming the conditional likelihood; the model does not need to specify how the covariates themselves were generated.

For one individual, the Bernoulli probability of observing \(A_i\) is \(e(X_i)\) when \(A_i=1\), and \(1-e(X_i)\) when \(A_i=0\). Both cases can be written together as \(e(X_i)^{A_i}\{1-e(X_i)\}^{1-A_i}\). Conditional independence makes the likelihood the product of those contributions:

A likelihood is a function of the unknown parameters with the observed data held fixed. It is not a probability distribution over parameter values. Taking its logarithm is convenient because the logarithm is increasing, so maximizing the likelihood and maximizing its log produce the same parameter estimates. Substituting the logistic probability simplifies the individual contribution: \(\log e(X_i)=\eta_i-\log(1+\exp(\eta_i))\), and \(\log\{1-e(X_i)\}=-\log(1+\exp(\eta_i))\), where \(\eta_i=\beta_0+\beta_1X_i\). Therefore:

This is the Bernoulli log-likelihood for the logistic model. Each observed treatment value contributes a term, and the covariate enters through the linear predictor. The corresponding score equations are obtained by differentiating with respect to each parameter and setting the derivatives to zero, when a finite interior maximum exists:

Thus the intercept score sets the sum of observed-minus-fitted treatment indicators to zero. The slope score sets the covariate-weighted sum of those residuals to zero. In a model with several covariates, each parameter has a corresponding score equation: the residuals are weighted by that parameter’s design variable.

Why the Score Equations Identify a Maximum

A zero score is a necessary condition for a differentiable log-likelihood’s interior maximum, but by itself it need not establish that the point is a maximum. For this logistic likelihood, the second derivative matrix is

For any vector \(v\), the quadratic form of this matrix is \(-\sum_i e(X_i)\{1-e(X_i)\}(v_0+v_1X_i)^2\), which is never positive. Hence the log-likelihood is concave. If the observed design has full column rank and all fitted probabilities are strictly between 0 and 1, the matrix is negative definite at finite parameter values; the log-likelihood is then strictly concave, so an interior solution to the score equations is the unique maximum. This justifies solving the score equations when those conditions hold.

A finite maximum is not guaranteed for every data set. If covariate values perfectly distinguish treated from untreated individuals, likelihood values can keep improving as a coefficient grows in magnitude, without reaching a finite maximizing coefficient. A rank-deficient design can also prevent unique estimation. These are not merely computational inconveniences: they indicate that the stated likelihood does not yield an ordinary, finite, unique estimate under the observed pattern.

Exact Fit When the Covariate Has Two Levels

With a binary covariate, the model has one fitted treatment probability for each of its two levels. When each level contains both treated and untreated individuals, the likelihood can be maximized directly from the two observed treatment proportions. This special case provides a transparent example of how the score equations determine the estimates. It is not a general shortcut for continuous covariates or more complicated models.

Reading the Score and Checking a Candidate

The score equations are also useful for checking whether a proposed pair of coefficients is at a stationary point. A nonzero score means that the log-likelihood has a nonzero local slope in at least one parameter direction, so the pair cannot be an interior maximum. The following calculation does not solve for the maximum; it shows how to evaluate the equations and why simply choosing plausible-looking coefficients is not enough.

Diagnostics, Alternatives, and Limitations

Several checks follow directly from the likelihood formulation. Confirm that treatment is coded consistently as 0 and 1, that the intended covariates are available for every modeled individual, and that the logistic specification corresponds to the conditional probability being modeled. Check whether treatment and comparison individuals occur across the covariate values in the data. If treatment is perfectly predicted by the covariate pattern, a finite maximum may fail to exist. If covariate columns are redundant, coefficients may not be uniquely identified. Very small or very large fitted probabilities can also signal sparse information or extrapolation, even when numerical software returns estimates.

The logistic model is one parametric way to represent \(P(A=1\mid X)\). A saturated model that assigns an unconstrained probability to each observed covariate pattern can reproduce group proportions, but it can become impractical when patterns are numerous or continuous covariates are present. Other links, such as the probit link, impose a different probability curve. These alternatives change the model’s assumptions and fitted probabilities; neither the Bernoulli likelihood nor the score derivation makes one link universally correct.

Most importantly, maximizing the treatment likelihood is a statistical estimation step, not a guarantee of causal validity. A well-fitted logistic likelihood does not establish that all confounders were measured, that treatment groups are comparable on the covariates relevant to the causal question, or that estimated probabilities provide adequate overlap. The propensity score’s role in causal adjustment depends on substantive design and assumptions beyond the treatment assignment likelihood.

• Mistake: “The likelihood is the probability that my parameter values are true.” The likelihood is instead a function of parameter values, proportional to the probability of the observed data under the model.
• Mistake: “A zero score always proves I have a valid, unique estimate.” A zero score is a stationary-point condition; existence and uniqueness require conditions such as a full-rank design and a finite interior solution.
• Mistake: “A high treatment-prediction fit proves confounding is controlled.” Prediction performance alone does not establish causal identification or covariate balance.