Score Equations for Logistic Treatment Models

From a Treatment Model to Fitted Weights

For a binary treatment \(A\), an inverse probability weight uses the probability of the treatment actually received, conditional on measured covariates. In practice, that probability is often estimated with logistic regression. The model’s score equations explain what the fitted probabilities satisfy in the observed sample—and, just as importantly, what they do not guarantee about the resulting weights.

Consider one treatment decision observed for \(n\) independent individuals. Let \(A_i\in\{0,1\}\) be individual \(i\)'s treatment and let \(\mathbf z_i\) be a vector of specified covariate functions, usually including an intercept. The logistic model is \(P(A_i=1\mid \mathbf z_i)=p_i(\boldsymbol\beta)=\operatorname{expit}(\mathbf z_i^\mathsf{T}\boldsymbol\beta)\), where \(\operatorname{expit}(u)=1/(1+\exp(-u))\). The model is conditional on the chosen covariates and functional form; fitting it does not by itself establish that these are adequate for causal weighting.

The corresponding unstabilized inverse probability of treatment weight for this single decision is \(\hat W_i=A_i/\hat p_i+(1-A_i)/(1-\hat p_i)\). Thus a treated individual receives \(1/\hat p_i\), while an untreated individual receives \(1/(1-\hat p_i)\). The score equations determine the fitted probabilities through the likelihood; the weights are then calculated from those probabilities.

Deriving the Score Equations

Conditional on the observed covariates, the Bernoulli likelihood and log-likelihood are

Since \(\partial p_i/\partial\boldsymbol\beta=p_i(1-p_i)\mathbf z_i\), differentiating an individual log-likelihood contribution gives

Summing these contributions yields the score \(U(\boldsymbol\beta)=\sum_i\mathbf z_i(A_i-p_i)\). The maximum-likelihood score equations set each component to zero. With an intercept, one equation is \(\sum_i(A_i-p_i)=0\); for each other model column \(z_{ij}\), the equation is \(\sum_i z_{ij}(A_i-p_i)=0\).

To verify the result, take the intercept component of the score equation. Its covariate value is \(1\) for every individual, so that component is \(\sum_i(A_i-\hat p_i)=0\), which rearranges to the stated equality. The other components follow by applying the same equation to their corresponding columns of \(\mathbf z_i\).

Existence, Concavity, and Uniqueness

The score equations describe a finite estimate only when a finite solution exists. The Hessian, the matrix of second derivatives of the log-likelihood, is

For any vector \(\mathbf v\), \(\mathbf v^\mathsf{T}H(\boldsymbol\beta)\mathbf v =-\sum_i p_i(1-p_i)(\mathbf z_i^\mathsf{T}\mathbf v)^2\leq 0\). Thus the log-likelihood is concave. If the design matrix with rows \(\mathbf z_i^\mathsf{T}\) has full column rank, then for any nonzero \(\mathbf v\) at least one \(\mathbf z_i^\mathsf{T}\mathbf v\ne0\). At finite \(\boldsymbol\beta\), every \(p_i(1-p_i)>0\), so the quadratic form is strictly negative. The log-likelihood is then strictly concave, and there can be at most one finite maximizer.

This proves uniqueness if a finite maximizer exists; it does not prove existence. For example, complete separation can make the likelihood approach its supremum as a coefficient diverges, without attaining that supremum at any finite coefficient. When a finite maximizer exists, concavity makes a score root a global maximum; with full column rank, it is the unique finite maximum. \(\square\)

What the Score Does—and Does Not—Balance

The score equation \(\sum_i\mathbf z_i(A_i-\hat p_i)=0\) is often described as a balance property. Precisely, the observed treatment indicators and fitted probabilities have matching totals when multiplied by each included design column. It does not generally say that the inverse-weighted covariate totals for treated and untreated individuals match.

The distinction is visible in the weight formula: treated records contribute \(\mathbf z_i/\hat p_i\) to a treated weighted sum, while the score involves \(\mathbf z_i(A_i-\hat p_i)\). These are different expressions. Exact balance can occur in special designs, as in the group-saturated example, but it does not follow from the logistic score equations alone. Nor does solving the score establish that the treatment model is correctly specified, that all confounders are measured, or that positivity holds.

Useful diagnostics include checking whether the fitting algorithm converged to a finite solution, inspecting the magnitude of coefficients and standard errors, and examining fitted probabilities for values close to zero or one. A small score norm, \(\lVert U(\hat{\boldsymbol\beta})\rVert\), indicates numerical satisfaction of the estimating equations; it is not evidence that the model is causally adequate. Separation, sparse covariate patterns, an ill-conditioned design matrix, or extreme fitted probabilities can undermine the practical use of the resulting weights.

Compared with an intercept-only model, a covariate-adjusted logistic model allows treatment probabilities to vary with measured covariates. A saturated model for discrete covariate patterns estimates a separate treatment proportion in each pattern, when data support it. Other treatment-probability estimators, including random forests and ensemble methods, use different fitting procedures and need not obey these logistic score equations. Whatever the fitting approach, the estimated probabilities must be assessed for their suitability as denominators; a converged logistic fit does not remove the need to consider overlap and model specification.