Comparing Three Tests for a Logistic Slope

One Logistic Slope, Three Routes to a Test

A logistic regression slope can be tested in three standard ways: the Wald test uses the estimated slope and its standard error; the score test evaluates the slope score at the fit constrained by the null; and the likelihood-ratio test compares the maximized likelihoods under the unrestricted and null models. In large regular samples, these tests have the same limiting reference distribution. In a particular finite sample, however, they use different information and need not give identical statistics or decisions.

We will compare them in a particularly transparent logistic model. Two independent groups have covariate values \(x_0=0\) and \(x_1=1\), with group sizes \(n_0,n_1>0\). Let \(Y_j\) be the number of events in group \(j\). The model is

The target is the two-sided test \(H_0:\beta=0\) against \(H_1:\beta\ne 0\). Under the null, the two groups have a common event probability. Under the unrestricted model, the two group probabilities can differ. We assume independent binomial sampling, positive group sizes, and interior observed proportions when using the ordinary finite maximum-likelihood estimates below. The usual chi-square reference distributions are large-sample approximations under regularity conditions, including an identifiable model and an interior null fit with nonsingular information.

What Each Test Fits

In this two-group model, the unrestricted estimates are \(\hat p_j=Y_j/n_j\), and the slope is the difference of the groups’ estimated logits:

The restricted fit instead pools the event counts: \(\tilde p=(Y_0+Y_1)/(n_0+n_1)\). The score test needs this null fit, not the unrestricted estimate of \(\beta\). The likelihood-ratio test needs both fits. Keeping these fitting requirements distinct is essential when calculating or interpreting the three tests.

Worked Comparison With Explicit Counts

Why the Score Information Has This Form

The score calculation includes the intercept as a nuisance parameter: it must account for estimating the intercept under the null rather than treating it as known. The following derivation shows exactly how that adjustment enters.

Proof. Apart from terms constant in \(\alpha,\beta\), the log-likelihood is

Write \(p_j=\operatorname{logit}^{-1}(\alpha+\beta x_j)\). Differentiating with respect to the slope gives \(U_\beta=\sum_j x_j(Y_j-n_jp_j)\). Under \(H_0\), \(p_0=p_1=\tilde p\), and \(x_0=0,x_1=1\), so \(U_\beta=Y_1-n_1\tilde p\). The intercept score is \(U_\alpha=Y_0+Y_1-(n_0+n_1)\tilde p=0\), by the definition of the pooled estimate.

For a binomial logistic observation, the information contribution for a linear predictor is \(n_jp_j(1-p_j)\). Thus, at the null fit, the information matrix in the order \((\alpha,\beta)\) is

Adjusting for the nuisance intercept gives the Schur complement

The score statistic is the squared nuisance-adjusted score divided by this information. Since the restricted intercept score is zero, the adjusted slope score equals \(U_\beta\). The stated result follows. The count condition ensures \(0<\tilde p<1\), so both weights and the adjusted information are positive. \(\square\)

Deriving the Likelihood-Ratio Comparison

For these two groups, the unrestricted model has one free probability for each group, while the null model constrains them to be equal. Their maximized log-likelihoods make the likelihood-ratio statistic especially direct to derive.

Derivation. The binomial log-likelihood for group \(j\), including its combinatorial constant, is \( \log {n_j\choose Y_j}+Y_j\log p_j+(n_j-Y_j)\log(1-p_j)\). At the unrestricted maximum, \(p_j=\hat p_j\); at the null maximum, both probabilities equal \(\tilde p\). Subtracting the null log-likelihood from the unrestricted one cancels the combinatorial constants. Multiplying the remaining difference by two gives the displayed expression. The statistic is nonnegative because the unrestricted model includes the null model and therefore cannot have a smaller maximized likelihood. Under the null and the stated regularity conditions, its asymptotic reference distribution is \(\chi^2_1\), since the null imposes one independent restriction. \(\square\)

Interpreting Differences and Avoiding Pitfalls

In the first example, the Wald statistic is smaller than the score and likelihood-ratio statistics, and its p-value lies above \(0.025\) while the other two lie below. This is not a contradiction: each statistic is built from a different aspect of the likelihood. The Wald test measures the unrestricted estimate relative to its estimated standard error; the score test measures the slope’s evidence at the null fit; and the likelihood-ratio test measures the loss in maximized log-likelihood when the null restriction is imposed. Their local equivalence explains why they often agree in large samples, not why they must agree exactly in finite samples.

A common mistake is to calculate the score statistic using the unrestricted slope estimate and the unrestricted information, or to call the score p-value “approximately at the cutoff” after rounding. The correction is to evaluate the score and its nuisance-adjusted information at the restricted fit, then compare the unrounded p-value with the chosen level. In the first example, \(S=500/99\) gives \(p\approx0.02465\), so the score test rejects at level \(0.025\).

There is also an important boundary caveat. If \(Y_0+Y_1=0\) or \(Y_0+Y_1=n_0+n_1\), the pooled null estimate is \(0\) or \(1\). Then \(w_0=w_1=0\), the restricted finite intercept does not exist, and the ordinary score statistic has zero information in its denominator. It is not valid to substitute these values into the interior score formula and report a \(0/0\) statistic as a test result. Boundary cases require separate treatment.