Wald, Score, and Likelihood-Ratio Tests in Logistic Association

Why Compare Three Tests?

For a binary trait, logistic regression can describe how the odds of the trait vary with a genotype predictor. Once a model has been fitted, a central question is whether the genotype effect differs from zero. The Wald, score, and likelihood-ratio tests all address this question, but they use different information: the Wald test uses the fitted coefficient and its uncertainty, the score test evaluates the slope at the null model, and the likelihood-ratio test compares the maximized likelihoods of two nested models.

The tests are asymptotically equivalent under the null in a regular, correctly specified model. In a finite sample, their statistics and p-values need not match. Knowing how each is constructed makes it easier to interpret software output and to recognize when a result depends on the choice of test.

Model and Null Hypothesis

Let \(Y_i\) be a binary outcome for individual \(i\), with \(Y_i=1\) indicating the trait and \(Y_i=0\) otherwise. Let \(G_i\) be a specified numerical genotype predictor, such as an allele dosage coded 0, 1, or 2, or an indicator for a chosen genotype contrast. The single-effect logistic model is

Y_i\mid G_i \sim \operatorname{Bernoulli}(\pi_i), \qquad \log\left(\frac{\pi_i}{1-\pi_i}\right)=\beta_0+\beta_1G_i.

Here \(\pi_i=P(Y_i=1\mid G_i)\), \(\beta_0\) is the intercept, and \(\beta_1\) is the genotype coefficient. Under this model, a one-unit increase in \(G_i\) multiplies the odds by \(\exp(\beta_1)\). The tests in this tutorial assess the coefficient for the chosen coding; they do not choose the coding or establish that a particular genetic inheritance pattern is biologically correct.

Definition: The single-effect null hypothesis is \(H_0:\beta_1=0\), against the two-sided alternative \(H_a:\beta_1\ne 0\). Under \(H_0\), the model has an intercept but no genotype effect. The alternative model includes both the intercept and genotype coefficient.

For the usual large-sample tests, assume independent observations, a correctly specified logistic mean model, and regularity conditions that allow estimation and likelihood-based approximations. In particular, the estimates should be finite and the relevant information should be nonzero. The approximations can be unreliable with sparse data, very small samples, or separation, where one or more logistic estimates may diverge.

Three Test Statistics

Write \(\hat{\beta}_1\) for the fitted genotype coefficient and \(SE(\hat{\beta}_1)\) for its estimated standard error. The Wald test measures the fitted coefficient in standard-error units. Its signed statistic is compared with a standard normal distribution; equivalently, its square is compared with a chi-square distribution with one degree of freedom.

Z_W=\frac{\hat{\beta}_1}{SE(\hat{\beta}_1)}, \qquad W=Z_W^2\ \mathrel{\dot\sim}\ \chi^2_1 \quad\text{under }H_0.

The score test asks whether the likelihood is increasing away from the null fit in the direction of the genotype effect. Let \(U_1\) be the score for \(\beta_1\), evaluated with the nuisance parameter \(\beta_0\) estimated under \(H_0\), and let \(I_{1\mid 0}\) be the corresponding efficient information for \(\beta_1\) after accounting for the intercept. Then the one-parameter score statistic is

S=\frac{U_1^2}{I_{1\mid 0}} \ \mathrel{\dot\sim}\ \chi^2_1 \quad\text{under }H_0.

The likelihood-ratio test compares the maximized log likelihood under the full model, \(\ell_1\), with that under the null model, \(\ell_0\). Since the full model contains the null model, its maximized log likelihood cannot be smaller. The statistic is

LR=2(\ell_1-\ell_0) \ \mathrel{\dot\sim}\ \chi^2_1 \quad\text{under }H_0.

For each squared statistic, the approximate p-value is the upper-tail probability under \(\chi^2_1\), such as \(P(\chi^2_1\ge W)\). A small p-value indicates that the observed data would be unusual under the null model and its large-sample approximation. It is not the probability that the null hypothesis is true.

Worked Calculations with a Binary Genotype Contrast

A useful setting for seeing all three procedures is a binary genotype predictor \(G\), coded 1 for carriers of a specified genotype group and 0 for the reference group. With an intercept and this predictor, the logistic model gives each group its own trait probability. The following hypothetical grouped data let us calculate all three tests directly.

Worked Example: Wald, Score, and Likelihood-Ratio Tests from a Two-by-Two Table

Suppose a hypothetical study records 30 individuals with the trait and 20 without it in the carrier group, and 20 with the trait and 30 without it in the reference group. Treat the observations as independent and use the specified carrier indicator as the single genotype predictor.

Genotype group	Trait present	Trait absent	Total
Carrier (\(G=1\))	30	20	50
Reference (\(G=0\))	20	30	50
Total	50	50	100

The estimated odds in carriers are \(30/20=1.5\), and those in the reference group are \(20/30=0.6667\). Thus the estimated odds ratio is \(1.5/0.6667=2.25\), and the fitted genotype coefficient is

\hat{\beta}_1=\log(2.25)=0.8109.

Wald test. For this two-group model, the estimated variance of the log odds ratio is the sum of the reciprocals of the four cell counts:

SE(\hat{\beta}_1) =\sqrt{\frac{1}{30}+\frac{1}{20}+\frac{1}{20}+\frac{1}{30}} =\sqrt{\frac{1}{6}} =0.4082.

Therefore \(Z_W=0.8109/0.4082\approx1.986\). Using the unrounded coefficient and standard error, \(W\approx3.946\). Its one-degree-of-freedom chi-square p-value is approximately 0.0470.

Score test. Under \(H_0\), both genotype groups have the same estimated trait probability, \(50/100=0.5\). The expected count in each cell is therefore \(50(0.5)=25\). In this two-by-two setting, the score statistic for the genotype coefficient equals the Pearson chi-square statistic:

S=\sum_{\text{four cells}}\frac{(O-E)^2}{E} =4\left(\frac{(30-25)^2}{25}\right) =4.000.

The score-test p-value is \(P(\chi^2_1\ge4.000)\), approximately 0.0455.

Likelihood-ratio test. The null model's fitted expected counts are 25 in each cell. The full model fits the observed group-specific proportions, so its fitted counts equal the observed counts. The likelihood-ratio statistic is the binomial deviance:

\begin{aligned} LR &=2\left[ 30\log\left(\frac{30}{25}\right) +20\log\left(\frac{20}{25}\right) +20\log\left(\frac{20}{25}\right) +30\log\left(\frac{30}{25}\right) \right]\\ &=2\left[60\log(1.2)+40\log(0.8)\right]\\ &\approx 4.027. \end{aligned}

The likelihood-ratio p-value is approximately \(P(\chi^2_1\ge4.027)=0.0448\). All three tests suggest evidence against no association at the 0.05 level, but their p-values differ: the Wald p-value is larger here than the score and likelihood-ratio p-values. The estimated odds ratio describes the fitted association; the tests address evidence against \(H_0\), not causation.

Why the Tests Are Similar but Not Identical

The three procedures are connected by the behavior of the log likelihood near the null. A short quadratic approximation explains why they agree in large samples. Let \(\beta_1\) be near zero, and suppose the nuisance intercept has been handled appropriately. Near the null, the log likelihood can be approximated by its value at zero plus a linear score term and a negative quadratic information term:

\ell(\beta_1)\approx\ell(0)+U_1\beta_1-\frac{1}{2}I_{1\mid 0}\beta_1^2.

Maximizing this quadratic gives \(\hat{\beta}_1\approx U_1/I_{1\mid 0}\). Its estimated variance is approximately \(1/I_{1\mid 0}\), so the Wald statistic is approximately \(U_1^2/I_{1\mid 0}\), the score statistic. Substituting the maximizing value into the quadratic gives twice the log-likelihood increase as \(U_1^2/I_{1\mid 0}\), also the score statistic. Thus the three statistics agree to the order captured by this local approximation. In actual finite samples, the likelihood may not be well represented by a quadratic over the full distance from the null to the fitted value, and the procedures can differ.

Worked Example: Comparing Tests from a Hypothetical Dosage-Model Output

Consider a separate hypothetical analysis of a binary trait using genotype dosage \(G=0,1,2\), with a single additive genotype coefficient and an intercept. Suppose the fitted output reports \(\hat{\beta}_1=0.200\), \(SE(\hat{\beta}_1)=0.0900\), a score \(U_1=18.0\) with null efficient information \(I_{1\mid 0}=65.0\), and maximized log likelihoods \(\ell_0=-402.316\) and \(\ell_1=-399.831\). These are illustrative rounded summaries from the same hypothetical fit.

For the Wald test,

Z_W=\frac{0.200}{0.0900}=2.222, \qquad W=\left(\frac{0.200}{0.0900}\right)^2\approx4.938.

Its approximate two-sided normal p-value, equivalently the upper-tail \(\chi^2_1\) probability for \(W\), is about 0.026. For the score test,

S=\frac{18.0^2}{65.0}=\frac{324}{65.0}\approx4.985,

giving an approximate \(\chi^2_1\) p-value of 0.02557. Finally,

$$ LR=2[-399.831-(-402.316)]=2(2.485)=4.970, $$

with an approximate \(\chi^2_1\) p-value of 0.02579. All three tests provide similar evidence against \(H_0:\beta_1=0\). Their small numerical differences are compatible with the large-sample approximation; because the reported summaries are rounded, the calculations should not be expected to agree to every final digit.

Choosing and Interpreting a Test

The Wald test is convenient when a fitted coefficient and standard error are available, and it directly reports the coefficient's distance from zero in standard-error units. The score test requires fitting only the null model, which can be useful when fitting the full model is difficult. The likelihood-ratio test requires fitting both nested models but uses their maximized likelihoods to measure the improvement from adding the genotype effect.

For a one-parameter test, the reference distribution has one degree of freedom. More generally, a joint test of \(q\) genotype parameters uses a chi-square approximation with \(q\) degrees of freedom under suitable regularity conditions; that is a different null hypothesis from the single-coefficient test developed here. A p-value should be reported alongside the tested coefficient or effect estimate and its direction, rather than as a stand-alone measure of effect size.

Common Pitfall

Wrong reasoning: “The Wald p-value is smaller, so it must be the correct test and the association is stronger.”
Correction: Wald, score, and likelihood-ratio tests use different finite-sample calculations for the same null hypothesis. A numerical difference does not by itself show that one test is correct and the others are wrong. State which test was used, check that the model and approximation are suitable, and interpret the result with the estimated effect and its uncertainty.

A related mistake is to interpret a non-small p-value as proof that the genotype has no effect. Failure to reject \(H_0\) means the data do not provide sufficiently strong evidence against that null at the chosen significance level; it does not establish that \(\beta_1=0\).

Key takeaway: For a single genotype coefficient, the Wald test uses the fitted estimate and standard error, the score test uses the null-model score and information, and the likelihood-ratio test compares nested-model likelihoods. Under regular large-sample conditions, each statistic is approximately \(\chi^2_1\) under \(H_0:\beta_1=0\), but the observed statistics can differ in finite samples.

Check Your Understanding

Try these questions without looking back.

In the dosage model with \(G=0,1,2\), what does \(H_0:\beta_1=0\) say about the genotype effect on the odds?
Which quantities from a fitted model are used directly in the Wald statistic?
Why can a score test be calculated without fitting the full model?
If the null and full log likelihoods are \(-120.4\) and \(-118.9\), respectively, calculate the likelihood-ratio statistic and state its large-sample reference distribution for a single coefficient.
Why is it incorrect to conclude that the genotype has no effect merely because a test does not reject \(H_0\)?