One-Sided Tests and Directional Alternatives

Why Direction Changes the Test

A two-sided test asks whether a coefficient differs from a specified value in either direction. A directional test asks whether it exceeds that value, or falls below it. That change affects which tail of the reference distribution is relevant. It also affects how a likelihood-ratio statistic is formed: evidence in the direction opposite to the alternative must not count as evidence for it.

Let \(\gamma\) be a scalar coefficient, or a scalar contrast of GLM coefficients, and let \(\gamma_0\) be its null boundary value. We develop the test for an increase; reversing the signs gives the corresponding test for a decrease. The Wald, score, and likelihood-ratio constructions use the same directional hypothesis but evaluate evidence in different ways.

Signed Wald and Score Statistics

Suppose \(\hat\gamma\) estimates \(\gamma\), and let \(\widehat{\operatorname{Var}}(\hat\gamma)\) be its estimated variance. The signed Wald statistic measures the estimated displacement from the null in standard-error units:

Under the null boundary, standard asymptotic normality gives \(Z_W\xrightarrow{d}N(0,1)\). The upper-tail one-sided p-value is therefore \(1-\Phi(Z_W)\), where \(\Phi\) is the standard normal distribution function. A large positive value supports an increase; a negative value points away from the specified alternative.

For a scalar coefficient \(\gamma\) and nuisance parameter vector \(\lambda\), the score test uses information after accounting for nuisance parameters. Write \(U_\gamma\) and \(U_\lambda\) for the score components and partition the Fisher information at the null fit as \(I_{\gamma\gamma}, I_{\gamma\lambda}, I_{\lambda\lambda}\). The efficient score and its information are

These quantities are evaluated at the restricted null fit. When the nuisance parameters have been maximized under the null, \(U_\lambda=0\), though the efficient information still accounts for their estimation. The signed score statistic is \(Z_S=U_{\mathrm{eff}}/\sqrt{I_{\mathrm{eff}}}\). Under regularity conditions—an interior null value for the nuisance parameters, positive efficient information, and the usual asymptotic normality of the score—\(Z_S\xrightarrow{d}N(0,1)\) at the null boundary.

Relating One-Sided and Two-Sided Reference Distributions

The signed Wald and score statistics have standard normal reference distributions under the null boundary. Their squared values have a \(\chi^2_1\) reference distribution and correspond to the familiar two-sided tests. For a positive signed statistic \(z\), the one-sided p-value is half the two-sided p-value. For a negative \(z\), it is not: the evidence points in the opposite direction.

To verify the result, first take \(Z>0\). Then \(|Z|=Z\), so \(p_{\mathrm{two}}=2[1-\Phi(Z)]\), and division by two gives the upper-tail p-value. If \(Z<0\), symmetry of the standard normal distribution gives \(\Phi(Z)=1-\Phi(|Z|)\). Consequently, \(1-\Phi(Z)=\Phi(|Z|)=1-[1-\Phi(|Z|)]=1-p_{\mathrm{two}}/2\). For \(Z=0\), both one-sided tails equal \(1/2\). \(\square\)

For example, a positive statistic is compared with the upper standard-normal critical value \(z_{1-\alpha}\) in a level-\(\alpha\) test. Equivalently, its square is compared with the \((1-2\alpha)\) quantile of \(\chi^2_1\). This is not the same cutoff as the \((1-\alpha)\) quantile used for a two-sided \(\chi^2_1\) test.

Directional Likelihood-Ratio Tests

An ordinary likelihood-ratio statistic compares the unrestricted maximum likelihood with the maximum under the equality null. For a one-sided alternative, instead compare the null-boundary fit with the best fit restricted to the alternative side. Define \(\ell_p(\gamma)\) as the profile log-likelihood, maximized over nuisance parameters for fixed \(\gamma\). The directional statistic is

It is nonnegative. If the unrestricted estimate lies above \(\gamma_0\), the constrained alternative includes it, and \(Q_+\) is the ordinary likelihood-ratio statistic. If the estimate lies below \(\gamma_0\), the constrained maximum is at the boundary in the usual unimodal regular case, and \(Q_+=0\). The sign and magnitude can also be represented by the signed root \(R=\operatorname{sign}(\hat\gamma-\gamma_0)\sqrt{Q}\), where \(Q\) is the unrestricted equality-null likelihood-ratio statistic. Then \(Q_+=\{\max(R,0)\}^2\) in this setting.

The one-sided likelihood-ratio statistic does not have an ordinary \(\chi^2_1\) null distribution. Under standard regularity conditions, its limiting distribution is a mixture: probability one-half at zero and probability one-half distributed as \(\chi^2_1\). The signed root, by contrast, has an asymptotic standard normal reference under the null boundary, so its upper-tail p-value is \(1-\Phi(R)\).

To derive this, express a local alternative as \(\gamma=\gamma_0+h/\sqrt{n}\), with \(h\ge0\). Let \(I_{\mathrm{eff}}\) denote the total efficient information for the \(n\) observations, and write \(Z_n=U_{\mathrm{eff}}/\sqrt{I_{\mathrm{eff}}}\) for the normalized efficient score at the null. The quadratic expansion of the profile log-likelihood can then be written as \(tZ_n-t^2/2+o_p(1)\), where \(t=h\sqrt{I_{\mathrm{eff}}/n}\ge0\), and \(Z_n\xrightarrow{d}N(0,1)\). The local maximum is therefore the maximum of \(tZ_n-t^2/2\) over \(t\ge0\).

If \(Z\le0\), then \(tZ-t^2/2\le0\) for every \(t\ge0\), and the maximum is zero at \(t=0\). If \(Z>0\), completing the square gives \(tZ-t^2/2=Z^2/2-(t-Z)^2/2\), whose maximum over \(t\ge0\) occurs at \(t=Z\) and equals \(Z^2/2\). Twice the maximized log-likelihood difference is thus zero for \(Z\le0\) and \(Z^2\) for \(Z>0\). A standard normal variable is nonpositive with probability one-half; conditional on being positive, its square has the \(\chi^2_1\) distribution by symmetry. This proves the claimed mixture limit. \(\square\)

For a positive signed root \(R\), the one-sided likelihood-ratio p-value can be calculated as \(P(Z\ge R)=1-\Phi(R)\). Equivalently, for \(Q_+>0\), the mixture-tail p-value is one-half the \(\chi^2_1\) upper-tail probability at \(Q_+\). At \(Q_+=0\), the mixture places probability one-half at zero, so the one-sided p-value is not obtained by treating zero as an ordinary positive \(\chi^2\) observation.

Choosing the Tail and Reporting the Result

For a directional test, the direction must be specified before examining the estimate. The one-sided Wald, score, and likelihood-ratio statistics are all oriented so that positive values support the stated alternative. Their asymptotic reference distributions give the same normal upper-tail calculation when expressed as signed statistics, although their finite-sample values need not agree.

In the binomial illustration, the three approximate p-values were about \(0.0412\), \(0.0368\), and \(0.03482\), respectively. These differences do not change which direction was tested; they arise because the tests use different estimates of evidence. Each calculation should report the alternative, the signed statistic or constrained statistic, and the reference distribution used.

Common Pitfall

• Halving a two-sided p-value without checking the sign. The shortcut \(p_{\mathrm{one}}=p_{\mathrm{two}}/2\) applies only when the observed signed statistic is in the direction specified by the alternative. If it points the other way, the upper-tail one-sided p-value is \(1-p_{\mathrm{two}}/2\), not half the two-sided value.
• Using an ordinary \(\chi^2_1\) tail for a directional likelihood ratio. The constrained statistic has a point mass at zero under its asymptotic null distribution. For a positive statistic, use half the ordinary \(\chi^2_1\) upper tail, or use the signed root and a standard normal upper tail.
• Choosing a direction after seeing the estimate. A one-sided p-value is calibrated for an alternative specified in advance. Selecting whichever direction matches the data and then reporting a one-sided tail does not preserve the stated test level.