One-Sided Likelihood Tests and Signed Root Statistics

Why Direction Matters in Likelihood Testing

A likelihood-ratio statistic measures how much worse the likelihood is at a null value than at its unrestricted maximum. By itself, that loss is nonnegative and does not say whether the estimate lies above or below the null value. That distinction matters when the research question is directional: evidence for a parameter being larger than a threshold is not the same as evidence for it being smaller.

The signed likelihood-root statistic combines the size of the likelihood loss with the direction of the estimate. Under standard regularity conditions, it is approximately standard normal under the null. Its sign determines which tail supplies the p-value, while its magnitude reflects the likelihood evidence against the null.

Definition: Let $\theta$ be a scalar parameter of interest, let $\eta$ denote any nuisance parameters, and let $\ell_p(\theta)$ be the profile log likelihood: the log likelihood maximized over $\eta$ at each fixed $\theta$. Let $\hat{\theta}$ maximize $\ell_p(\theta)$. The signed likelihood root for testing $\theta=\theta_0$ is $$ r(\theta_0)=\operatorname{sign}(\hat{\theta}-\theta_0)\sqrt{2\{\ell_p(\hat{\theta})-\ell_p(\theta_0)\}}. $$ If there are no nuisance parameters, $\ell_p$ is simply the log likelihood. The quantity under the square root is the likelihood-ratio statistic for the point null; the sign records whether the estimate is above or below $\theta_0$.

Directional Hypotheses and Tail Areas

For the point null $H_0:\theta=\theta_0$, a regular scalar model gives $r(\theta_0)$ an approximate $N(0,1)$ distribution under $H_0$. The approximation relies on an interior null value, an identifiable parameter, a sufficiently smooth likelihood, positive information, and the usual conditions supporting likelihood asymptotics. When nuisance parameters are present, the profile likelihood must also be regular. These conditions are the setting for the standard likelihood results used here; nonstandard parameter-space boundaries require separate treatment.

For an upper-sided alternative $H_a:\theta>\theta_0$, large positive roots support the alternative. For a lower-sided alternative $H_a:\theta<\theta_0$, large negative roots support it. Thus the approximate p-values are different tail areas, not the same likelihood loss interpreted in two directions.

One-sided p-values: With $\Phi$ denoting the standard normal cumulative distribution function, the approximate p-value is $1-\Phi(r_{\mathrm{obs}})$ for $H_a:\theta>\theta_0$, and $\Phi(r_{\mathrm{obs}})$ for $H_a:\theta<\theta_0$. For a two-sided alternative $H_a:\theta\ne\theta_0$, the corresponding approximation is $2\{1-\Phi(|r_{\mathrm{obs}}|)\}$.

The two-sided likelihood-ratio test uses $2\{\ell_p(\hat{\theta})-\ell_p(\theta_0)\}=r(\theta_0)^2$. Under the regular null its limiting distribution is chi-squared with one degree of freedom. Squaring removes the sign, which is appropriate when departures in either direction count as evidence, but not when the direction is part of the question. A one-sided test uses the sign as well as the magnitude.

Worked Example: An Upper-Sided Test in an Exponential Model

Suppose 12 hypothetical independent waiting times $X_1,\ldots,X_{12}$ follow an exponential model with rate $\theta>0$, so the density is $f(x;\theta)=\theta e^{-\theta x}$ for $x>0$. Assume the model is correctly specified and the observations are independent. The hypothetical waiting times have sum $9.6$ time units, giving $\hat{\theta}=12/9.6=1.25$ per time unit. Test $H_0:\theta=1$ against $H_a:\theta>1$.

The log likelihood, omitting no parameter-dependent terms, is $\ell(\theta)=12\log(\theta)-9.6\theta$. Its maximum occurs at $\hat{\theta}=1.25$, as follows either by differentiating or by using the exponential-rate maximum-likelihood estimate $n/\sum_i X_i$. The log-likelihood loss at the null is

\ell(1.25)-\ell(1) =12\log(1.25)-9.6(1.25-1) =12\{\log(1.25)-1+0.8\} =0.2777226.

Because $\hat{\theta}-\theta_0=0.25>0$, the signed root is positive:

r(1)=+\sqrt{2(0.2777226)} =\sqrt{0.5554452} \approx 0.7453.

For the upper-sided alternative, the approximate p-value is $1-\Phi(0.7453)\approx0.2281$. At a 5% significance level, this is not enough evidence to reject the point null in favor of a larger rate. The calculation is an asymptotic likelihood test; with only 12 observations, its calibration may be less accurate than in a large sample.

How the Normal Approximation Arises

The signed root has its normal approximation because the profile log likelihood is locally quadratic near its maximum. The curvature measures the information available about the scalar parameter after nuisance parameters have been profiled out. The following argument gives the link between that local quadratic shape and the root’s standard normal limit.

Use the local likelihood expansion.
Near the true value $\theta_0$, regularity gives a quadratic approximation to the profile log likelihood. Write $I_{\mathrm{eff}}$ for the efficient information for $\theta$, on the scale of the full sample. At the maximum, this yields $2\{\ell_p(\hat{\theta})-\ell_p(\theta_0)\}\approx I_{\mathrm{eff}}(\hat{\theta}-\theta_0)^2$.

Retain the direction.
Taking the signed square root of that approximation gives $r(\theta_0)\approx\sqrt{I_{\mathrm{eff}}}(\hat{\theta}-\theta_0)$. The sign is unchanged because $I_{\mathrm{eff}}>0$.

Apply asymptotic normality.
Under the regular null, the maximum-likelihood estimator satisfies $\sqrt{I_{\mathrm{eff}}}(\hat{\theta}-\theta_0)$ approximately distributed as $N(0,1)$. Therefore the signed root is also approximately $N(0,1)$. This is an asymptotic justification, not a claim that the root is exactly normal in every finite sample.

Worked Directional and Two-Sided Comparisons

Worked Example: Known-Variance Normal Mean

Suppose 25 hypothetical independent observations follow $X_i\sim N(\theta,0.8^2)$, with known standard deviation $0.8$. Assume this normal model and independence are appropriate. The observed sample mean is $\bar{x}=1.2$. Test $H_0:\theta=1$ against $H_a:\theta>1$.

For a normal mean with known variance, the log-likelihood difference from the unrestricted maximum to the value $\theta_0$ is $n(\bar{x}-\theta_0)^2/(2\sigma^2)$. Substituting the data gives

\ell(\hat{\theta})-\ell(1) =\frac{25(1.2-1)^2}{2(0.8)^2} =\frac{25(0.04)}{1.28} =0.78125.

The estimate is above the null value, so

r(1)=+\sqrt{2(0.78125)} =\sqrt{1.5625} =1.25.

The upper-sided approximate p-value is $1-\Phi(1.25)\approx0.1056$. The data therefore do not provide 5%-level evidence that the mean exceeds 1. The two-sided likelihood-ratio test instead has approximate p-value $2\{1-\Phi(1.25)\}\approx0.2113$. That larger p-value answers a different question: whether the mean differs from 1 in either direction.

A nuisance parameter does not change the direction rule. It changes how the likelihood loss is calculated: at each candidate value of $\theta$, maximize over the nuisance parameter, then compare the resulting profile log likelihoods.

Worked Example: Profiling an Unknown Variance

Suppose 16 hypothetical independent observations follow $X_i\sim N(\theta,\sigma^2)$, where both the mean $\theta$ and variance $\sigma^2>0$ are unknown. Assume the normal model is appropriate and the true parameter is an interior point. The summaries are $\bar{x}=2.4$ and $\sum_i(X_i-\bar{x})^2=15$. Test $H_0:\theta=2$ against $H_a:\theta>2$, profiling over the variance.

At a fixed $\theta$, the variance maximum-likelihood estimate is $16^{-1}\sum_i(X_i-\theta)^2$. The identity $\sum_i(X_i-\theta)^2=\sum_i(X_i-\bar{x})^2+16(\bar{x}-\theta)^2$ therefore gives the profile likelihood loss:

2\{\ell_p(\hat{\theta})-\ell_p(\theta_0)\} =16\log\left( \frac{15+16(2.4-2)^2}{15} \right) =16\log\left(1+\frac{2.56}{15}\right) \approx 2.5212.

Since $\hat{\theta}=\bar{x}=2.4>2$, the root is positive:

r(2)=+\sqrt{2.5212}\approx1.5878.

The upper-sided approximate p-value is $1-\Phi(1.5878)\approx0.0562$. This is close to, but above, 0.05, so the asymptotic test does not reject at the 5% level. The result is based on the profile likelihood and its regular large-sample normal calibration; it is not a claim that the finite-sample one-sided p-value is exactly 0.0562.

Point Nulls, Composite Nulls, and Interpretation

A directional research question is often written with a composite null, such as $H_0:\theta\le\theta_0$ against $H_a:\theta>\theta_0$, rather than with the point null $H_0:\theta=\theta_0$. In a regular one-sided test, the boundary value $\theta_0$ is commonly the least-favorable null value: it is the null value most likely to produce an upper-tail rejection. The signed-root tail calculation is then made at that boundary. This statement relies on the usual regularity and null-calibration conditions; for a more general composite null, the size of the test must be controlled over all null values, not merely assumed from one calculation.

For the point-null signed root, a negative observed value in an upper-sided test points away from the alternative, so the upper-tail p-value exceeds one half. A test based only on $r^2$ would erase that information and could treat a large estimate in the wrong direction as strong evidence for the directional alternative. The sign is therefore part of the statistic’s inferential meaning, not a cosmetic convention.

Interpretation: A one-sided p-value is the probability, assuming the null model and its calibration conditions, of obtaining a signed-root statistic at least as far in the direction of the alternative as the observed statistic. It is not the probability that the null hypothesis is true, nor the probability that the observed direction is caused by chance.

Common Mistake

Wrong reasoning: “The likelihood-ratio statistic is positive, so I can compare its square root with the upper tail of the normal distribution, even if the estimate is below the null.” The likelihood-ratio statistic is always nonnegative; its square root alone contains no direction.

Correction: First determine the sign of $\hat{\theta}-\theta_0$, attach that sign to the square root, and then select the tail matching $H_a$. For an upper-sided alternative use $1-\Phi(r_{\mathrm{obs}})$; for a lower-sided alternative use $\Phi(r_{\mathrm{obs}})$. For a two-sided alternative, discard direction only after taking the absolute value, using $2\{1-\Phi(|r_{\mathrm{obs}}|)\}$.

Key takeaway: The signed likelihood root preserves both the likelihood loss and the direction of the estimate. Use its sign to align the test with the stated alternative; use the squared root only when the question is genuinely two-sided.

Check Your Understanding

Try these questions using the definitions and tail rules above.

If $\hat{\theta}<\theta_0$, what is the sign of the signed likelihood root, and which tail gives the p-value for $H_a:\theta<\theta_0$?
Why does a two-sided likelihood-ratio test not distinguish an estimate above the null from one equally far below it?
A signed root is $r_{\mathrm{obs}}=-1.4$. Write the approximate p-value for $H_a:\theta>\theta_0$ and explain why its direction is not favorable to that alternative.
In a model with nuisance parameters, which likelihood is used to calculate the signed root for the scalar target?
What regularity feature makes the standard normal approximation for the signed root potentially inappropriate at a parameter-space boundary?