Choosing a Critical Constant for a Target Size

From a Target Size to a Likelihood-Ratio Cutoff

The Neyman–Pearson test ranks sample outcomes by how strongly they favor the alternative over the null. To use that ranking, a test still needs a cutoff. A prescribed significance level, or target size, determines the cutoff through the probability of rejection when the null hypothesis is true.

The calculation is straightforward when the likelihood ratio has a continuous distribution under the null: integrate the null density over the rejection region and set the result equal to the target size. When the likelihood ratio has probability mass at the cutoff, the distinction between including and excluding the boundary matters. A deterministic test may then be unable to attain the target size exactly.

Definition: Suppose the hypotheses are simple, $H_0:\theta=\theta_0$ and $H_1:\theta=\theta_1$, and the sample outcome is $x$. Let $f_0(x)$ and $f_1(x)$ denote the probability mass functions or densities under the two hypotheses. On outcomes with $f_0(x)>0$, the likelihood ratio in favor of $H_1$ is $\Lambda(x)=f_1(x)/f_0(x)$. A likelihood-ratio cutoff test rejects $H_0$ for outcomes with sufficiently large $\Lambda(x)$.

For a discrete model, $f_j(x)$ denotes the probability of the outcome $x$ under $H_j$. For a continuous model, it denotes a density, and probabilities are obtained by integration. The likelihood ratio compares the two likelihoods at the same observed outcome; it is not a posterior probability.

The size of a nonrandomized test is its rejection probability under $H_0$. A test with target size $\alpha$ should have null rejection probability no greater than $\alpha$; it has size exactly $\alpha$ when that probability equals $\alpha$. The calculations below use the convention that a test rejects when $\Lambda(x)>c$, with $c$ the critical constant. Including outcomes for which $\Lambda(x)=c$ changes the size if those outcomes have positive probability under $H_0$.

Key result: For a threshold $c$, the strict-tail size is $P_0(\Lambda(X)>c)$. To locate a cutoff at a target size $\alpha$, account for both strict and inclusive tails: $$ P_0(\Lambda(X)>c)\ \leq\ \alpha\ \leq\ P_0(\Lambda(X)\geq c). $$ If the likelihood ratio has no probability mass at $c$, the two tails are equal. If it has positive boundary mass, they differ, and a nonrandomized threshold may not give size exactly $\alpha$.

Calculating the Null Probability

The key is to express the rejection region in terms of the observed data, then calculate its probability under $H_0$. For a discrete sample space, add the null probabilities of the outcomes in the region. For a continuous sample, integrate the null density over the region. If the region has several pieces, include every piece.

\begin{aligned} \text{Discrete:}\qquad P_0(\Lambda(X)>c) &=\sum_{x:\,\Lambda(x)>c} P_0(X=x),\\ \text{Continuous:}\qquad P_0(\Lambda(X)>c) &=\int_{\{x:\,\Lambda(x)>c\}} f_0(x)\,dx. \end{aligned}

These formulas follow directly from the definition of probability. In a discrete model, the event $\{\Lambda(X)>c\}$ consists of the outcomes whose likelihood ratios exceed the cutoff, so its probability is the sum of their null probabilities. In a continuous model, the same event is a region on the sample space, and its null probability is the integral of $f_0$ over that region. The likelihood ratio determines which outcomes enter the region; the null distribution determines the probability assigned to it.

A useful way to see the role of a boundary is to let $T=\Lambda(X)$ under $H_0$. As $c$ increases, $P_0(T>c)$ cannot increase: raising the cutoff can only remove outcomes from the rejection region. If $T$ is continuous at the cutoff, a small change in $c$ changes the tail probability smoothly. If $T$ has a point mass, the tail probability jumps by $P_0(T=c)$ as the cutoff passes that value.

Worked Example: A Continuous Likelihood Ratio

A single measurement $X$ has density $f_0(x)=1$ for $0<x<1$ under $H_0$, and density $f_1(x)=2x$ for $0<x<1$ under $H_1$. Both densities are zero outside this interval. Find the likelihood-ratio cutoff for target size $\alpha=0.20$, and calculate the power of the resulting test. Use the strict rejection rule $\Lambda(X)>c$.

Method. Form the likelihood ratio, express its strict rejection region as a range of $x$, and integrate the null density over that range.

Step 1: Calculate the likelihood ratio. For $0<x<1$, $$ \Lambda(x)=\frac{f_1(x)}{f_0(x)}=\frac{2x}{1}=2x. $$

Step 2: Express the rejection region. The rule $\Lambda(x)>c$ becomes $2x>c$, or $x>c/2$. For a cutoff between 0 and 2, its null probability is $$ P_0(X>c/2)=\int_{c/2}^{1}1\,dx=1-\frac{c}{2}. $$

Step 3: Set the null probability equal to the target size. $$ 1-\frac{c}{2}=0.20 \quad\Longrightarrow\quad \frac{c}{2}=0.80 \quad\Longrightarrow\quad c=1.60. $$ The rejection region is $X>0.80$. Under the null, $P_0(X>0.80)=1-0.80=0.20$, as required.

Step 4: Calculate and check the power. Power is the rejection probability under $H_1$: $$ P_1(X>0.80)=\int_{0.80}^{1}2x\,dx =\left[x^2\right]_{0.80}^{1} =1-0.64=0.36. $$ The test has size $0.20$ and power $0.36$. The likelihood ratio is continuous under the null, so the boundary $X=0.80$ has null probability zero.

Common mistake to avoid: Setting $P_0(X<c/2)$ equal to $0.20$ would use the lower tail, even though the test rejects for large likelihood ratios and therefore for large $X$. The correct probability is the upper tail $P_0(X>c/2)$.

Boundary Mass in a Discrete Model

For a discrete observation, several outcomes can have the same likelihood ratio. If a target size falls between the strict-tail probability and the inclusive-tail probability at that ratio, the boundary outcomes are the reason a deterministic cutoff cannot match the target exactly. It is essential to identify which outcomes are strictly above the cutoff and which are equal to it.

Worked Example: A Target Size Between Two Attainable Sizes

A hypothetical quality audit records one of three outcomes, $A$, $B$, or $C$. Under $H_0$, their probabilities are $0.70$, $0.20$, and $0.10$, respectively. Under $H_1$, they are $0.20$, $0.40$, and $0.40$. Find the likelihood-ratio cutoff for target size $\alpha=0.25$, and determine the size and power of the nonrandomized cutoff test that does not exceed the target.

Method. Calculate each outcome’s likelihood ratio, order the outcomes, then add null probabilities according to the strict rejection rule.

Step 1: Calculate the likelihood ratios. $$ \Lambda(A)=\frac{0.20}{0.70}=\frac{2}{7}, \qquad \Lambda(B)=\frac{0.40}{0.20}=2, \qquad \Lambda(C)=\frac{0.40}{0.10}=4. $$ Thus $C$ has the largest ratio, followed by $B$, then $A$.

Step 2: Locate the target among null tail probabilities. At $c=2$, the strict rejection region $\{\Lambda>2\}$ contains only $C$, so $$ P_0(\Lambda>2)=P_0(C)=0.10. $$ The inclusive region $\{\Lambda\geq 2\}$ contains $B$ and $C$, so $$ P_0(\Lambda\geq 2)=P_0(B)+P_0(C)=0.20+0.10=0.30. $$ Therefore $0.10\leq 0.25\leq 0.30$. The target lies within the boundary jump at $c=2$.

Step 3: Apply the deterministic strict rule. With $c=2$ and rejection only for $\Lambda>2$, the test rejects only for $C$. Its size is $0.10$, which does not exceed $0.25$. Including $B$ as well would give size $0.30$, which exceeds the target. No deterministic likelihood-ratio cutoff attains size exactly $0.25$.

Step 4: Check the power. Under $H_1$, the probability of $C$ is $0.40$, so the strict-rule test has power $0.40$. Its size is $0.10$ under $H_0$, confirming that it meets the size constraint.

Common mistake to avoid: Treating $P_0(\Lambda\geq2)=0.30$ as the size of the strict rule counts $B$ even though $\Lambda(B)=2$, not more than 2. The strict rule’s size is $0.10$; the inclusive rule’s size is $0.30$.

A Normal-Model Cutoff

A continuous distribution often lets the target size be translated into a familiar tail probability. The likelihood-ratio algebra must still be done first: it identifies whether the rejection region is an upper tail or a lower tail of the observation. The null distribution then supplies the cutoff.

Worked Example: A Normal Mean Shift With Known Variance

A measurement $X$ has distribution $N(0,1)$ under $H_0$ and $N(1,1)$ under $H_1$, where the second parameter is the variance. Find the critical constant and observation cutoff for target size $\alpha=P(Z>1)$, where $Z\sim N(0,1)$. Then calculate the power.

Method. Form the ratio of the two normal densities and solve the size equation using the upper-tail probability under $H_0$.

Step 1: Form and simplify the likelihood ratio. The common normalizing constants cancel: $$ \Lambda(x) =\frac{\exp\!\left(-\frac{(x-1)^2}{2}\right)} {\exp\!\left(-\frac{x^2}{2}\right)} =\exp\!\left(\frac{x^2-(x-1)^2}{2}\right) =\exp\!\left(x-\frac12\right). $$ Since this expression increases with $x$, the rejection region $\Lambda(x)>c$ is an upper tail. Taking logarithms gives $x>\log(c)+\tfrac12$.

Step 2: Use the target size to find the observation cutoff. Under $H_0$, $X\sim N(0,1)$. The cutoff $x=1$ gives $$ P_0(X>1)=P(Z>1)=\alpha\approx0.1587. $$ Thus the rejection region is $X>1$.

Step 3: Convert the observation cutoff to a likelihood-ratio cutoff. At $x=1$, the likelihood ratio is $$ c=\Lambda(1)=\exp\!\left(1-\frac12\right)=e^{1/2}\approx1.6487. $$ The rule $\Lambda(X)>e^{1/2}$ is equivalent to $X>1$. Because the null distribution is continuous, equality at the boundary has probability zero.

Step 4: Calculate and check the power. Under $H_1$, $X\sim N(1,1)$, so $$ P_1(X>1)=P\!\left(\frac{X-1}{1}>0\right)=P(Z>0)=0.5000. $$ The null rejection probability is approximately $0.1587$, matching the target, and the power is $0.5000$.

Common mistake to avoid: The null size is not $P_0(X<1)$. The likelihood ratio increases with $x$, so large values favor $H_1$; the size calculation must use $P_0(X>1)$.

Practical Checks and Limits

Before reporting a critical constant, verify the direction of the rejection region and calculate its probability under the null—not under the alternative. Then check whether the boundary has positive null probability. These checks prevent two common errors: using the wrong tail and silently treating a discrete boundary as if it had probability zero.

State the rule precisely. Write whether rejection occurs for $\Lambda>c$ or for $\Lambda\geq c$. The two rules need not have the same size.
Use the null model for size. The likelihood ratio ranks outcomes, but the null probabilities or density determine the size.
Check the boundary. If $P_0(\Lambda=c)>0$, compare the strict and inclusive tails before claiming the target is attained.
Check the support. The ratio $f_1/f_0$ is defined directly where $f_0>0$. If an outcome can occur under $H_1$ but has zero probability under $H_0$, it has no null size contribution and must be handled as an outcome with infinite likelihood ratio.

The critical constant is therefore not chosen by a universal numerical rule. It is determined by the model, the likelihood-ratio ordering, the target size, and—when the ratio has atoms—the mass at the boundary. This calculation is aligned with the SOA VEE Mathematical Statistics syllabus.

Check Your Understanding

For each problem, identify the likelihood-ratio rejection region, use the null distribution to set the cutoff, and check the resulting size.

Under $H_0$, $X$ is uniform on $(0,1)$; under $H_1$, its density is $3x^2$ on $(0,1)$. For target size $0.19$, find the likelihood-ratio cutoff, the observation cutoff, and the power. (Answer: $c=1.9683$, reject for $X>0.81$, power $0.4686$.)
Under $H_0$, $X\sim N(0,4)$; under $H_1$, $X\sim N(2,4)$. For target size $P(Z>1)$, find the observation cutoff and likelihood-ratio cutoff. (Answer: reject for $X>2$; $c=e^{1/2}\approx1.6487$; size $P(Z>1)\approx0.1587$.)
A categorical observation has null probabilities $(0.60,0.25,0.15)$ and alternative probabilities $(0.30,0.50,0.20)$ for outcomes $A,B,C$, respectively. For target size $0.32$, find the boundary ratio and the size of the strict-cutoff test that does not exceed the target. (Answer: ratios $0.5,2,4/3$; use $c=4/3$, reject only for $B$, and the size is $0.25$; including $C$ would make it $0.40$.)
Under $H_0$, $X$ has an exponential distribution with rate $1$; under $H_1$, it has an exponential distribution with rate $2$. For target size $0.25$, find the likelihood-ratio cutoff, the observation rejection region, and the power. (Answer: $c=1.5$, reject for $X<-\log(0.75)\approx0.2877$, and power $0.4375$.)

What You'll Learn