Discrete Neyman–Pearson Construction Workshop

Constructing a Most Powerful Discrete Test

For discrete data, the most powerful test of one simple hypothesis against another is built by comparing the probability of each possible outcome under the two hypotheses. The key is to rank outcomes by how strongly they favor the alternative, then spend the permitted probability of rejection under the null on the most informative outcomes first. Because discrete outcomes have probability mass, a deterministic rejection region may not have exactly the requested size. Randomization at a boundary resolves that issue.

This tutorial develops that construction for a single observation or a discrete sample summarized by its possible outcomes. It is aligned with the SOA VEE Mathematical Statistics syllabus.

The likelihood ratio used here is

A large ratio means that the observed outcome is relatively more probable under \(H_1\) than under \(H_0\), so it is a strong candidate for rejection. If \(p_0(x)=0\) but \(p_1(x)>0\), the outcome can be rejected without adding to the test’s size; it has an infinite likelihood ratio for this ranking purpose. Outcomes with zero probability under both hypotheses do not affect the test.

The Discrete Construction

For a desired size \(\alpha\), order the outcomes from largest to smallest likelihood ratio. Add their probabilities under \(H_0\) in that order. Reject on outcomes above the boundary; if the next outcome would take the cumulative null probability beyond \(\alpha\), reject on that boundary outcome with only the probability needed to reach \(\alpha\). If several outcomes tie at the boundary, a common randomization probability on those outcomes is a convenient choice.

In the usual threshold form, a most powerful test rejects for \(\Lambda(x)>c\), accepts for \(\Lambda(x)<c\), and rejects with probability \(\gamma\) for \(\Lambda(x)=c\), where \(c\) and \(\gamma\) are chosen to attain the target size when possible. The Neyman–Pearson result says this test has at least as much power as any other test whose size is no greater than \(\alpha\). The boundary randomization is part of the test, not an adjustment to the reported power.

Why Ranking Likelihood Ratios Works

The ranking rule can be justified by comparing the threshold test with any competing test. Let \(\phi^*\) be the threshold test with boundary \(c\), and let \(\psi\) be any test of size at most \(\alpha\). At an outcome above the threshold, \(\phi^*(x)=1\) and \(p_1(x)-c p_0(x)>0\); below it, \(\phi^*(x)=0\) and \(p_1(x)-c p_0(x)<0\). At a boundary outcome, the latter difference is zero. Consequently, at every outcome,

Summing over the sample space gives

The final inequality holds because \(E_0[\phi^*]=\alpha\), \(E_0[\psi]\leq\alpha\), and \(c\geq0\). Thus the threshold test has power at least as large as the competing test. This explains why probability mass under \(H_0\) should be assigned first to outcomes with the largest likelihood ratios.

Worked Example: A Binomial Sample

Suppose \(X\) is the number of positive results in \(n=3\) independent trials. Test \(H_0:p=0.20\) against \(H_1:p=0.60\) at size \(\alpha=0.10\). The model is \(X\sim\operatorname{Binomial}(3,p)\), with the specified success probability under each hypothesis. The outcomes are \(k=0,1,2,3\); there is no interest rate or mortality assumption.

Method: calculate the binomial likelihood ratios and accumulate null probabilities. The probability at outcome \(k\) is \(\binom{3}{k}p^k(1-p)^{3-k}\). Under \(H_0\), the probabilities for \(k=0,1,2,3\) are \(0.512,0.384,0.096,0.008\), respectively. Under \(H_1\), they are \(0.064,0.288,0.432,0.216\), respectively. Dividing the alternative probabilities by the null probabilities gives:

The ranking is \(k=3,2,1,0\). Rejecting only when \(k=3\) uses size \(0.008\). Adding \(k=2\) in full would give size \(0.008+0.096=0.104\), exceeding \(0.10\). Thus reject at \(k=3\) always and at \(k=2\) with probability \(\gamma\):

The power is

Check. The size is \(0.008+(23/24)(0.096)=0.008+0.092=0.10\), and the power is \(0.630\). The common mistake is to reject for \(k\geq2\) without checking the null probability: that rule has size \(0.104\). Randomizing at \(k=2\) attains the requested size exactly.

Common Construction Errors

Discrete tests require careful bookkeeping. A likelihood-ratio ranking determines the order of outcomes, but the target size is always calculated with probabilities under \(H_0\), while power is calculated with probabilities under \(H_1\). Keep those two calculations separate.

• Using alternative probabilities to set the size. The size constraint is \(\sum_x p_0(x)\phi(x)=\alpha\), not a sum under \(H_1\).
• Taking outcomes in numerical order automatically. The most powerful region is determined by likelihood ratios, which need not increase with the observed value.
• Rejecting the whole boundary outcome. If doing so exceeds \(\alpha\), calculate the boundary randomization probability rather than reporting an oversized test.
• Forgetting the randomization in the power calculation. A boundary outcome’s contribution to power is \(p_1(x)\gamma\), not \(p_1(x)\) in full.
• Breaking a likelihood-ratio tie without justification. Tied outcomes provide the same likelihood ratio; a common randomization probability is a direct way to meet the size target.

When the size obtained by including the highest-ranked outcomes lands exactly on \(\alpha\), the test is nonrandomized. Otherwise, randomization is generally what allows a discrete test to attain the specified size rather than merely stay below it. This distinction matters when comparing tests at a fixed size or reporting their power.