From Hypotheses to a Rule for Every Possible Observation
A hypothesis test is a decision rule: after observing data, it determines whether to reject the null hypothesis. To describe that rule precisely, it helps to specify what it does for every possible observation, rather than only giving a verbal instruction such as “reject for a sufficiently large value.” A test function provides this description. It also gives a common way to calculate the test’s behavior under each hypothesis, whether the rule is deterministic or includes randomization.
This tutorial considers a simple null hypothesis and a simple alternative hypothesis. “Simple” refers to how much each hypothesis specifies about the distribution of the data; it does not mean that the decision rule or its calculations must be simple.
For example, the statements \(H_0:X\sim N(0,1)\) and \(H_1:X\sim N(1,1)\) specify one normal distribution each, so both hypotheses are simple. In contrast, \(H_0:X\sim N(\mu,1)\) for any \(\mu\leq 0\) allows multiple distributions and is composite. The distribution under each hypothesis must be specified before its rejection probability can be calculated.
Test Functions
Let \(X\) denote the observed data, taking values in a sample space \(\mathcal{X}\). A test function, written \(\phi(x)\), assigns a rejection probability to every possible observation \(x\).
A nonrandomized rule can be described by a rejection region \(R\subseteq\mathcal{X}\): reject when \(X\in R\), and do not reject otherwise. Its test function is the indicator \(\phi(x)=\mathbf{1}_{\{x\in R\}}\), which equals 1 inside \(R\) and 0 outside \(R\). A randomized rule can, for instance, reject with probability 0.4 when \(X=x\). One implementation is to generate an independent random variable \(U\sim\text{Uniform}(0,1)\) and reject if \(U\leq\phi(x)\). This added randomization is part of the decision rule, not part of the data distribution.
A test function is useful because the same calculation covers either kind of rule. Under a specified distribution, its expected value is the probability of rejection. The null and alternative distributions therefore determine two distinct expectations.
For a discrete sample space, the expectation under \(P_j\) is \(\sum_{x\in\mathcal{X}}\phi(x)P_j(X=x)\), for \(j=0,1\). For a continuous observation with density \(f_j\), it is \(\int_{\mathcal{X}}\phi(x)f_j(x)\,dx\). If the model includes both discrete and continuous parts, the expectation is taken with respect to the specified probability distribution, including both parts.
The size is the probability of rejecting when the null distribution is true. If a test is designed to have size no greater than a target \(\alpha\), its requirement is \(E_0[\phi(X)]\leq\alpha\). The actual size need not equal the target. Power is the probability of rejecting when the specified alternative distribution is true. Because the alternative here is simple, this is one particular power value; it is not a guarantee about every distribution outside the null.
Why the Expectation Gives a Rejection Probability
The expectation formula follows by considering the rejection indicator. Let \(D\) equal 1 if the rule rejects and 0 otherwise. Given \(X=x\), the test function is exactly the conditional rejection probability, so \(E[D\mid X=x]=\phi(x)\). Taking expectations under either hypothesis gives \(E_j[D]=E_j[E_j(D\mid X)]=E_j[\phi(X)]\). Since a zero-one indicator has expectation equal to the probability of the event it indicates, \(E_j[D]=P_j(\text{reject }H_0)\).
For a discrete observation, the same reasoning can be seen directly: the probability of observing \(x\) under \(P_j\) is \(P_j(X=x)\), and the probability of rejection conditional on that observation is \(\phi(x)\). Adding the joint probabilities of observing \(x\) and rejecting gives \(\sum_x P_j(X=x)\phi(x)\). Thus changing from \(P_0\) to \(P_1\) changes the weights in the sum, not the test function.
Worked Example: Size and Power for a Randomized Categorical Test
A hypothetical screening device reports one of three outcomes: A, B, or C. Under \(H_0\), the probabilities are \(P_0(A)=0.50\), \(P_0(B)=0.30\), and \(P_0(C)=0.20\). Under \(H_1\), they are \(P_1(A)=0.20\), \(P_1(B)=0.30\), and \(P_1(C)=0.50\). A test does not reject for A, rejects with probability 0.25 for B, and always rejects for C. Find its size, power, and Type II error probability.
Method. Represent the rule by \(\phi(A)=0\), \(\phi(B)=0.25\), and \(\phi(C)=1\). For each operating characteristic, weight these same rejection probabilities by the outcome probabilities under the relevant hypothesis.
Under \(H_0\), the size is \(\alpha_\phi=0(0.50)+0.25(0.30)+1(0.20)=0+0.075+0.20=0.275\).
Under \(H_1\), the power is \(\pi_\phi=0(0.20)+0.25(0.30)+1(0.50)=0+0.075+0.50=0.575\). Therefore the Type II error probability is \(\beta_\phi=1-0.575=0.425\).
The checks are that all three probabilities under each hypothesis sum to 1, and that power plus Type II error is \(0.575+0.425=1\). The size is 0.275, not the probability of outcome C alone: outcome B also leads to rejection some of the time.
Worked Example: A Nonrandomized Test With Normal Models
A hypothetical measurement \(X\) has distribution \(N(0,1)\) under \(H_0\) and \(N(1,1)\) under \(H_1\), where the second parameter is the variance. The test rejects \(H_0\) when \(X>0.5\). Find its size, power, and Type II error probability. Also state the decision for an observed value \(x=0.4\).
Method. This is a nonrandomized test with \(\phi(x)=\mathbf{1}_{\{x>0.5\}}\). Write each expectation as the probability of entering the rejection region under that hypothesis. Both distributions have standard deviation 1.
Under \(H_0\), standardize using \(Z=X\sim N(0,1)\). The size is \(\alpha_\phi=P_0(X>0.5)=1-\Phi(0.5)=1-0.6915=0.3085\), where \(\Phi\) is the standard normal cumulative distribution function.
Under \(H_1\), write \(Z=X-1\sim N(0,1)\). Then \(\pi_\phi=P_1(X>0.5)=P(Z>-0.5)=\Phi(0.5)=0.6915\). Thus \(\beta_\phi=1-0.6915=0.3085\). The equality of size and Type II error here results from these particular symmetric distributions and threshold; it is not a general identity.
For \(x=0.4\), the observed value does not satisfy \(x>0.5\), so \(\phi(0.4)=0\) and the test does not reject. The probability calculations are rounded to four decimal places. As a check, the null and alternative rejection probabilities are both between 0 and 1, and the alternative rejection probability is larger for this threshold.
Reading and Checking Test-Function Calculations
A reliable calculation keeps three items separate: the test function, the probability distribution, and the operating characteristic being requested. The function describes the decision. The distribution supplies the probabilities or density used to average that function. The size uses the null distribution; power uses the alternative distribution. The test function itself does not change when calculating these two quantities.
Write the single distribution under \(H_0\) and the single distribution under \(H_1\), including any relevant parameters.
Give \(\phi(x)\) for each possible observation, or state a rejection region and its indicator function.
Compute \(E_0[\phi(X)]\) to obtain the test size.
Compute \(E_1[\phi(X)]\) to obtain power, then subtract it from 1 for the Type II error probability at that alternative.
Worked Example: Comparing a Randomized Rule With a Deterministic Rule
A hypothetical sensor reports L, M, or H. Under \(H_0\), the probabilities are \(P_0(L)=0.60\), \(P_0(M)=0.30\), and \(P_0(H)=0.10\). Under \(H_1\), they are \(P_1(L)=0.20\), \(P_1(M)=0.40\), and \(P_1(H)=0.40\). Rule R rejects with probabilities 0.10, 0.50, and 0.90 for L, M, and H, respectively. Rule D rejects only for H. Calculate the size and power of each rule.
Method. First write the functions: \(\phi_R=(0.10,0.50,0.90)\) and \(\phi_D=(0,0,1)\) in the order L, M, H. Apply each function to the null probabilities for size, then to the alternative probabilities for power.
For Rule R, the size is \(\alpha_R=0.10(0.60)+0.50(0.30)+0.90(0.10)=0.06+0.15+0.09=0.30\). Its power is \(\pi_R=0.10(0.20)+0.50(0.40)+0.90(0.40)=0.02+0.20+0.36=0.58\).
For Rule D, the size is \(\alpha_D=0(0.60)+0(0.30)+1(0.10)=0.10\). Its power is \(\pi_D=0(0.20)+0(0.40)+1(0.40)=0.40\). The corresponding Type II error probabilities are \(1-0.58=0.42\) for Rule R and \(1-0.40=0.60\) for Rule D.
The check is that each rule’s size and power are weighted averages of its rejection probabilities, so each lies between the smallest and largest function values. Rule R has both greater size and greater power than Rule D in these specified models. This comparison alone does not establish that either rule is optimal.
Common Mistake: Using the Wrong Distribution
A frequent error is to calculate power using the null probabilities, or size using the alternative probabilities. That swaps the questions: size asks how often the test rejects when \(H_0\) is true, while power asks how often it rejects when \(H_1\) is true. In the sensor example, using the null probabilities for Rule R gives \(0.30\). That is the size, not the power. The correct power uses the alternative probabilities and is \(0.58\).
Another error is to treat \(\phi(x)\) as the probability that the observation equals \(x\). It is not. It is the conditional probability of rejecting after observing \(x\). The model probabilities \(P_j(X=x)\) describe how likely the observation is under hypothesis \(j\); multiplying them by \(\phi(x)\) accounts for how often that outcome leads to rejection.
Check Your Understanding
For each problem, identify which distribution is used for each requested operating characteristic.
- Under \(H_0\), an outcome has probabilities \(0.70\) for A and \(0.30\) for B; under \(H_1\), the probabilities are \(0.40\) for A and \(0.60\) for B. A test has \(\phi(A)=0.20\) and \(\phi(B)=0.80\). Find size, power, and Type II error. (Answer: size \(=0.38\), power \(=0.56\), Type II error \(=0.44\).)
- Let \(X\sim N(0,1)\) under \(H_0\) and \(X\sim N(0.5,1)\) under \(H_1\), with variance 1 in both models. A nonrandomized test rejects when \(X>0\). Find size and power to four decimal places. Use \(\Phi(0.5)=0.6915\). (Answer: size \(=0.5000\), power \(=0.6915\).)
- A test function is \(\phi(x)=0\) for \(x<2\), \(\phi(2)=0.35\), and \(\phi(x)=1\) for \(x>2\). Is the test randomized? What is its rejection probability when the observation is exactly 2? (Answer: It is randomized at \(x=2\); the rejection probability there is \(0.35\).)
- For outcomes A, B, C, suppose \(P_0=(0.20,0.50,0.30)\), \(P_1=(0.50,0.30,0.20)\), and \(\phi=(1,0.40,0)\), in that order. Find the size, power, and Type II error. (Answer: size \(=0.40\), power \(=0.62\), Type II error \(=0.38\).)
- A researcher states \(H_0:X\sim N(\mu,1)\) for any \(\mu\leq 0\), and \(H_1:X\sim N(1,1)\). Which hypothesis is composite, and why can its size not be determined from one null distribution alone? (Answer: \(H_0\) is composite because it permits multiple values of \(\mu\); its size must account for rejection probabilities across the permitted null distributions, not just one unspecified distribution.)