Why Look at Nonrejection Across Parameter Values?
A test is often summarized by its significance level and its power at one alternative value. Those summaries are useful, but they do not show how the test behaves across a range of possible parameter values. An operating-characteristic (OC) curve provides that broader view: it plots the probability that the test does not reject as the parameter varies.
The curve is useful for seeing whether a test separates values in the null hypothesis from values in an alternative hypothesis. A high probability of nonrejection at a null value is generally desirable; a low probability of nonrejection at an alternative value is generally desirable. The OC curve and the power curve describe the same test from complementary perspectives.
At a parameter value in the alternative, the probability of nonrejection is the Type II error probability at that value, often denoted \(\beta(\theta)\). For a composite alternative, this probability generally changes with \(\theta\); there is not one Type II error probability that describes every alternative value. At a null value, the probability of rejection contributes to the test’s size. For a composite null, the size is the supremum of the rejection probability over all parameter values in the null hypothesis.
Constructing and Reading an OC Curve
To construct an OC curve, first state the model and the test’s rejection rule. Then, for each parameter value of interest, find the probability that the observed data fall outside the rejection region. Plot those probabilities against the parameter values. The calculation may use a probability mass function for discrete data or a cumulative distribution function for continuous data.
State the sampling distribution as a function of the parameter and identify the rejection and nonrejection regions.
For each parameter value, add or integrate the probabilities over the nonrejection region.
Plot \(L(\theta)\) against \(\theta\). A value near 1 means nonrejection is likely there; a value near 0 means rejection is likely there.
The relationship between OC and power follows directly from complementary events. For any fixed \(\theta\), the event “do not reject” is the complement of “reject.” Therefore their probabilities sum to one:
This identity is the key to switching between the two curves: the OC curve decreases wherever the power curve increases, and vice versa. In a test whose power increases as \(\theta\) moves farther into a one-sided alternative, its OC curve decreases in that direction. This pattern is common, but it is not automatic for every model and test; the rejection rule and sampling distribution determine the shape.
Worked Example: A Normal-Model OC Curve
Assume the statistic \(X\) has distribution \(N(\theta,1)\), where the second parameter is the variance. Test \(H_0:\theta=0\) against a right-sided alternative by rejecting when \(X>1.645\). Find the OC probabilities at \(\theta=0,1,2\), and determine the smallest \(\theta\) for which the probability of nonrejection is at most 0.20. Give probabilities to four decimal places.
Method and nonrejection region. The test does not reject exactly when \(X\leq 1.645\). Under a given \(\theta\), standardizing \(X\) gives \(Z=X-\theta\sim N(0,1)\). Hence \(L(\theta)=P_\theta(X\leq1.645)=\Phi(1.645-\theta)\), where \(\Phi\) is the standard normal cumulative distribution function.
At \(\theta=0\). The standardized cutoff is \(1.645-0=1.645\), so \(L(0)=\Phi(1.645)\approx0.9500\). The rejection probability at the null value is approximately \(1-0.9500=0.0500\).
At \(\theta=1\). The standardized cutoff is \(1.645-1=0.645\), so \(L(1)=\Phi(0.645)\approx0.7405\). The power is \(1-0.7405=0.2595\), to four decimal places.
At \(\theta=2\). The standardized cutoff is \(1.645-2=-0.355\), so \(L(2)=\Phi(-0.355)\approx0.3613\). The power is \(1-0.3613=0.6387\), to four decimal places.
Find the parameter threshold. We need \(\Phi(1.645-\theta)\leq0.20\). The 0.20 standard normal quantile is approximately \(-0.8416\). Since \(\Phi\) is increasing, \(1.645-\theta\leq-0.8416\), so \(\theta\geq1.645+0.8416=2.4866\). Thus the smallest value is approximately \(2.4866\). As \(\theta\) moves farther above zero, nonrejection becomes less likely and power increases.
Check and trap avoided. At \(\theta=1\), the two probabilities sum to \(0.7405+0.2595=1.0000\). A common mistake is to call \(0.7405\) the power; it is the OC value, or probability of nonrejection. The power is its complement, \(0.2595\).
Discrete Models and Stepwise Behavior
For discrete data, an OC curve is calculated by summing probability masses in the nonrejection region. It may still be a smooth function of a model parameter, but a test’s attainable rejection probabilities can be restricted by the discreteness of its statistic. In particular, the test’s size need not equal a desired nominal level exactly when its rejection rule is nonrandomized.
Worked Example: A Binomial Test with a Discrete Rejection Rule
A hypothetical quality-control study counts \(X\), the number of defective units in a sample of five. Assume independent units, each defective with probability \(p\), so \(X\sim\operatorname{Binomial}(5,p)\). Test \(H_0:p=0.30\) against larger defect probabilities by rejecting when \(X\geq4\). Find the OC values at \(p=0.30,0.60,0.80\), and interpret them. Give probabilities to four decimal places.
Method and nonrejection event. Nonrejection occurs when \(X\leq3\). It is convenient to calculate the complement: \(L(p)=1-P_p(X\geq4)\). The only rejection counts are 4 and 5, so the binomial formula gives \(P_p(X\geq4)=\binom54p^4(1-p)+p^5=5p^4-4p^5\).
At \(p=0.30\). Here \(p^4=0.0081\) and \(p^5=0.00243\). Therefore \(P(X\geq4)=5(0.0081)-4(0.00243)=0.0405-0.00972=0.03078\). Thus \(L(0.30)=1-0.03078=0.96922\), or \(0.9692\) to four decimal places. This test’s rejection probability at the null value is \(0.0308\), not exactly \(0.05\).
At \(p=0.60\). Here \(p^4=0.1296\) and \(p^5=0.07776\). Thus \(P(X\geq4)=5(0.1296)-4(0.07776)=0.64800-0.31104=0.33696\), and \(L(0.60)=1-0.33696=0.66304\), or \(0.6630\).
At \(p=0.80\). Here \(p^4=0.4096\) and \(p^5=0.32768\). Thus \(P(X\geq4)=5(0.4096)-4(0.32768)=2.04800-1.31072=0.73728\), and \(L(0.80)=1-0.73728=0.26272\), or \(0.2627\).
Interpretation and check. The OC values decrease as the defect probability rises: the test is less likely to miss a larger defect probability. At \(p=0.60\), the power is \(1-0.6630=0.3370\) to four decimal places. At \(p=0.80\), it is \(1-0.2627=0.7373\). Each OC value and its power complement sum to one before rounding. The calculation also shows why one should report the test’s actual null rejection probability rather than assume a discrete test has exactly the intended nominal level.
Interpreting the Curve as Separation
An OC curve is most informative when interpreted with the hypotheses in view. Under null values, low rejection probability is usually desirable, so the OC curve should be high. Under alternative values, high rejection probability is desirable, so the OC curve should be low. The transition between these regions indicates how quickly the test begins to reject as the parameter moves away from the null.
A steep transition can indicate strong discrimination over a relatively narrow parameter range. A gradual transition indicates that nonrejection remains fairly likely across a wider range of parameter values. The curve does not, by itself, say that every decision is correct: it describes repeated-sampling probabilities under each specified value of the parameter.
For a composite null, inspect the OC curve over the entire null parameter set, not only at one convenient point. A test may have a satisfactory rejection probability at one null value but a larger rejection probability elsewhere in the null. Likewise, for a composite alternative, power should be assessed across the alternative values that matter for the application. A curve makes these ranges visible in a way that a single power calculation cannot.
Worked Example: A Poisson OC Curve and Practical Interpretation
Suppose a hypothetical monitoring system records \(X\), the number of alerts in a fixed interval, with \(X\sim\operatorname{Poisson}(\lambda)\). Test \(H_0:\lambda=1\) against \(\lambda>1\), rejecting when \(X\geq3\). Calculate the OC values at \(\lambda=1,2,4\), find power at \(\lambda=2\) and \(\lambda=4\), and explain where the test distinguishes the null from larger rates more effectively. Give probabilities to four decimal places.
Method and nonrejection region. Nonrejection means \(X\leq2\). From the Poisson probability mass function, \(L(\lambda)=P_\lambda(X\leq2)=e^{-\lambda}(1+\lambda+\lambda^2/2)\).
At \(\lambda=1\). The sum in parentheses is \(1+1+1/2=2.5\), so \(L(1)=2.5e^{-1}=2.5(0.3678794)\approx0.9197\). The rejection probability at the null value is \(1-0.9197=0.0803\), to four decimal places.
At \(\lambda=2\). The sum is \(1+2+2^2/2=5\), so \(L(2)=5e^{-2}=5(0.1353353)\approx0.6767\). The power is \(1-0.6767=0.3233\), to four decimal places.
At \(\lambda=4\). The sum is \(1+4+4^2/2=13\), so \(L(4)=13e^{-4}=13(0.0183156)\approx0.2381\). The power is \(1-0.2381=0.7619\), to four decimal places.
Interpretation and check. The test has considerably greater power at \(\lambda=4\) than at \(\lambda=2\): the corresponding powers are \(0.7619\) and \(0.3233\). The curve indicates that this rejection rule distinguishes a rate of 4 from the null more effectively than it distinguishes a rate of 2 from the null. At each value, the OC probability plus power equals 1 before rounding. The null rejection probability here is \(0.0803\); the chosen rule should not be described as having a different size merely because a nominal level was intended.
Common Interpretive Traps
- Reversing OC and power. The OC value is the probability of nonrejection. Power is its complement. Always name the event before substituting a probability.
- Calling every OC value a Type II error. At alternative values, nonrejection is Type II error probability. At null values, it is the probability of correctly not rejecting under that particular parameter value.
- Assuming the curve must be decreasing. It decreases in common one-sided tests as the parameter moves into the alternative, but the direction follows from the model and rejection rule. Check those before interpreting the graph.
- Assuming a discrete test has an exact nominal size. A nonrandomized count-based rule may give a null rejection probability below the desired level. Calculate the probability from the actual rejection region.
- Judging a composite hypothesis from one point. Examine the curve across the null and alternative ranges relevant to the question, rather than relying on one isolated value.
Check Your Understanding
For each problem, state the model and rejection rule in your working, then use the complement relationship to interpret the result.
- Assume \(X\sim N(\theta,1)\) and reject \(H_0:\theta=0\) when \(X>1.5\). Find the null rejection probability and the OC value at \(\theta=1\). Give probabilities to four decimal places. (Answer: null rejection probability \(0.0668\); \(L(1)=\Phi(0.5)=0.6915\).)
- Let \(X\sim\operatorname{Binomial}(4,p)\), and reject when \(X=4\). Find the OC value at \(p=0.50\) and the null rejection probability when \(p_0=0.20\). (Answer: \(L(0.50)=0.9375\); null rejection probability \(0.0016\).)
- Let \(X\sim\operatorname{Poisson}(\lambda)\), with rejection when \(X\geq2\). Calculate the OC value and power at \(\lambda=1\), to four decimal places. (Answer: \(L(1)=2e^{-1}=0.7358\); power \(=0.2642\).)
- A test has OC value \(0.82\) at one alternative parameter value and \(0.27\) at a farther alternative value. Give the two power values and state which parameter value the test distinguishes from the null more effectively. (Answer: powers \(0.18\) and \(0.73\), respectively; the farther alternative is distinguished more effectively.)
- Assume \(X\sim N(\theta,1)\), and reject when \(X>1.645\). Find the smallest \(\theta\) for which the OC value is at most \(0.10\). Use \(\Phi^{-1}(0.10)\approx-1.2816\). (Answer: \(\theta\geq1.645+1.2816=2.9266\), so the boundary value is approximately \(2.9266\).)