Why the Runs-Test Hypotheses Matter
A sequence can contain the same number of two outcomes as another sequence yet tell a different story about order. For example, repeated outcomes grouped together suggest clustering, while frequent switching suggests alternation. The runs test uses the number of runs to assess whether the ordering of two types of observations is consistent with randomness.
The hypotheses must reflect the question being asked. A test aimed at detecting clustering looks for unusually few runs; a test aimed at detecting alternation looks for unusually many. These are different directional alternatives, even though they use the same sequence and the same run count.
This formulation conditions on the observed label counts. It asks whether the order is unusual given how many of each label occurred; it does not test whether the two labels occur equally often. One standard justification is an independent sequence of binary observations with a common probability of either label: conditional on the total count of each label, every arrangement has the same probability. The runs test then compares the observed ordering with those possible arrangements.
Choose the Alternative Before Looking at the Tail
The alternative hypothesis describes the kind of departure from random ordering that matters. A low value of \(R\) indicates that like labels tend to occur together. A high value indicates that labels switch more often than expected under random ordering. The test direction should therefore follow the scientific question, not whichever direction makes the observed result look more striking.
| Research question | Alternative in words | Evidence is in |
|---|---|---|
| Do similar outcomes cluster? | Fewer runs than under random ordering | The lower tail of \(R\) |
| Do outcomes alternate unusually often? | More runs than under random ordering | The upper tail of \(R\) |
| Is the ordering nonrandom in either direction? | Too few or too many runs | Both tails, using a stated two-sided convention |
In formal notation, the null is random ordering conditional on the two observed counts. A clustering alternative is a lower-tail alternative, often written \(H_a:\) the ordering has fewer runs than expected under the null. An alternation alternative is an upper-tail alternative, \(H_a:\) the ordering has more runs than expected. A two-sided alternative says that the ordering is nonrandom, with either unusually few or unusually many runs. These descriptions concern the distribution of \(R\) under the null, not a claim that a fixed observed count is itself random.
Worked Examples: Formulating and Interpreting the Tails
Worked Example: Testing for Clustering in Inspection Outcomes
A hypothetical maintenance team records eight successive checks as \(A,A,B,B,B,A,A,B\). The analyst asks whether outcomes are clustered. Use the exact conditional runs test at \(\alpha=0.05\), and calculate its lower-tail p-value.
Method. State a lower-tail alternative because clustering means fewer runs. Then calculate \(R\) and the conditional probability of getting that many or fewer runs under random ordering.
Step 1: Count each label and the runs. There are four \(A\)'s and four \(B\)'s. The sequence consists of \(AA\), \(BBB\), \(AA\), and \(B\), so \(R=4\).
Step 2: State the hypotheses. \(H_0\) says that, conditional on four \(A\)'s and four \(B\)'s, the order is random. The clustering alternative is that \(R\) is unusually small.
Step 3: Calculate the lower-tail probability. There are \(\binom{8}{4}=70\) arrangements. Exactly 2 have 2 runs, and 18 have 4 runs, so
Step 4: Conclude. Since \(0.286>0.05\), do not reject \(H_0\). This sequence does not provide significant evidence of clustering at the 5% level.
Common mistake avoided. A lower-tail p-value is not the probability of observing exactly four runs. It includes outcomes at least as supportive of clustering: here, both \(R=2\) and \(R=4\). The value is \(2/7\), not \(0.031\).
Worked Example: Testing for Alternation in Sensor Status
A hypothetical sensor records seven successive statuses: pass, fail, pass, fail, pass, fail, pass. The concern is unusually frequent switching. Use an exact conditional upper-tail test at \(\alpha=0.05\).
Method. Frequent switching corresponds to an unusually large \(R\), so use an upper-tail alternative.
Step 1: Count labels and runs. There are four passes and three fails. Each observation differs from the one before it, so there are seven runs: \(R=7\).
Step 2: State the hypotheses. \(H_0\) says that the order is random conditional on four passes and three fails. \(H_a\) says there are too many runs, indicating alternation.
Step 3: Calculate the upper-tail probability. There are \(\binom{7}{4}=35\) possible arrangements. To have seven runs, the labels must alternate. With four passes and three fails, only the arrangement beginning and ending with pass does so. Thus
Step 4: Conclude. Since \(0.0286<0.05\), reject \(H_0\). The sequence gives evidence of more alternation than random ordering would suggest.
Common mistake avoided. The test is upper-tailed because the research question concerns switching. The exact upper-tail probability is \(1/35\), not \(0.018\); the denominator must include all 35 arrangements with the observed label counts.
Worked Example: Two-Sided Evidence in Benchmark Measurements
A hypothetical monitor records nine measurements relative to a benchmark of 10 units: \(12,7,11,13,6,5,14,8,15\). Label a measurement \(+\) if it is above 10 and \(-\) if it is below 10. Test for nonrandom ordering at \(\alpha=0.10\), using the doubled-smaller-tail convention for a two-sided exact p-value.
Method. A two-sided runs test looks for either unusually few or unusually many runs. The stated convention doubles the smaller of the two inclusive one-sided tail probabilities, with the result capped at 1.
Step 1: Form the sequence. The labels are \(+,-,+,+,-,-,+,-,+\). There are five plus labels and four minus labels.
Step 2: Count the runs. Reading left to right gives \(+\), \(-\), \(++\), \(--\), \(+\), \(-\), \(+\). Therefore \(R=7\).
Step 3: Calculate the relevant upper tail. There are \(\binom{9}{5}=126\) arrangements. For these counts, 18 arrangements have 7 runs, 8 have 8 runs, and 1 has 9 runs. Therefore
The lower-tail probability is larger than the upper-tail probability, so the doubled-smaller-tail p-value is
Step 4: Conclude. Since \(0.4286>0.10\), do not reject the random-ordering null. The observed \(R=7\) does not provide significant two-sided evidence of nonrandom ordering.
Common mistake avoided. A two-sided p-value requires a stated convention and a calculation from the conditional runs distribution. Here the upper-tail probability is about \(0.214\), not \(0.12\); under the convention specified in the question, the two-sided value is about \(0.429\).
Why the Conditional Null Is Useful
Conditioning on the observed counts isolates order from overall frequency. Suppose a sequence contains many more \(A\)'s than \(B\)'s. A test of equal label probabilities would address whether the counts are balanced; the runs test instead asks whether the observed \(A\)'s and \(B\)'s are arranged randomly given those counts. This distinction is useful in applications where the order itself matters, such as monitoring successive classifications or examining whether outcomes occur in streaks.
The conditional null also explains why a runs-test alternative must be directional or explicitly two-sided. “Nonrandom” alone does not say whether to look for clustering or alternation. If the research concern is clustering, a large run count points in the opposite direction from the stated alternative. If the concern is alternation, a small run count does so. A test direction selected after observing the sequence does not have the same interpretation as a direction specified from the question in advance.
Identify the two categories and preserve their observed sequence. Use an explicit rule for any observations that do not fit either label.
Specify random ordering given the observed numbers of each label. Do not replace this with a claim that the label counts must be equal.
Use a lower-tail alternative for clustering, an upper-tail alternative for alternation, or a stated two-sided procedure for either kind of departure.
Compare the appropriate tail probability with the significance level, then state what the data do or do not support about ordering.
Common Hypothesis-Formulation Traps
- Using the wrong tail. Few runs support clustering, while many runs support alternation. Reversing these directions reverses the evidence.
- Testing equal counts instead of random order. The conditional null fixes the observed counts; the question is whether their arrangement is random.
- Calling every departure “clustering.” An unusually large run count suggests alternation, not clustering. A two-sided test must allow both directions.
- Leaving the two-sided convention unstated. With a discrete runs distribution, two-sided p-value conventions can differ. State the convention used and calculate the corresponding probability.
- Interpreting failure to reject as proof of randomness. A nonsignificant result means the data do not provide sufficient evidence against the specified null at the chosen level; it does not establish that the ordering is random.
Check Your Understanding
For each problem, identify the relevant tail or tails and interpret the result using the information given.
- A hypothetical sequence has labels \(C,C,D,D,D,C,C,D\). The question is whether the labels cluster. It has four of each label, and the exact lower-tail p-value for the observed run count is \(2/7\). State the hypotheses in words and the decision at \(\alpha=0.05\). (Answer: \(H_0\): random ordering given four of each label; \(H_a\): too few runs (clustering); do not reject because \(2/7\approx0.286>0.05\).)
- A sequence has three \(U\)'s and three \(V\)'s and alternates perfectly. There are \(\binom{6}{3}=20\) conditional arrangements. Only two arrangements alternate perfectly. What is the upper-tail p-value, and what is the decision at \(\alpha=0.05\)? (Answer: \(2/20=0.1000\); do not reject at \(0.05\).)
- A two-sided test uses the doubled-smaller-tail convention. For an observed sequence, the smaller inclusive tail probability is \(0.08\). What two-sided p-value is reported, and is the null rejected at \(\alpha=0.10\)? (Answer: \(2(0.08)=0.16\); do not reject.)
- A process analyst wants to detect unusually long streaks of the same status. Should the alternative be lower-tailed or upper-tailed in \(R\), and what pattern would support it? (Answer: Lower-tailed; unusually few runs support clustering or long streaks.)
- A researcher tests for unusually frequent switching and obtains an observed \(R\) in the lower tail, with an upper-tail alternative specified in advance. Explain the direction of the result. (Answer: The observed result is opposite the specified alternation alternative; it does not provide evidence of unusually frequent switching.)