What a Runs Test Can Tell You
A runs test reduces an ordered sequence to a sequence of labels and counts how often the label changes. For example, observations classified as below or above a reference value become a string such as \(--++--++\). The count of runs in that string is a diagnostic of one kind of ordering: unusually long runs can be consistent with clustering, while unusually short runs can be consistent with alternation.
That diagnostic is useful, but narrow. A runs test does not establish that observations are independent, nor does it determine whether their values follow a particular distribution. Its conclusion concerns the evidence in the observed run count, under a specified null model and symbol-assignment rule. A non-significant result means the count was not sufficiently unusual by that test; it does not certify randomness.
The Null Hypothesis Is Narrow
For a binary sequence, let \(n_1\) and \(n_2\) be the fixed counts of the two labels, and let \(N=n_1+n_2\). Under the standard random-ordering null, every ordering with those counts is equally likely. This conditional formulation isolates the order: the test does not ask whether the two labels occur equally often, because their counts are held fixed.
One model that justifies this null is a sequence of independent, identically distributed Bernoulli observations with probability \(p\) of label 1, where \(0<p<1\). Conditional on observing \(n_1\) ones and \(n_2\) twos, each arrangement has probability \(p^{n_1}(1-p)^{n_2}\), the same for every arrangement. Thus, conditional on the counts, all \(\binom{N}{n_1}\) arrangements are equally likely. The test is not automatically valid for a process whose label probabilities change over time or whose sampling design does not make the orderings equally likely.
This conditional null explains both the value and the limitation of the test. It focuses on one summary of ordering, \(R\). Distinct sequences can have the same \(R\), and therefore the same runs-test result, even if their detailed patterns differ. The test does not examine every dependence relationship among observations.
Worked Example: A Periodic Sequence That Does Not Trigger Rejection
Worked Example: A Periodic Sequence That Does Not Trigger Rejection
A hypothetical sensor records eight readings, classified as below or above a fixed calibration threshold. The resulting labels are \(--++--++\). Test the random-ordering null at significance level \(\alpha=0.05\) using an exact two-sided runs test. Explain what the result does and does not establish.
Method and counts. There are four observations of each label, so \(n_1=n_2=4\), \(N=8\), and the number of possible orderings under the conditional null is \(\binom{8}{4}=70\). The observed sequence has four runs, so \(R=4\).
Center of the null distribution. Under random ordering, the expected number of runs is \(E[R]=1+\frac{2n_1n_2}{N}=1+\frac{2(4)(4)}{8}=5\). The observed count is one run below this center.
Exact two-sided probability. For four labels of each type, the numbers of arrangements with \(R=2,3,4,5,6,7,8\) are respectively \(2,6,18,18,18,6,2\). A count at least as far from 5 as the observed count satisfies \(|R-5|\geq1\), which includes \(R\leq4\) or \(R\geq6\). Therefore, \[ P\bigl(|R-5|\geq1\bigr)=\frac{2+6+18+18+6+2}{70}=\frac{52}{70}=0.7429. \]
Decision and interpretation. Since \(0.7429>0.05\), do not reject the random-ordering null. But the sequence repeats a four-symbol pattern, so it has a visible periodic structure. The large p-value does not prove that the sensor readings are independent; it says only that this runs count is not unusual under the conditional random-ordering model. The mistake to avoid is translating “do not reject” into “randomness has been established.”
Failure to Reject Is Not Proof of Independence
A hypothesis test does not generally prove its null hypothesis. The runs test is no exception. A non-significant result may occur because the data are compatible with the null, because the sample is too small to reveal a departure, or because the departure affects a feature that the statistic does not measure well. The test's p-value is calculated under the null model; it is not the probability that the null is true.
Power is the probability that a test rejects the null when a particular alternative is true. It depends on the alternative, sample size, significance level, and test statistic. A runs test can have useful power against some forms of clustering or alternation, but a single run count is not equally sensitive to all dependence patterns. A low-power test may fail to reject even when a structured process generated the sequence.
The form of the alternative also matters. A test designed to detect too few runs, too many runs, or either departure uses different rejection criteria. A two-sided result can conceal which direction the data point toward unless the observed count is also reported. In practice, state the observed count, the reference model, the chosen alternative, the p-value or rejection decision, and the limited scope of the conclusion.
Worked Example: Strong Alternation Can Still Miss a Cutoff
Worked Example: Strong Alternation Can Still Miss a Cutoff
A hypothetical quality-control sequence contains four readings on each side of a target. In time order, its labels are \(+-+-+-+\). Use an exact two-sided runs test at \(\alpha=0.05\). What is the conclusion, and why should it not be described as evidence that the order is random?
Method and observed statistic. Here \(n_1=n_2=4\), so there are \(\binom{8}{4}=70\) possible orderings under the conditional null. Every label differs from the next, giving the maximum possible count \(R=8\).
Exact tail probability. The expected count is \(E[R]=1+\frac{2(4)(4)}{8}=5\). For a two-sided test based on distance from 5, a result at least as extreme as \(R=8\) has \(R\geq8\) or \(R\leq2\). There are 2 arrangements with \(R=8\) and 2 with \(R=2\), so \[ p=\frac{2+2}{70}=\frac{4}{70}=0.0571. \]
Decision and interpretation. Because \(0.0571>0.05\), do not reject at the stated level. The observed sequence nevertheless alternates perfectly. The decision reflects the exact probability and the chosen cutoff, not a finding that alternation is absent. At a different significance level the formal decision could differ, while the observed pattern would remain the same. Avoid calling a p-value just above the cutoff proof of randomness.
Patterns and Information the Runs Count Can Miss
The run count compresses a sequence into one integer. That compression is useful for a focused test, but it discards information. In particular, it does not record the lengths of individual runs beyond their contribution to the total count, the magnitudes of the original observations, or every possible relationship between observations at different time lags. A sequence can therefore have an ordinary-looking run count while retaining structure that matters for the application.
- Periodic patterns: a repeated cycle may produce a run count that is not sufficiently extreme, especially in a short sequence.
- Lagged relationships: observations separated by several time steps may be associated even if consecutive labels yield an unremarkable total number of runs.
- Magnitude patterns: a test based on above/below labels discards how far observations lie from the threshold. Large and small deviations receive the same label.
- Changing processes: a shift in the distribution or label probability over time can violate the simple random-ordering model; a single total run count may not describe the change adequately.
A runs test is not a general-purpose test of all serial dependence. If the application requires assessing a particular kind of temporal structure, the diagnostic should match that structure. The runs result can be one piece of evidence, but it should not replace examination of the sequence and relevant process assumptions.
Worked Example: Random Ordering Is Not Distributional Fit
Worked Example: Random Ordering Is Not Distributional Fit
A hypothetical analyst observes the numerical sequence \(-100,-90,90,100,-80,-70,70,80\). The analyst uses the sample median to label values below it as “−” and values above it as “+”, then applies the exact two-sided runs test. Calculate the test result and explain whether it assesses a normal distribution.
Find the threshold and labels. In sorted order, the values are \(-100,-90,-80,-70,70,80,90,100\). The sample median is the average of the fourth and fifth values: \[ \frac{-70+70}{2}=0. \] In the original time order, the labels are \(--++--++\). There are four values below 0 and four above 0, and the labels form \(R=4\) runs.
Compute the exact result. With \(n_1=n_2=4\), there are 70 possible label orderings. As calculated from the exact run-count distribution, the expected count is 5 and the two-sided probability for a result at least as far from 5 as 4 is \[ p=P(R\leq4\text{ or }R\geq6)=\frac{52}{70}=0.7429. \] At \(\alpha=0.05\), do not reject the random-ordering null.
Interpret the scope. The values have a large gap around 0, but the runs test does not test whether the numerical values fit a normal distribution. After thresholding, the test uses only the labels and their order; it discards the distances between values. Thus the result concerns the ordering of these two categories under the specified null, not distributional fit. The common mistake here is to treat a non-significant runs test as a normality test. It is not one.
Keep Sequence Randomness and Distributional Fit Separate
A distribution describes the possible values of an observation and their probabilities. A randomness diagnostic for a sequence concerns how observations are ordered or related over time. These are different questions. A sequence can have values that do not fit a proposed distribution yet show no unusual run count after categorization. Conversely, values may fit a proposed marginal distribution while their order shows clustering or alternation.
A median-based runs test makes this distinction especially clear: it converts numerical observations to two labels. It can indicate an unusual arrangement of those labels under its null model, but it cannot evaluate the full shape of the numerical distribution. Distributional fit requires evidence directed at the proposed distribution and its relevant assumptions; a run count alone cannot supply that evidence.
Common Interpretation Traps
- “We did not reject, so observations are independent.” The test did not find a sufficiently unusual run count; it did not prove independence.
- “The p-value is the probability the sequence is random.” A p-value is a probability of a result at least as extreme as the observed one, calculated under the null model. It is not the probability that the null is true.
- “A runs test checks normality.” A median-based test uses only the order of two labels. It does not test the shape of the numerical distribution.
- “A non-significant result means there is no visible pattern.” A sequence can show a pattern that the chosen statistic, sample size, or rejection rule does not identify as statistically significant.
- “Any ordering is equally likely under any independent process.” Equal conditional probabilities require an appropriate model, such as independent, identically distributed binary labels. Changing probabilities or a different sampling design can invalidate that assumption.
When reporting a runs test, describe the sequence and symbol rule, state the null model and whether the test is one- or two-sided, give the observed run count and result, and limit the conclusion to the ordering feature tested. That is a more defensible interpretation than a blanket claim that a sequence is or is not random.
Check Your Understanding
For each problem, distinguish the run-count evidence from a broader claim about randomness or distribution.
- A sequence has three labels of each type and is \(--+++ -\), with the space removed. Find the number of runs. State whether that count alone proves dependence. (Answer: \(R=3\); no, the count alone does not prove dependence.)
- For four labels of each type, an alternating sequence has \(R=8\). Using the exact two-sided test centered at 5, calculate the p-value and give the decision at \(\alpha=0.05\). (Answer: \(p=4/70=0.0571\); do not reject at 0.05.)
- A median-based runs test on continuous observations gives \(p=0.38\). A colleague concludes that the observations are normally distributed. Is that conclusion justified? (Answer: No; the runs test concerns the ordering of the two labels, not normality.)
- Under the random-ordering null, a binary sequence has \(n_1=5\) and \(n_2=3\). Find the expected number of runs. (Answer: \(E[R]=1+\frac{2(5)(3)}{8}=4.75\).)
- A test gives \(p=0.12\) at \(\alpha=0.05\), but the plotted sequence appears to repeat a short cycle. Give the formal decision and one appropriate interpretation. (Answer: Do not reject; the test did not find a sufficiently unusual run count, but this does not establish independence or rule out the visible cycle.)