Notation and Rank Events for Order Statistics

Why Order Statistics Need Careful Notation

A sample can be summarized not only by its average or spread, but also by its position in the sorted list. The smallest value, the median position, and the largest value are all examples of order statistics. Their notation is compact, but threshold statements about them require care: “below $x$” and “at most $x$” are different conditions when observations can equal $x$.

The key idea is simple. The $k$th ordered value is at most a threshold exactly when at least $k$ observations are at most that threshold. This tutorial defines the notation, makes the tie convention explicit, and develops the corresponding event identities. These identities describe events; deriving the distribution of an order statistic is a separate step.

Ordered Values and Sample Ranks

Definition: Given real numbers $x_1,\ldots,x_n$, their ordered values are the values rearranged into nondecreasing order: $$ x_{(1)}\le x_{(2)}\le\cdots\le x_{(n)}. $$ The value $x_{(k)}$, for $1\le k\le n$, is the $k$th order statistic. In particular, $x_{(1)}$ is the sample minimum and $x_{(n)}$ is the sample maximum. Parentheses in $x_{(k)}$ indicate an order position; they do not denote an observation with original label $k$.

If all observations are distinct, each observation occupies exactly one position in the sorted list, so its rank is unambiguous. With ties, the ordered values are still well-defined, but there may be more than one way to assign the original observation labels to the tied positions. For example, if two observations both equal 7 and occupy positions 2 and 3, neither tied observation has a unique position unless an extra tie-breaking rule is imposed. The value at each position remains unambiguous: $x_{(2)}=x_{(3)}=7$.

Consequently, order statistics describe values at positions, not necessarily distinct observations. A tie does not remove a position: if three observations equal the same number, that number appears three times in the ordered list.

Worked Example: Sorting a Sample With Ties

Suppose the observed values, in their original order, are $12,7,12,4,9,7,15,10$. Sort them and identify the third and fifth order statistics.

Arrange all eight values from smallest to largest, retaining both copies of each repeated value: $$ x_{(1)}=4,\quad x_{(2)}=7,\quad x_{(3)}=7,\quad x_{(4)}=9,\quad x_{(5)}=10,\quad x_{(6)}=12,\quad x_{(7)}=12,\quad x_{(8)}=15. $$ Thus $x_{(3)}=7$ and $x_{(5)}=10$. The two observations equal to 7 occupy positions 2 and 3. The two observations equal to 12 occupy positions 6 and 7. Those values and positions are determined even though the original labels within each tied pair cannot be ordered uniquely.

For this sample, exactly three observations are at most 7, which is why the third ordered value is 7. Exactly four observations are at most 9, so the fifth ordered value is greater than 9. These count descriptions extend directly to events involving random samples.

Order Statistics of Random Variables

Let $X_1,\ldots,X_n$ be real-valued random variables defined on the same probability space. At each outcome, sort their realized values into nondecreasing order. The resulting random variables are denoted $X_{(1)},\ldots,X_{(n)}$, with $X_{(k)}$ the $k$th order statistic. This definition does not require independence or identical distributions. If values tie at an outcome, the ordered values still include every tied value, with its full multiplicity.

Definition: For a real threshold $x$, define the counts $$ N_{<x}=\sum_{i=1}^n \mathbf{1}\{X_i<x\},\qquad N_{\le x}=\sum_{i=1}^n \mathbf{1}\{X_i\le x\}, $$ where $\mathbf{1}\{A\}$ equals 1 when event $A$ occurs and 0 otherwise. Define $N_{>x}$ and $N_{\ge x}$ in the same way, replacing the corresponding indicator conditions by $X_i>x$ and $X_i\ge x$.

These counts are random variables: each is a finite sum of indicators of events. The order statistics are also random variables. Indeed, for any real $x$, $$ \{X_{(k)}\le x\} = \bigcup_{\substack{I\subseteq\{1,\ldots,n\}\\ |I|=k}} \ \bigcap_{i\in I}\{X_i\le x\}. $$ There are only finitely many subsets $I$, so the right-hand side is measurable whenever the $X_i$ are random variables. Since the sublevel events $\{X_{(k)}\le x\}$ are measurable for every real $x$, $X_{(k)}$ is measurable. This also confirms that the pointwise sorting definition produces a legitimate random variable.

Translating Threshold Events Into Counts

Theorem: For $1\le k\le n$ and any real $x$, the following event identities hold, including when some observations equal $x$: $$ \begin{aligned} \{X_{(k)}\le x\} &= \{N_{\le x}\ge k\},\\ \{X_{(k)}<x\} &= \{N_{<x}\ge k\},\\ \{X_{(k)}>x\} &= \{N_{\le x}\le k-1\},\\ \{X_{(k)}\ge x\} &= \{N_{<x}\le k-1\}. \end{aligned} $$ Equivalently, the first two statements can be written using counts above the threshold: $$ \{X_{(k)}\le x\}=\{N_{>x}\le n-k\},\qquad \{X_{(k)}<x\}=\{N_{\ge x}\le n-k\}. $$

Proof. Fix one outcome and sort its observed values as $X_{(1)}\le\cdots\le X_{(n)}$. If $X_{(k)}\le x$, then the first $k$ ordered values are all at most $x$. Therefore $N_{\le x}\ge k$. Conversely, if $N_{\le x}\ge k$, at least $k$ observed values are at most $x$, so the $k$th value in the nondecreasing list is at most $x$. This proves the first identity.

The same argument with strict inequalities gives the second: $X_{(k)}<x$ exactly when at least $k$ observations are strictly less than $x$. The third identity is the complement of the first, since $X_{(k)}>x$ is the complement of $X_{(k)}\le x$, and $N_{\le x}\ge k$ fails exactly when $N_{\le x}\le k-1$. The fourth follows in the same way by taking complements of the second identity.

Finally, $N_{>x}=n-N_{\le x}$, so $N_{\le x}\ge k$ is equivalent to $N_{>x}\le n-k$. Likewise, $N_{\ge x}=n-N_{<x}$, so $N_{<x}\ge k$ is equivalent to $N_{\ge x}\le n-k$. All arguments apply outcome by outcome, including in the presence of ties. $\square$

The complement versions are useful when it is easier to count observations on the other side of the threshold. For example, saying the $k$th ordered value is at most $x$ means that no more than $n-k$ observations can be strictly greater than $x$.

Worked Example: Translating Events for a Six-Observation Sample

Let $X_1,\ldots,X_6$ be any real-valued observations, and consider the fourth order statistic at a fixed threshold $c$. Rewrite the event $X_{(4)}>c$ as a count statement, and give its equivalent description using observations greater than $c$.

The strict event $X_{(4)}>c$ is the complement of $X_{(4)}\le c$. The theorem gives $$ \{X_{(4)}>c\}=\{N_{\le c}\le 3\}. $$ Thus at most three of the six observations may be at or below $c$. Since the sample has six observations, this is equivalent to at least three being strictly greater than $c$: $$ \{X_{(4)}>c\}=\{N_{>c}\ge 3\}. $$ For instance, if the observations are $1,3,4,6,8,10$ and $c=5$, then $N_{\le 5}=3$ and $N_{>5}=3$. The fourth ordered value is 6, so both count conditions correctly indicate $X_{(4)}>5$.

The equality boundary matters: observations equal to $c$ are included in $N_{\le c}$, but not in $N_{>c}$. That is why the two counts partition all observations without overlap.

Worked Example: A Discrete Model and a Tied Threshold

Let $X_1,\ldots,X_4$ be independent Bernoulli random variables with $P(X_i=1)=0.3$ and $P(X_i=0)=0.7$. Find $P(X_{(2)}\ge 1)$.

Apply the identity for $X_{(k)}\ge x$, with $k=2$ and $x=1$: $$ \{X_{(2)}\ge 1\}=\{N_{<1}\le 1\}. $$ Here each $X_i$ is either 0 or 1, so $N_{<1}$ counts zeros. The event occurs exactly when there is at most one zero, or equivalently at least two ones. Independence lets us add the probabilities of the disjoint cases of exactly two, three, or four ones: $$ \begin{aligned} P(X_{(2)}\ge 1) &=\binom{4}{2}(0.3)^2(0.7)^2 +\binom{4}{3}(0.3)^3(0.7) +\binom{4}{4}(0.3)^4\\ &=6(0.09)(0.49)+4(0.027)(0.7)+0.0081\\ &=0.2646+0.0756+0.0081\\ &=0.3483. \end{aligned} $$ The threshold is itself a possible observation, so it would be incorrect to replace $N_{<1}$ by $N_{\le 1}$. In particular, an observation equal to 1 satisfies $X_i\ge 1$, not $X_i<1$.

Common Mistake: Losing the Equality Case

A frequent error is to treat “at least $k$ observations below $x$” as interchangeable with “the $k$th order statistic is at most $x$.” The first phrase uses a strict inequality; the second permits equality. The correct pairing is $X_{(k)}\le x$ with a count of observations satisfying $X_i\le x$, and $X_{(k)}<x$ with a count satisfying $X_i<x$.

For a concrete check, take the sample $2,5,5,8$, let $k=2$, and set $x=5$. Then $x_{(2)}=5$, so $x_{(2)}\le5$ is true. There are two observations at most 5, but only one strictly below 5. Thus “at least two observations are strictly below 5” is false and cannot represent the event $x_{(2)}\le5$.

Key takeaway: Keep the inequality on the order statistic and the inequality in the count matched: $\le$ goes with $N_{\le x}$, while $<$ goes with $N_{<x}$. Ties are exactly where the distinction becomes visible.

Why Rank Events Matter

Order-statistic questions often become manageable once restated as questions about how many observations fall on each side of a threshold. The count identities do this without assuming a particular distribution or independence structure. If a later calculation does assume independent, identically distributed observations, that model can then be used to find the probability of the count event. The translation itself is purely a consequence of sorting.

The notation also helps separate two ideas that are easy to conflate: an order statistic is a value at a specified position, while a count records how many sample values meet a threshold condition. Exact threshold conventions preserve that link for continuous, discrete, and mixed distributions alike.

Check Your Understanding

Use the ordered-value definition and count identities to answer these questions.

For the sample $3,1,3,6,2$, write the complete ordered list and identify $x_{(2)}$ and $x_{(4)}$.
State the count event equivalent to $X_{(3)}\le a$. Why must the count use observations at most $a$, rather than strictly below $a$?
Rewrite $X_{(5)}>t$ as a condition on the number of observations strictly greater than $t$, for a sample of size 8.
If three observations equal $b$, how are those tied observations represented in the ordered list? Does the tied value determine a unique original observation at each tied position?
For $X_{(k)}\ge x$, which count is bounded above by $k-1$: $N_{<x}$ or $N_{\le x}$?