The Kth Order Statistic As A Counting Event

Why Counting Finds the Distribution

An order statistic is a ranked observation in a sample. For example, the second order statistic is the second-smallest observation. To find its probability of falling below a threshold, it is often more useful to count how many observations are below that threshold than to reason directly about which observation takes which rank.

This counting approach gives the cumulative distribution function (cdf) of any rank in an iid sample. It applies whether the parent distribution is continuous or discrete. The key is to express the event carefully: the kth-smallest observation is at most $x$ exactly when at least k observations are at most $x$.

Definition: Given observations $X_1,\ldots,X_n$, arrange their values in nondecreasing order. The kth value in that arrangement is the kth order statistic, denoted $X_{(k)}$, where $1\leq k\leq n$. Ties are retained in the ordered list.

The parent observations are assumed to be independent and identically distributed (iid), with common cdf $F(x)=P(X_i\leq x)$. “Identically distributed” means that each observation has the same cdf; “independent” means that knowing one observation does not change the probabilities for the others. These assumptions are what allow the count to have a binomial distribution.

Translate the Rank Event Into a Count

Fix a threshold $x$, and let $N_x$ be the number of sample observations that are at most $x$. Each observation contributes either 1 to this count, if it is at most $x$, or 0 otherwise. Under the iid assumptions, these contributions are independent Bernoulli variables with success probability $F(x)$. Therefore, $N_x$ has a binomial distribution with sample size $n$ and success probability $F(x)$.

N_x=\sum_{i=1}^{n}\mathbf{1}_{\{X_i\leq x\}} \sim \operatorname{Binomial}\bigl(n,F(x)\bigr).

The ordering definition gives the event equivalence directly. If at least $k$ sample values are at most $x$, then the kth-smallest value is at most $x$. If fewer than $k$ sample values are at most $x$, the kth-smallest value must exceed $x$. Thus the cdf of $X_{(k)}$ is a binomial upper tail.

Theorem: If $X_1,\ldots,X_n$ are iid with common cdf $F$, then for $1\leq k\leq n$, $$ F_{X_{(k)}}(x)=P\bigl(X_{(k)}\leq x\bigr) =P(N_x\geq k) =\sum_{j=k}^{n}\binom{n}{j}[F(x)]^j[1-F(x)]^{n-j}. $$ Equivalently, $F_{X_{(k)}}(x)=1-\sum_{j=0}^{k-1}\binom{n}{j}[F(x)]^j[1-F(x)]^{n-j}$.

Why the Binomial-Tail Formula Holds

The formula follows from two facts: the order-statistic event is a count event, and the count has a binomial distribution. The following steps make the connection explicit.

Mark observations below the threshold.
For each $i$, the indicator $\mathbf{1}_{\{X_i\leq x\}}$ is 1 with probability $F(x)$ and 0 with probability $1-F(x)$.

Count the marked observations.
The indicators are independent because the $X_i$ are independent. Their sum $N_x$ is therefore binomial with parameters $n$ and $F(x)$.

Impose the rank condition.
The kth value in sorted order is at most $x$ if and only if $N_x\geq k$. Summing the binomial probabilities for $j=k,\ldots,n$ gives the cdf.

For a fixed $j$, the binomial probability in the sum is the probability of exactly $j$ observations at most $x$: there are $\binom{n}{j}$ ways to choose those observations, each chosen observation contributes a factor $F(x)$, and each of the others contributes $1-F(x)$. The cdf includes every count from $k$ through $n$, not just one count.

The result needs no assumption that the parent distribution has a density. In a discrete model, observations can tie, but the event equivalence still holds because the count is defined using “at most $x$.” In particular, $F(x)=P(X_i\leq x)$, not $P(X_i<x)$. Replacing “at most” with “less than” at a point with positive probability can change the answer.

Worked Applications

Worked Example: A Rank From a Uniform Sample

Five independent measurements are each uniformly distributed on $[0,1]$. Find the probability that the third-smallest measurement is at most $0.4$. Give the answer to four decimal places.

Method. Count the observations at most $0.4$. For a Uniform$[0,1]$ observation, $F(0.4)=0.4$. Thus $N_{0.4}\sim\operatorname{Binomial}(5,0.4)$, and $X_{(3)}\leq0.4$ is the event $N_{0.4}\geq3$.

Compute each possible count:

\begin{aligned} P(N_{0.4}=3)&=\binom{5}{3}(0.4)^3(0.6)^2 =10(0.064)(0.36)=0.2304,\\ P(N_{0.4}=4)&=\binom{5}{4}(0.4)^4(0.6) =5(0.0256)(0.6)=0.0768,\\ P(N_{0.4}=5)&=\binom{5}{5}(0.4)^5 =0.01024. \end{aligned}

Add the probabilities for counts 3, 4, and 5: $0.2304+0.0768+0.01024=0.31744$. Rounded only at the final step, the probability is 0.3174 to four decimal places.

Check. The result is between 0 and 1, and the event requires three or more of the five measurements to be below the threshold. The calculation includes all three possible counts that satisfy that condition.

Mistake avoided. Using only the probability of exactly three observations, $0.2304$, would omit the cases with four or five observations at most $0.4$. Those cases also make the third-smallest value at most $0.4$.

Worked Example: A Discrete Lifetime With a Tie at the Threshold

Five iid components have hypothetical lifetimes $L$ with $P(L=8)=0.1$, $P(L=12)=0.1$, and $P(L=20)=0.8$, measured in months. Find the probability that the second-smallest lifetime is at most 12 months.

Method. First find the parent cdf at the threshold, including both values at or below 12: $F(12)=P(L=8)+P(L=12)=0.1+0.1=0.2$. Let $N_{12}$ count the component lifetimes at most 12 months. Then $N_{12}\sim\operatorname{Binomial}(5,0.2)$. The second-smallest lifetime is at most 12 precisely when $N_{12}\geq2$.

Compute the probabilities for counts 2 through 5:

\begin{aligned} P(N_{12}=2)&=\binom{5}{2}(0.2)^2(0.8)^3 =10(0.04)(0.512)=0.2048,\\ P(N_{12}=3)&=\binom{5}{3}(0.2)^3(0.8)^2 =10(0.008)(0.64)=0.0512,\\ P(N_{12}=4)&=\binom{5}{4}(0.2)^4(0.8) =5(0.0016)(0.8)=0.0064,\\ P(N_{12}=5)&=(0.2)^5=0.00032. \end{aligned}

Their sum is $0.2048+0.0512+0.0064+0.00032=0.26272$. Rounded to four decimal places, the requested probability is 0.2627.

Check. The count probability is based on $F(12)=0.2$, which includes the probability mass at 12 itself. Using $P(L<12)=0.1$ would describe a different event and produce a different answer.

Mistake avoided. A discrete distribution can place positive probability exactly at the threshold. Since the question says “at most 12,” components with lifetime 12 months count toward $N_{12}$.

Worked Example: The Probability the Rank Exceeds a Threshold

Six iid sensor errors are uniformly distributed on $[-1,1]$. Find the probability that the fourth-smallest error is greater than 0. Give the answer to four decimal places.

Method. For one error, $P(X_i\leq0)=F(0)=1/2$. Let $N_0$ count errors at most 0, so $N_0\sim\operatorname{Binomial}(6,1/2)$. The event $X_{(4)}>0$ occurs exactly when fewer than four errors are at most 0; that is, $N_0\leq3$.

The probabilities for counts 0 through 3 are:

\begin{aligned} P(N_0\leq3) &=\frac{\binom{6}{0}+\binom{6}{1}+\binom{6}{2}+\binom{6}{3}}{2^6}\\ &=\frac{1+6+15+20}{64} =\frac{42}{64} =0.65625. \end{aligned}

Rounded only at the final step, $P(X_{(4)}>0)=\mathbf{0.6563}$ to four decimal places. As a check, $P(X_{(4)}\leq0)$ is the complementary count $N_0\geq4$, with probability $(15+6+1)/64=22/64=0.34375$. The two probabilities sum to 1 before rounding.

Mistake avoided. For a rank that must be greater than $x$, the count at most $x$ must be less than $k$. Reusing the cdf’s condition $N_x\geq k$ would calculate the opposite event.

Common Traps and Scope

The most reliable way to avoid a rank-direction error is to translate the event into words before writing a probability. For $X_{(k)}\leq x$, at least $k$ observations must be at most $x$. For $X_{(k)}>x$, at most $k-1$ observations can be at most $x$. These are complementary events, so either the binomial upper tail or the corresponding lower tail can be used.

Counting exactly $k$ instead of at least $k$. The cdf includes counts $k,k+1,\ldots,n$.
Using the wrong threshold probability. Use $F(x)=P(X_i\leq x)$. At an atom, this may differ from $P(X_i<x)$.
Assuming independence alone is enough. The binomial formula also requires identical distributions, so every observation has the same success probability $F(x)$.
Confusing the parent cdf with the order-statistic cdf. $F(x)$ describes one observation; the binomial-tail expression describes $X_{(k)}$.

This counting result provides the cdf directly. It does not require finding a density, and it remains valid when the parent distribution is discrete. The density of an iid kth order statistic is a separate question. The counting method is aligned with the SOA Exam P (Probability) syllabus because it combines iid sampling, binomial probabilities, and careful event translation.

Key takeaway: For iid observations with parent cdf $F$, count how many are at most $x$. The kth order statistic is at most $x$ if and only if that count is at least $k$, so its cdf is the binomial upper tail with success probability $F(x)$.

Check Your Understanding

For each problem, identify the count event before calculating. Answers are rounded to four decimal places where needed.

Four iid observations have $F(5)=0.25$. Find the probability that their minimum is at most 5. (Answer: 0.6836)
Five iid observations have $F(10)=0.3$. Find the probability that the fourth order statistic is at most 10. (Answer: 0.0308)
Seven iid observations have $F(x)=0.4$. Find the probability that the third order statistic is at most $x$. (Answer: 0.5801)
Four iid observations have $F(2)=0.7$, including any probability mass at 2. Find the probability that the third order statistic is at most 2. (Answer: 0.6517)
Three iid observations have $F(6)=0.6$. Find the probability that the second order statistic is greater than 6. (Answer: 0.3520)

What You'll Learn