Transforming Order Statistics Through The Parent Cdf

Why Transform an Order-Statistic Problem?

Order statistics rank observations from smallest to largest. Their probabilities can look tied to the particular parent distribution: the relevant thresholds may be expressed in dollars, time, or another scale. If the parent cumulative distribution function is continuous, applying it to each observation converts the problem to the unit interval. Probabilities can then be calculated from uniform variables and counts, without needing to work with the original measurement scale.

The transformation is useful only when its conditions are respected. In particular, the observations must be independent and identically distributed, and the parent cdf must be continuous. The transformed observations are uniform and independent, but they are not a new sample of observations from the original parent distribution.

Definition: Let $X_1,\ldots,X_n$ be independent and identically distributed (iid) observations with continuous cdf $F_X$. Define $U_i=F_X(X_i)$. Then $U_1,\ldots,U_n$ are iid Uniform$(0,1)$. Write $X_{(k)}$ for the $k$th smallest of the $X_i$'s and $U_{(k)}$ for the $k$th smallest of the $U_i$'s.

Why the Transformed Observations Are Uniform

For a Uniform$(0,1)$ variable $U$, the cdf satisfies $P(U\leq u)=u$ for $0\leq u\leq1$. The key fact is that the same probability holds for $F_X(X)$, even if $F_X$ is not strictly increasing.

Choose a probability level.
Fix $u$ with $0<u<1$, and let $q_u=\inf\{x:F_X(x)\geq u\}$, the lower $u$-quantile of the parent distribution.

Use continuity.
Because $F_X$ is continuous, $F_X(q_u)=u$. Thus $P(X\leq q_u)=u$.

Translate the event.
Values $X\leq q_u$ have $F_X(X)\leq u$. Values beyond $q_u$ for which $F_X(X)=u$ can only lie on a flat part of the cdf, which has probability zero. Therefore $P(F_X(X)\leq u)=u$.

This proves that each $U_i=F_X(X_i)$ has the Uniform$(0,1)$ distribution. Since each transformed value is a function of its corresponding $X_i$, independence of the original observations implies independence of the transformed observations. The $U_i$'s are therefore iid uniform.

Since a cdf is nondecreasing, transforming observations preserves their order except possibly when distinct observations map to the same cdf value. Under a continuous parent distribution this exceptional event has probability zero. Hence the order-statistic problem can be transferred to the uniform scale. A particularly reliable route is to express a rank event as a count below a threshold.

Theorem: If $X_1,\ldots,X_n$ are iid with continuous cdf $F_X$, then, for any real $x$ and rank $k$, $$ P(X_{(k)}\leq x) =P\bigl(\operatorname{Bin}(n,F_X(x))\geq k\bigr) =P(U_{(k)}\leq F_X(x)). $$ Here $\operatorname{Bin}(n,p)$ denotes a binomial random variable with $n$ trials and success probability $p$.

The first equality follows because $X_{(k)}\leq x$ exactly when at least $k$ of the $n$ observations are at most $x$. Each observation has probability $F_X(x)$ of being at most $x$, independently of the others. The second equality is the same counting event after the cdf transformation. This count argument also avoids needing to calculate a density for the order statistic.

Thresholds and Intervals on the Uniform Scale

For a single threshold, the theorem reduces the problem to a binomial tail. For $a<b$, the event that the $k$th order statistic falls in $(a,b]$ is the difference between two nested threshold events: $$ P(a<X_{(k)}\leq b) =P(X_{(k)}\leq b)-P(X_{(k)}\leq a). $$ Thus, if $p_a=F_X(a)$ and $p_b=F_X(b)$, calculate the two binomial tails with success probabilities $p_b$ and $p_a$, then subtract. This is not generally the probability that one observation falls between $a$ and $b$: the rank requirement involves the whole sample.

Worked Example: An Exponential Parent and an Order-Statistic Interval

Suppose $X_1,\ldots,X_4$ are iid exponential random variables with rate $\lambda=2$ per unit of time, so $F_X(x)=1-e^{-2x}$ for $x\geq0$. Let $a=\ln(4/3)/2$ and $b=\ln(4)/2$. Find $P(a<X_{(2)}\leq b)$.

Method. Transform the thresholds through the parent cdf and use the binomial-count formula for the second-smallest observation. The stated exponential model gives $F_X(a)=1-e^{-\ln(4/3)}=1-3/4=1/4$ and $F_X(b)=1-e^{-\ln4}=1-1/4=3/4$.

For $X_{(2)}\leq b$, at least two of the four observations must be at most $b$. The count has a Binomial$(4,3/4)$ distribution, so $$ P(X_{(2)}\leq b) =1-P(\text{count}=0)-P(\text{count}=1) =1-(1/4)^4-4(3/4)(1/4)^3 =1-\frac{1}{256}-\frac{12}{256} =\frac{243}{256} =0.94921875. $$

Likewise, the count below $a$ is Binomial$(4,1/4)$, giving $$ P(X_{(2)}\leq a) =1-(3/4)^4-4(1/4)(3/4)^3 =1-\frac{81}{256}-\frac{108}{256} =\frac{67}{256} =0.26171875. $$

Subtract the nested events: $$ P(a<X_{(2)}\leq b) =\frac{243-67}{256} =\frac{176}{256} =0.6875. $$ The answer is $0.6875$. As a check, the probability is nonnegative and does not exceed $P(X_{(2)}\leq b)=0.94921875$. The method avoids confusing the interval probability for a ranked observation with the single-observation probability $F_X(b)-F_X(a)=1/2$.

Several Threshold Regions

Some questions impose more than one rank condition at once. In that case, divide the real line at the given thresholds. The probabilities of the resulting regions are differences of cdf values, and the numbers of observations in the regions have a multinomial distribution. This is the same iid counting principle used for one threshold, now applied to several disjoint categories.

Worked Example: Two Rank Conditions at Different Thresholds

Let $X_1,\ldots,X_5$ be iid from a continuous parent distribution. Two thresholds $a<b$ satisfy $F_X(a)=0.2$ and $F_X(b)=0.6$. Find $P(X_{(2)}\leq a,\ X_{(4)}>b)$.

Method. Classify each observation into one of three regions: at most $a$, between $a$ and $b$, or greater than $b$. Their probabilities are $0.2$, $0.6-0.2=0.4$, and $1-0.6=0.4$, respectively.

The condition $X_{(2)}\leq a$ requires at least two observations in the first region. The condition $X_{(4)}>b$ requires at most three observations in the first two regions combined, or equivalently at least two observations greater than $b$. If $i$, $j$, and $r$ are the counts in the three regions, the valid possibilities are $(i,j,r)=(2,0,3),(2,1,2),(3,0,2)$. They sum to five observations and meet both rank conditions.

Using the multinomial probability for each count triple, $$ \begin{aligned} P(X_{(2)}\leq a,\ X_{(4)}>b) &=\frac{5!}{2!0!3!}(0.2)^2(0.4)^0(0.4)^3\\ &\quad+\frac{5!}{2!1!2!}(0.2)^2(0.4)^1(0.4)^2\\ &\quad+\frac{5!}{3!0!2!}(0.2)^3(0.4)^0(0.4)^2\\ &=10(0.04)(0.064)+30(0.04)(0.4)(0.16)+10(0.008)(0.16)\\ &=0.0256+0.0768+0.0128\\ &=0.1152. \end{aligned} $$

The answer is $0.1152$. The count triples are mutually exclusive, and each satisfies the original rank conditions, so adding their probabilities is valid. This approach avoids treating the two rank events as independent; they are defined using the same observations and are not independent in general.

Common Mistakes and Conditions

A tempting shortcut for an interval problem is to use only $F_X(b)-F_X(a)$. That is the probability for one observation to fall in the interval, not the probability for a specified order statistic. For example, in the exponential example above, that shortcut gives $0.75-0.25=0.50$, whereas the correct probability for the second-smallest of four observations is $0.6875$. The rank condition changes the event and must be included in the count.

Forgetting the direction of the count. $X_{(k)}\leq x$ means at least $k$ observations are at most $x$, not exactly $k$.
Subtracting in the wrong order. For $a<X_{(k)}\leq b$, subtract the threshold probability at $a$ from the one at $b$.
Assuming every parent cdf is eligible. A discontinuous parent distribution can produce transformed values that are not uniform. Do not apply the ordinary probability integral transform without its continuity condition.
Assuming transformed values are independent without checking the sample. Applying a function separately preserves independence, but it does not make dependent observations independent.

The cdf transformation is especially valuable when a problem supplies $F_X$ at specified thresholds but gives no convenient formula for the order-statistic density. The threshold and region-count methods use only those cdf probabilities. They are aligned with the SOA Exam P (Probability) syllabus and provide a direct way to turn ranking conditions into binomial or multinomial calculations.

Key takeaway: For iid observations from a continuous parent cdf, $F_X(X_i)$ are iid Uniform$(0,1)$. Translate rank events into counts below the relevant thresholds, use the corresponding uniform-scale probabilities, and retain the rank condition throughout the calculation.

Check Your Understanding

For each problem, use the parent cdf values to set up the relevant count event. Give probabilities to four decimal places.

Four iid observations have a continuous parent cdf with $F_X(c)=0.4$. Find $P(X_{(2)}\leq c)$. (Answer: 0.5248)
Five iid observations have a continuous parent cdf with $F_X(a)=0.2$ and $F_X(b)=0.6$, where $a<b$. Find $P(a<X_{(3)}\leq b)$. (Answer: 0.6246)
Six iid observations have a continuous parent cdf with $F_X(t)=0.5$. Find $P(X_{(4)}\leq t)$. (Answer: 0.3438)
Five iid observations have a continuous parent cdf with $F_X(a)=0.2$ and $F_X(b)=0.6$, where $a<b$. Find $P(X_{(2)}\leq a,\ X_{(4)}>b)$. (Answer: 0.1152)
Seven iid observations have a continuous parent cdf with $F_X(c)=0.3$. Find $P(X_{(4)}\leq c)$. (Answer: 0.1260)

What You'll Learn