Worked Problems on Order Statistic Densities

From Densities to Specific Questions

Order-statistic formulas become useful when they answer a concrete question: how likely is a particular sample quantile to lie in an interval, what is the chance that one order statistic exceeds a threshold given another, or what is the expected value of an order statistic under a specified parent law? The main work is matching the event to the correct marginal or joint density and integrating over the right region.

Let $X_1,\ldots,X_n$ be independent and identically distributed observations from a continuous distribution with cumulative distribution function $F$ and density $f$. Write $X_{(1)}\le \cdots \le X_{(n)}$ for the ordered observations. For the derivations and examples below, assume the parent density is continuous on the interior of its support. The formulas apply at interior points where the densities are defined.

Marginal density: For $1\le k\le n$, $$ f_{k:n}(x)=\frac{n!}{(k-1)!(n-k)!}F(x)^{k-1}\{1-F(x)\}^{n-k}f(x). $$ Joint density: For $1\le r<s\le n$ and $x<y$, $$ f_{r,s:n}(x,y)= \frac{n!}{(r-1)!(s-r-1)!(n-s)!} F(x)^{r-1}\{F(y)-F(x)\}^{s-r-1}\{1-F(y)\}^{n-s}f(x)f(y). $$ The joint density is zero outside the ordered region $x<y$.

Why These Density Formulas Have These Factors

The marginal formula counts how the observations can be placed below, near, and above $x$. In a short interval $(x,x+h)$, the event that $X_{(k)}$ lies there is, to first order in $h$, the event that exactly $k-1$ observations lie below $x$, one lies in the short interval, and $n-k$ lie above it.

There are $n!/((k-1)!(n-k)!)$ ways to assign observations to those three groups. The probability for any such assignment is, to first order, $F(x)^{k-1} f(x)h\{1-F(x)\}^{n-k}$. Configurations with at least two observations in the short interval have probability of order $h^2$, because the density is locally bounded. Dividing by $h$ and taking $h$ to zero gives the marginal density formula.

For two order statistics at distinct locations $x<y$, take short intervals around each. To first order, one observation must fall near $x$, one near $y$, $r-1$ must fall below $x$, $s-r-1$ must fall between $x$ and $y$, and $n-s$ must fall above $y$. The respective probabilities contribute $F(x)^{r-1}$, $\{F(y)-F(x)\}^{s-r-1}$, and $\{1-F(y)\}^{n-s}$. The two short-interval probabilities contribute $f(x)f(y)hq$. The number of assignments is the multinomial coefficient in the joint formula. Terms with two observations in either short interval vanish after division by $hq$ and taking both interval lengths to zero. This proves the joint density formula.

The joint density must integrate to one over $x<y$, and its marginal in either coordinate can be found by integrating out the other. These are useful checks, but the formulas are not substitutes for carefully specifying the region of integration.

Worked Interval and Conditional-Probability Calculations

Worked Example: Two Uniform Order Statistics in Separate Intervals

Let $X_1,\ldots,X_4$ be independent Uniform$(0,1)$ observations. Find $P(0.25<X_{(2)}<0.50,\ 0.50<X_{(3)}<0.75)$.

Use the joint density with $n=4$, $r=2$, and $s=3$. The coefficient is $4!/(1!0!1!)=24$, and the power of $F(y)-F(x)$ is zero. Since $F(t)=t$ and $f(t)=1$ on $(0,1)$,

f_{2,3:4}(x,y)=24x(1-y),\qquad 0<x<y<1.

Throughout the requested intervals, $x<0.50<y$, so the whole rectangular region is in the ordered support. Therefore,

\begin{aligned} P&=\int_{0.25}^{0.50}\int_{0.50}^{0.75}24x(1-y)\,dy\,dx\\ &=24\left(\int_{0.25}^{0.50}x\,dx\right) \left(\int_{0.50}^{0.75}(1-y)\,dy\right)\\ &=24\left(\frac{3}{32}\right)\left(\frac{3}{32}\right) =\frac{27}{128}=0.2109375. \end{aligned}

The product of the two one-dimensional integrals is valid here because the density factors into a function of $x$ and a function of $y$, and the requested region is a rectangle. It would not generally be valid for a region cut by the boundary $x<y$.

Worked Example: A Conditional Probability Given an Order Statistic

For the same sample of four Uniform$(0,1)$ observations, find the conditional probability $P(X_{(3)}>0.70\mid X_{(2)}=0.50)$. Conditioning on a continuous random variable taking one exact value is understood through its conditional density.

The joint density is $24x(1-y)$ for $x<y$. The marginal density of $X_{(2)}$, either from the marginal formula or by integrating the joint density over $y>x$, is

f_{2:4}(x)=12x(1-x)^2,\qquad 0<x<1.

For $0<x<y<1$, the conditional density of $X_{(3)}$ given $X_{(2)}=x$ is the joint density divided by this marginal density:

f_{X_{(3)}\mid X_{(2)}=x}(y) =\frac{24x(1-y)}{12x(1-x)^2} =\frac{2(1-y)}{(1-x)^2},\qquad x<y<1.

At $x=0.50$, the desired probability is the integral of this conditional density from $0.70$ to $1$:

\begin{aligned} P(X_{(3)}>0.70\mid X_{(2)}=0.50) &=\int_{0.70}^{1}\frac{2(1-y)}{(1-0.50)^2}\,dy\\ &=\frac{2}{0.25}\left[\;y-\frac{y^2}{2}\;\right]_{0.70}^{1} =8(0.045)=0.36. \end{aligned}

As a check, conditional on $X_{(2)}=0.50$, the two observations above it must both exceed $0.50$; for $X_{(3)}$ to exceed $0.70$, both must in fact lie in $(0.70,1)$. The corresponding probability is $(0.30/0.50)^2=0.36$, in agreement with the density calculation.

Expectations from a Marginal Density

Once the marginal density is known, an expectation is an integral against that density. In general, if $E[|X_{(k)}|]<\infty$, then $E[X_{(k)}]=\int x f_{k:n}(x)\,dx$, with integration over the parent support. The expectation exists as a finite number when the absolute first moment is finite.

Worked Example: The Second Order Statistic in an Exponential Sample

Let $X_1,X_2,X_3$ be independent exponential random variables with rate $\lambda>0$, so $F(x)=1-e^{-\lambda x}$ and $f(x)=\lambda e^{-\lambda x}$ for $x\ge0$. Find $E[X_{(2)}]$.

For $k=2$ and $n=3$, the marginal density formula gives

\begin{aligned} f_{2:3}(x) &=6F(x)\{1-F(x)\}f(x)\\ &=6(1-e^{-\lambda x})e^{-\lambda x}\lambda e^{-\lambda x}\\ &=6\lambda(e^{-2\lambda x}-e^{-3\lambda x}),\qquad x\ge0. \end{aligned}

Both terms have finite first moments. Using $\int_0^\infty x e^{-ax}\,dx=1/a^2$ for $a>0$, which follows by integration by parts, we obtain

\begin{aligned} E[X_{(2)}] &=\int_0^\infty x\,6\lambda(e^{-2\lambda x}-e^{-3\lambda x})\,dx\\ &=6\lambda\left(\frac{1}{(2\lambda)^2}-\frac{1}{(3\lambda)^2}\right)\\ &=\frac{5}{6\lambda}. \end{aligned}

The units are also consistent: an exponential rate has units of inverse time, so $1/\lambda$, and hence the expectation, has units of time. The result is positive, as it must be for a nonnegative random variable.

Checks That Prevent Miscalculations

In a worked problem, the most common source of error is not the factorial coefficient but the integration region. If $X_{(r)}=x$ and $X_{(s)}=y$ with $r<s$, then $x<y$. An interval specification that includes points with $y\le x$ must be split or restricted before integration. Also check that any conditional density integrates to one over its permitted support.

Do not confuse a joint density with a probability. A density value can exceed one; probabilities come from integrating over a region.
Do not omit the ordering constraint. The joint density is supported only on $x<y$ when $r<s$.
Do not condition by dividing probabilities informally. For conditioning on an exact value of a continuous variable, use a conditional density, obtained as a joint density divided by the relevant marginal density where that marginal is positive.
Check the parent support. For example, an exponential order statistic has no density for $x<0$, so its expectation integral begins at zero.

Key takeaway: Identify the order-statistic indices, write the relevant density on its support, and integrate over the event or expectation of interest. For conditional questions involving an exact order-statistic value, first form the conditional density by dividing the joint density by the conditioning variable’s marginal density.

Check Your Understanding

For each question, identify the density and integration region you would use before evaluating the integral.

For a sample of three Uniform$(0,1)$ observations, write the marginal density of $X_{(1)}$.
For a sample of four Uniform$(0,1)$ observations, what is the support of the joint density of $X_{(1)}$ and $X_{(3)}$?
In the first worked example, why can the probability integral be written as a product of two one-dimensional integrals?
For independent exponential observations with rate $\lambda$, which marginal density would you integrate to find $E[X_{(3)}]$ in a sample of size four?
What condition on $f_{2:4}(x)$ is needed to form the conditional density of $X_{(3)}$ given $X_{(2)}=x$ by taking a density ratio?