Order Statistics From Discrete Populations and Ties

Why Discrete Order Statistics Need Special Care

For a random sample, the ordered observations are written \(X_{(1)}\leq X_{(2)}\leq\cdots\leq X_{(n)}\). The smallest and largest observations are \(X_{(1)}\) and \(X_{(n)}\), respectively. In a discrete population, different observations can have the same value with positive probability. A calculation for an order statistic must therefore allow for ties rather than treating each sample value as distinct.

The key is to count observations on either side of a candidate value. For example, the event that the \(k\)th ordered observation is at most \(x\) occurs exactly when at least \(k\) sample observations are at most \(x\). Because the sample is independent and identically distributed, this count has a binomial distribution. Point probabilities follow by comparing counts at and below the candidate value.

Definition: Let \(X_1,\ldots,X_n\) be an independent and identically distributed sample from a discrete population, and write its ordered values as \(X_{(1)}\leq\cdots\leq X_{(n)}\). For a population value \(x\), define \(N_{<x}\) as the number of observations strictly less than \(x\), and \(N_{\leq x}\) as the number at most \(x\). Then \(X_{(k)}=x\) exactly when \(N_{<x}\leq k-1\) and \(N_{\leq x}\geq k\).

Point Probabilities from Cumulative Counts

Let \(F(x)=P(X_i\leq x)\) be the population cumulative distribution function. The sample count \(N_{\leq x}\) has a binomial distribution with sample size \(n\) and success probability \(F(x)\). Thus the cumulative probability for the \(k\)th order statistic is

P(X_{(k)}\leq x) =P(N_{\leq x}\geq k) =\sum_{j=k}^{n}\binom{n}{j}F(x)^j[1-F(x)]^{n-j}.

If \(x\) is a possible population value, observations strictly below \(x\) occur with probability \(F(x-)=P(X_i<x)\). Subtracting the probability that at least \(k\) observations are strictly below \(x\) from the probability that at least \(k\) are at most \(x\) gives the point probability:

P(X_{(k)}=x) =\sum_{j=k}^{n}\binom{n}{j}F(x)^j[1-F(x)]^{n-j} -\sum_{j=k}^{n}\binom{n}{j}F(x-)^j[1-F(x-)]^{n-j}.

The subtraction removes the event \(X_{(k)}<x\), leaving exactly \(X_{(k)}=x\). Equivalently, this probability is the chance that fewer than \(k\) observations are below \(x\), but at least \(k\) are at most \(x\). This count interpretation is usually the most direct way to set up a calculation.

Worked Example: Finding a Point Probability with Ties

A sample of three independent observations comes from a population with \(P(X=0)=0.2\), \(P(X=1)=0.5\), and \(P(X=3)=0.3\). Find \(P(X_{(2)}=1)\). State the distribution used for each count, and give the probability to four decimals.

Method. The second ordered observation equals 1 if at least two observations are at most 1, but it is not the case that at least two are strictly less than 1. The counts use binomial distributions because each observation independently either falls in the specified set or does not.

The probability an observation is at most 1 is \(0.2+0.5=0.7\). Therefore,

P(N_{\leq 1}\geq 2) =\binom{3}{2}(0.7)^2(0.3)+(0.7)^3 =3(0.49)(0.3)+0.343 =0.441+0.343 =0.784.

The probability an observation is strictly below 1 is \(0.2\). Hence,

P(N_{<1}\geq 2) =\binom{3}{2}(0.2)^2(0.8)+(0.2)^3 =3(0.04)(0.8)+0.008 =0.096+0.008 =0.104.

Subtract the second probability from the first:

P(X_{(2)}=1)=0.784-0.104=0.6800.

Check. The probability lies between 0 and 1. The subtraction removes samples with at least two zeros; in every remaining sample with at least two observations in \(\{0,1\}\), the second ordered value is 1. The common mistake is to use only \(P(N_{\leq1}\geq2)\), which includes samples whose second value is 0.

Joint Events and Multinomial Counts

For joint order-statistic events, divide the population values into categories and count how many observations fall in each category. If the categories have probabilities \(p_1,\ldots,p_m\), then the probability of category counts \(c_1,\ldots,c_m\), where the counts sum to \(n\), is

\frac{n!}{c_1!\cdots c_m!}\prod_{j=1}^{m}p_j^{c_j}.

The coefficient counts the distinct sample arrangements with those category counts. To use this formula for order statistics, translate each requested ordered value into inequalities on the counts. Ties are not a special correction to the multinomial formula: they are included by allowing multiple observations in the same category.

Worked Example: A Joint Event with a Tied Category

Four independent observations come from a discrete population with \(P(X=A)=0.5\), \(P(X=B)=0.3\), and \(P(X=C)=0.2\), where \(A<B<C\). Find \(P(X_{(2)}=B,\ X_{(4)}=C)\). Give the answer to four decimals.

Method. Let \(a,b,c\) be the counts of \(A,B,C\). For the second ordered value to be \(B\), there can be at most one \(A\), and at least two observations must be \(A\) or \(B\). For the largest value to be \(C\), at least one observation must be \(C\). Thus \(a\leq1\), \(a+b\geq2\), and \(c\geq1\), with \(a+b+c=4\).

The count combinations satisfying all three conditions are \((0,2,2)\), \((0,3,1)\), \((1,1,2)\), and \((1,2,1)\). Their probabilities are:

\begin{aligned} P(0,2,2)&=\frac{4!}{0!2!2!}(0.5)^0(0.3)^2(0.2)^2 =6(0.09)(0.04)=0.0216,\\ P(0,3,1)&=\frac{4!}{0!3!1!}(0.5)^0(0.3)^3(0.2) =4(0.027)(0.2)=0.0216,\\ P(1,1,2)&=\frac{4!}{1!1!2!}(0.5)(0.3)(0.2)^2 =12(0.5)(0.3)(0.04)=0.0720,\\ P(1,2,1)&=\frac{4!}{1!2!1!}(0.5)(0.3)^2(0.2) =12(0.5)(0.09)(0.2)=0.1080. \end{aligned}

These count events are mutually exclusive, so add their probabilities:

P(X_{(2)}=B,\ X_{(4)}=C) =0.0216+0.0216+0.0720+0.1080 =0.2232.

Check. The event requires both a \(B\) in the second position and at least one \(C\) at the maximum; every listed count combination satisfies both, including cases with tied \(B\) values. The common mistake is to require exactly one observation of each value, even though ties are allowed.

When Two Order Statistics Are Equal

For ranks \(r\leq s\), the event \(X_{(r)}=X_{(s)}=x\) occurs precisely when fewer than \(r\) observations are below \(x\), and at least \(s\) observations are at most \(x\). In symbols, the count conditions are \(N_{<x}\leq r-1\) and \(N_{\leq x}\geq s\). This differs from requiring just one observation to equal \(x\): multiple observations can occupy the ranks from \(r\) through \(s\).

The next example sums the probabilities of this event over every possible population value. It illustrates that positive probability of ties is a normal feature of discrete order statistics.

Worked Example: Probability of a Tie Between Adjacent Ranks

Four independent observations come from a population with \(P(X=0)=0.25\), \(P(X=1)=0.50\), and \(P(X=2)=0.25\). Find \(P(X_{(2)}=X_{(3)})\). Give the answer to four decimals.

Method. Calculate the probability that both middle ordered observations equal each possible value, then add those disjoint probabilities.

For \(x=0\), at least three observations must equal 0:

P(X_{(2)}=X_{(3)}=0) =\binom{4}{3}(0.25)^3(0.75)+(0.25)^4 =0.046875+0.00390625 =0.05078125.

For \(x=1\), let \(a,b,c\) count the observations equal to \(0,1,2\), respectively. The conditions are \(a\leq1\), \(a+b\geq3\), and \(a+b+c=4\). The possible count triples are \((0,3,1)\), \((0,4,0)\), \((1,2,1)\), and \((1,3,0)\). Their probabilities are

\begin{aligned} P(0,3,1)&=4(0.50)^3(0.25)=0.125,\\ P(0,4,0)&=(0.50)^4=0.0625,\\ P(1,2,1)&=12(0.25)(0.50)^2(0.25)=0.1875,\\ P(1,3,0)&=4(0.25)(0.50)^3=0.125. \end{aligned}

Their sum is \(0.125+0.0625+0.1875+0.125=0.5000\). For \(x=2\), at least three observations must equal 2, giving the same probability as for \(x=0\), namely \(0.05078125\). Therefore,

P(X_{(2)}=X_{(3)}) =0.05078125+0.5000+0.05078125 =0.6015625 \approx 0.6016.

Check. The three cases correspond to different common values, so they cannot occur together. The total is between 0 and 1. A common mistake is to count only samples in which all four observations are equal; ties between the middle ranks also occur when just enough observations share a value to occupy both positions.

Why the Continuous Density Formula Does Not Transfer

For an independent sample from a continuous population with cumulative distribution function \(F\) and density \(f\), the \(k\)th order statistic has density

f_{X_{(k)}}(x) =\frac{n!}{(k-1)!(n-k)!} [F(x)]^{k-1}[1-F(x)]^{n-k}f(x).

This formula depends on a density and on the continuous setting, where the probability of an observation taking any one exact value is zero. In particular, continuous independent observations have probability zero of tying. A discrete population instead assigns positive probability to some exact values. Its distribution has jumps, and an ordinary density with respect to length does not describe its point probabilities. Applying the continuous formula at a discrete value therefore does not calculate \(P(X_{(k)}=x)\).

The right-hand alternative is not an adjusted continuous density formula. Use the binomial count formula for one order statistic, or multinomial counts for joint events, and calculate probabilities at the population’s actual support values. The continuous formula remains useful when its continuous-density assumptions hold; it is not a general formula for all populations.

Exam Checks

Identify the support and probabilities.
List the distinct population values in order, with their probabilities. State that the observations are independent and identically distributed when using binomial or multinomial counts.

Translate the ranks into counts.
For \(X_{(k)}=x\), require fewer than \(k\) observations strictly below \(x\) and at least \(k\) at most \(x\).

Calculate and check.
Use binomial probabilities for a single threshold or enumerate valid multinomial counts for a joint event. Check that the event includes the possible ties and that disjoint cases are added only once.

A useful final check is to distinguish “at most \(x\)” from “strictly less than \(x\).” Those probabilities differ whenever \(P(X=x)>0\), exactly the situation in which ties matter. This count-based reasoning is aligned with the SOA VEE Mathematical Statistics syllabus.

Check Your Understanding

For each problem, identify the relevant counts before calculating. Probabilities are given to four decimals where appropriate.

A sample of three independent observations comes from a population with \(P(X=2)=0.4\) and \(P(X=5)=0.6\). Find \(P(X_{(2)}=2)\). (Answer: 0.3520)
A sample of four comes from a population with \(P(X=A)=0.2\), \(P(X=B)=0.5\), and \(P(X=C)=0.3\), where \(A<B<C\). Find \(P(X_{(3)}=B)\). (Answer: 0.6245)
Three independent observations have probabilities \(0.2,0.5,0.3\) at values \(0,1,2\), respectively. Find the probability that the sample minimum is 0 and the sample maximum is 2. (Answer: 0.2700)
In a discrete population, \(P(X=0)=0.3\), \(P(X=1)=0.4\), and \(P(X=2)=0.3\). For a sample of four, explain why the continuous order-statistic density formula cannot be used to find \(P(X_{(2)}=1)\), and state the count-based form of the calculation without evaluating it. (Answer: The population has point masses, so an ordinary continuous density formula does not give the point probability; calculate \(P(N_{\leq1}\geq2)-P(N_{<1}\geq2)\), with binomial probabilities using success probabilities \(0.7\) and \(0.3\), respectively.)

What You'll Learn