Mixed Order Statistic Calculation Workshop

Choosing a Calculation That Matches the Rank Event

Order-statistic problems often become manageable once the event is stated precisely. A threshold event such as “the second-smallest observation is at most \(t\), but the third-smallest is greater than \(t\)” is a count: exactly two observations are at most \(t\). A density question about an iid sample, by contrast, calls for the order-statistic density. These are related calculations, but they use different representations.

The distinction between identical and nonidentical distributions matters. Independent observations with different distributions do not have interchangeable ranks. In an iid continuous sample, symmetry gives a particularly simple density formula, and a strictly increasing parent cdf transforms all observations to iid uniforms without changing their order.

Threshold Events for Independent, Nonidentical Observations

Suppose \(X_1,\ldots,X_n\) are independent, continuous random variables, but their distributions need not be the same. At a fixed threshold \(t\), define \(p_i=F_i(t)=P(X_i\le t)\), where \(F_i\) is the cdf of \(X_i\). The indicators \(I_i=1_{\{X_i\le t\}}\) are independent Bernoulli variables with success probabilities \(p_i\), which may differ.

The event \(X_{(k)}\le t<X_{(k+1)}\) means exactly \(k\) observations are at most \(t\). Its probability is found by summing over all subsets of \(k\) observations that could fall at or below \(t\). Each term multiplies the probabilities for those observations by the probabilities that all remaining observations exceed \(t\).

This subset formula is a Poisson-binomial calculation: it is the probability of exactly \(k\) successes among independent Bernoulli trials with potentially unequal success probabilities. For the more general event \(X_{(k)}\le t\), at least \(k\) observations must be at or below \(t\), so the probabilities for exactly \(k,k+1,\ldots,n\) observations are added.

Order-Statistic Density for an Iid Continuous Sample

Now suppose \(X_1,\ldots,X_n\) are iid from a continuous distribution with cdf \(F_X\) and density \(f_X\). The density of \(X_{(k)}\) at \(x\) is obtained by selecting which observation lies near \(x\), requiring exactly \(k-1\) observations below it, and requiring the remaining \(n-k\) observations above it.

The formula requires identical distributions as well as independence. It does not apply unchanged to independent observations with different cdfs; a particular rank in a nonidentical sample may instead require summing over identities or integrating over possible values, as in the next example.

Why the Iid Density Formula Holds

For a small interval from \(x\) to \(x+dx\), one specified observation has probability approximately \(f_X(x)\,dx\) of falling in that interval. For it to be the \(k\)th-smallest, exactly \(k-1\) of the other \(n-1\) observations must be below \(x\), and the other \(n-k\) must be above \(x\). Independence gives the product of those probabilities.

There are \(n\) choices for which observation falls in the small interval. Multiplying the expression by \(n\) and dividing by \(dx\) gives the density. The coefficient is \(n\binom{n-1}{k-1}=n!/((k-1)!(n-k)!)\). This argument depends on the iid assumption: if the observations have different distributions, the probability of being below \(x\) depends on which observations were selected.

Transforming Iid Ranks Through the Parent Cdf

If \(F_X\) is continuous and strictly increasing on the support of \(X\), then \(U_i=F_X(X_i)\) is uniform on \((0,1)\). For an iid sample, the \(U_i\) are also independent. Because a strictly increasing function preserves order, \(U_{(k)}=F_X(X_{(k)})\). Thus rank calculations can be made on the uniform scale.

For \(n\) iid uniform observations, the density of \(U_{(k)}\) is

This is a Beta density with parameters \(k\) and \(n-k+1\). Returning to the original scale uses the density transformation \(f_{X_{(k)}}(x)=f_{U_{(k)}}(F_X(x))f_X(x)\), which agrees with the iid order-statistic density above. The strictly increasing condition matters for rank preservation. If different cdfs \(F_i\) are applied to nonidentical observations, the transformed variables may each be uniform, but applying different transformations need not preserve comparisons between the original observations.

A Rank Calculation With Nonidentical Supports

For a nonidentical sample, a specified ordering can be calculated by conditioning on one observation’s value and integrating the probabilities that the others fall in the required ranges. The limits of integration and the cdfs or densities must match the identities of the observations.

Practical Checks Before You Calculate

Rank problems reward careful event translation. Before calculating, decide whether the event is about a count below a threshold, a particular identity occupying a rank, or the value of an iid order statistic. Then check the assumptions and the direction of any transformation.

• For independent nonidentical observations, use the relevant \(F_i(t)\) values or integrate using the individual densities. Do not use a binomial count unless the threshold probabilities are equal.
• For an iid continuous sample, the order-statistic density includes both the parent density \(f_X(x)\) and the below/above factors involving \(F_X(x)\).
• For an iid sample transformed by one common strictly increasing cdf, ranks are preserved. Applying different transformations to nonidentical observations does not generally preserve ranks.
• When an event says “exactly \(k\) are below \(t\),” count the subset possibilities. When it says “the \(k\)th is at most \(t\),” include all counts of \(k\) or more.

These representations are useful well beyond uniform or exponential examples: the threshold method depends only on the individual cdf values, while the iid density formula applies across continuous parent distributions. Matching the method to the event is the central step in a reliable rank calculation.