Spacings from General Continuous Samples

From Observations to Probability-Scale Spacings

The numerical distance between two observations does not by itself tell us how much probability lies between them. For a distribution with a rapidly changing density in one region and a slowly changing density in another, equal-length intervals can contain very different probabilities. The cumulative distribution function provides a common scale: the difference \(F(b)-F(a)\) is the probability assigned to \((a,b]\).

For an independent sample from a continuous distribution, applying its CDF to each observation gives a uniform sample. Ordering those transformed values therefore gives uniform order statistics, and the gaps between them have a distribution that does not depend on the original continuous distribution. These are probability-scale spacings; they are generally not the same as the gaps between the original observations.

The first and last spacings measure probability before the sample minimum and after the sample maximum. Each interior spacing measures the probability between two adjacent order statistics. In particular, \(D_i\) is a CDF increment, not the raw difference \(X_{(i)}-X_{(i-1)}\).

Why the Probability Integral Transform Works

The probability integral transform states that \(F(X)\) is uniform on \((0,1)\) when \(X\) has a continuous CDF \(F\). Strict increase of \(F\) is not required. A continuous CDF can be constant over intervals, but these intervals have probability zero under the distribution, so they do not disrupt the transform.

Proof. Fix \(u\) with \(0<u<1\), and set \(b=\sup\{x:F(x)\le u\}\). Because \(F(x)\) tends to 0 as \(x\) tends to \(-\infty\) and to 1 as \(x\) tends to \(+\infty\), this supremum is finite, and the set is nonempty. Monotonicity and continuity imply that \(\{x:F(x)\le u\}=(-\infty,b]\) and \(F(b)=u\). Indeed, if \(F(b)<u\), continuity would make \(F(x)\le u\) for some \(x>b\), contradicting the definition of \(b\). Therefore

This is the uniform CDF for \(0<u<1\). At \(u=0\) the probability is 0, as follows by taking \(u\) down to 0; at \(u=1\) it is 1. Thus \(F(X)\sim\operatorname{Uniform}(0,1)\). \(\square\)

Now apply this result separately to each observation: \(F(X_1),\ldots,F(X_n)\) are independent uniform random variables, because the \(X_j\) are independent and each transform is a function of its own observation. Since \(F\) is nondecreasing, sorting the observations also sorts their transformed values. Consequently, \(F(X_{(i)})\) is the \(i\)th order statistic in an independent uniform sample.

The Joint Law of the Spacings

Because the transformed observations are independent uniform random variables, their order statistics have joint density \(n!\) on the region \(0<u_1<\cdots<u_n<1\). To see the factor \(n!\), away from ties any ordered vector can arise from exactly \(n!\) permutations of the original uniform observations, each with joint density 1. The ties have probability zero.

Proof. Write \(u_i=F(X_{(i)})\). In terms of the spacings,

The transformation from \((d_1,\ldots,d_n)\) to \((u_1,\ldots,u_n)\) is triangular with diagonal entries 1, so its Jacobian determinant is 1. The ordered uniform vector has density \(n!\) on \(0<u_1<\cdots<u_n<1\). Under this transformation that region becomes \(d_j>0\) and \(\sum_{j=1}^n d_j<1\); the remaining spacing is \(d_{n+1}=1-\sum_{j=1}^n d_j>0\). The change-of-variables formula therefore gives density \(n!\) on the stated simplex. This is precisely the Dirichlet density with parameters all equal to 1. \(\square\)

The law is distribution-free: it is the same for every continuous \(F\). For example, all \(n+1\) spacings have the same mean. Their joint law is unchanged by permuting their coordinates, and because they sum to one, the common expectation must be \(1/(n+1)\). This statement concerns probability-scale gaps; it does not say that the raw gaps in the original units have equal means or identical distributions.

Interpreting and Using Probability-Scale Gaps

For adjacent sample order statistics \(X_{(i-1)}\) and \(X_{(i)}\), the interior spacing is

where the probability on the right is for a new draw \(X\) from the same distribution, treating the observed endpoints as fixed. Continuity makes the choice of including or excluding an endpoint immaterial to this probability. The spacing is therefore an interval probability, not a length in the units of \(X\).

If \(F\) has a density \(f\), then for finite adjacent endpoints \(a<b\), the corresponding increment can also be written \(\int_a^b f(x)\,dx\). When a density is larger between the endpoints, a given raw interval length can account for a larger probability gap. For a strictly increasing CDF with inverse \(F^{-1}\), raw order-statistic gaps can be recovered from the uniform order statistics as differences such as \(F^{-1}(U_{(i)})-F^{-1}(U_{(i-1)})\); their distribution consequently depends on \(F^{-1}\), even though the uniform spacings do not depend on \(F\).

Common Mistake

A frequent error is to calculate a probability-scale gap as \(F(X_{(i)}-X_{(i-1)})\). This applies the CDF to the length of the interval rather than subtracting the CDF values at its endpoints. The correct calculation is \(F(X_{(i)})-F(X_{(i-1)})\). For the exponential example, the interior gap between \(0.2\) and \(0.7\) is \(e^{-0.2}-e^{-0.7}\), not \(F(0.5)\). Also, when computing decimal values, subtract the unrounded CDF expressions first and round only the final gap; subtracting already rounded CDF values can change the last reported digit.