Distribution-Free Empirical CDFs Under the Probability Integral Transform

Why Transform an Empirical CDF?

An empirical CDF depends on the population distribution through the locations of its observations. The probability integral transform separates those locations from their ranks: when the population CDF is continuous, transforming each observation by that CDF produces a uniform random variable. Consequently, the empirical CDF evaluated at population quantiles has an exact finite-sample law that does not depend on the original continuous distribution.

Continuity is essential, but strict increase is not. A continuous CDF can be flat across gaps in its support, so its inverse needs a precise convention. We will use the left generalized inverse and show why flat regions do not spoil the result.

Quantiles and the Probability Integral Transform

Definition: For a CDF $F$ and $0<u<1$, define its left generalized inverse by $$ F^{-1}(u)=\inf\{x\in\mathbb{R}:F(x)\geq u\}. $$ When $F$ is continuous, this value is finite and $F(F^{-1}(u))=u$. The probability integral transform of a random variable $X$ with CDF $F$ is $U=F(X)$.

The generalized inverse accommodates CDFs that are not strictly increasing. If $F$ is continuous, its limits at minus and plus infinity, together with continuity, ensure that it reaches every level $u$ strictly between zero and one. At the first point where it reaches $u$, its value is exactly $u$.

Theorem: If $X$ has a continuous CDF $F$, then $U=F(X)$ has the uniform distribution on $(0,1)$.

To see why flat regions cause no problem, fix $u\in(0,1)$ and write $q=F^{-1}(u)$. We have $F(q)=u$, so $P(X\leq q)=u$. Every $x\leq q$ satisfies $F(x)\leq u$. Any additional points satisfying $F(x)\leq u$ must have $F(x)=u$ and lie to the right of $q$. Such points are in a level set on which $F$ is constant, which has probability zero. Indeed, the probability assigned to an interval on which a CDF is constant is zero; endpoints also have zero probability because a continuous CDF has no atoms. Thus $P(F(X)\leq u)=u$, the CDF of a uniform random variable. This proves the theorem.

The Empirical CDF at Population Quantiles

Let $X_1,\ldots,X_n$ be independent observations with common continuous CDF $F$. Their empirical CDF is $$ F_n(x)=\frac{1}{n}\sum_{i=1}^n \mathbf{1}\{X_i\leq x\}. $$ Define the transformed empirical CDF, for $0<u<1$, by $$ H_n(u)=F_n(F^{-1}(u)). $$ If $U_i=F(X_i)$, the uniform-sample empirical CDF is $$ G_n(u)=\frac{1}{n}\sum_{i=1}^n \mathbf{1}\{U_i\leq u\}. $$ The transformed observations are independent uniforms, but the conclusion is stronger than that fact alone: the threshold events in these two empirical CDFs agree almost surely.

Theorem: Suppose $X_1,\ldots,X_n$ are independent and identically distributed with continuous CDF $F$. With $U_i=F(X_i)$, for each $u\in(0,1)$, $$ F_n(F^{-1}(u))=G_n(u)\quad\text{almost surely}. $$ In fact, the equality holds simultaneously for all $u\in(0,1)$ on an event of probability one. Therefore the transformed empirical CDF has the same exact finite-sample law as the empirical CDF from an independent uniform sample.

First fix $u\in(0,1)$ and put $q=F^{-1}(u)$. We have $P(X_i\leq q)=F(q)=u$. Also, $X_i\leq q$ implies $F(X_i)\leq u$. Any discrepancy between the events $\{X_i\leq q\}$ and $\{F(X_i)\leq u\}$ can only occur when $X_i>q$ and $F(X_i)=u$. The latter set has probability zero, since $F$ is constant there. Hence $$ \mathbf{1}\{X_i\leq F^{-1}(u)\}=\mathbf{1}\{U_i\leq u\}\quad\text{almost surely}. $$ Summing over $i$ proves the equality for this $u$.

For the simultaneous statement, a continuous CDF has at most countably many nontrivial intervals on which it is constant: their disjoint interiors each contain a distinct rational number. The union of these intervals and their endpoints has probability zero. Outside this union, $X=F^{-1}(F(X))$. Apply this fact to all finitely many observations. For $U_i=F(X_i)$, monotonicity of the generalized inverse and $F(F^{-1}(v))=v$ show that $F^{-1}(U_i)\leq F^{-1}(u)$ exactly when $U_i\leq u$. Thus the indicator equality holds for every $u\in(0,1)$ on the same probability-one event. Since each $U_i$ is uniform, the transformed process is exactly a uniform-sample empirical CDF under this coupling. $\square$

Equivalently, the finite-dimensional law can be derived from multinomial counts. For $0<u_1<\cdots<u_k<1$, the numbers of observations falling in successive quantile intervals have cell probabilities $$ u_1,\quad u_2-u_1,\quad \ldots,\quad u_k-u_{k-1},\quad 1-u_k. $$ These are precisely the probabilities of the corresponding intervals for a uniform sample. The cumulative counts therefore have the same joint law as $G_n(u_1),\ldots,G_n(u_k)$. This derivation makes clear that the result is exact for every finite $n$, not an approximation that requires a large sample.

Worked Examples

Worked Example: An Exponential Population

Suppose $X_1,\ldots,X_5$ are independent exponential observations with rate $2$, so $F(x)=1-e^{-2x}$ for $x\geq0$. Find the distribution of $F_5(F^{-1}(0.4))$.

Solving $1-e^{-2x}=u$ gives $F^{-1}(u)=-\frac12\log(1-u)$, so $$ F^{-1}(0.4)=-\frac12\log(0.6). $$ Each observation is at most this quantile with probability $F(F^{-1}(0.4))=0.4$. The number of such observations is therefore binomial with parameters $5$ and $0.4$, and the empirical CDF is that count divided by $5$: $$ 5F_5(F^{-1}(0.4))\sim\operatorname{Binomial}(5,0.4). $$ For instance, the probability that the empirical CDF equals $0.4$ is the probability of exactly two observations below the quantile: $$ P\left(F_5(F^{-1}(0.4))=0.4\right) =\binom{5}{2}(0.4)^2(0.6)^3 =10(0.16)(0.216)=0.3456. $$ The same answer holds for any continuous population CDF.

Worked Example: Two Quantiles from a Normal Population

Let $X_1,\ldots,X_4$ be independent with $X_i\sim N(0,1)$. Find the probability that exactly one observation is at most $F^{-1}(0.25)$ and exactly three are at most $F^{-1}(0.75)$.

The corresponding uniform intervals are $(0,0.25]$, $(0.25,0.75]$, and $(0.75,1)$, with probabilities $0.25$, $0.50$, and $0.25$. To have the stated cumulative counts, the three interval counts must be $1,2,1$, respectively. Their multinomial probability is $$ \frac{4!}{1!2!1!}(0.25)^1(0.50)^2(0.25)^1 =12(0.25)(0.25)(0.25) =\frac{3}{16}. $$ This is a joint statement about the two empirical CDF values; multiplying two separate binomial probabilities would not give the correct answer because the counts are dependent.

Worked Example: A Continuous CDF with a Flat Region

Suppose $X$ has probability $1/2$ uniformly distributed on $[0,1]$ and probability $1/2$ uniformly distributed on $[2,3]$. Its CDF is continuous but constant at $1/2$ on $[1,2]$. For a sample of size four, find the probability that $F_4(F^{-1}(0.5))=1/4$.

The left generalized inverse gives $F^{-1}(0.5)=1$, and $F(1)=0.5$. Therefore each observation is at most this quantile with probability $1/2$. The count is $\operatorname{Binomial}(4,0.5)$, so $$ P\left(F_4(F^{-1}(0.5))=\frac14\right) =\binom{4}{1}\left(\frac12\right)^1\left(\frac12\right)^3 =\frac{4}{16} =\frac14. $$ The gap in the support does not affect the result: it contains no probability mass. The CDF need not be strictly increasing.

When Continuity Fails

If $F$ has an atom, $F(X)$ need not be uniform. For example, let $P(X=0)=P(X=1)=1/2$. Then $F(X)$ equals $1/2$ or $1$, each with probability $1/2$, rather than being continuously uniform. At $u=0.4$, the generalized inverse is $F^{-1}(0.4)=0$, so $P(X\leq F^{-1}(0.4))=1/2$, not $0.4$. Even for a single observation, the threshold count at this quantile does not have the uniform-model success probability.

A common mistake is to reason that $F(F^{-1}(u))=u$ for every CDF. That equality is guaranteed here by continuity; for a CDF with a jump, the value at the quantile can exceed $u$. Another mistake is to demand strict increase. Strict increase is unnecessary: continuity ensures the desired quantile level, while flat regions carry zero probability. The distribution-free result concerns evaluation at population quantiles; it does not say that the untransformed observations themselves have a uniform distribution.

Key takeaway: For an independent sample from any continuous CDF $F$, evaluating $F_n$ at $F^{-1}(u)$ gives the same empirical-process law as evaluating a uniform-sample empirical CDF at $u$. Continuity, not strict increase, is the decisive condition.

Check Your Understanding

Work through these questions using the generalized inverse convention above.

For a continuous CDF, why does $F(F^{-1}(u))=u$ hold for $0<u<1$, even if the CDF has a flat region?
If $n=6$, what is the distribution of $6F_n(F^{-1}(0.3))$ under a continuous population CDF?
For $0<u_1<u_2<1$, name the three multinomial cell probabilities that determine the joint empirical CDF values at these quantiles.
Why does a flat gap in the support not invalidate the probability integral transform?
Give a discrete example in which $F(X)$ is not uniform, and identify the role of the atom.