The Exact Pointwise Law of the Empirical CDF

One Threshold Turns the Empirical CDF Into a Count

The empirical distribution function records the fraction of a sample that falls at or below a threshold. At a single fixed threshold $x$, this fraction is determined entirely by how many observations satisfy $X_i\le x$. That count has a familiar exact distribution, so probabilities about $F_n(x)$ can be calculated without approximation.

The key is to keep the threshold fixed while the sample varies. The population probability $F(x)$ is then a fixed number, whereas $F_n(x)$ is random because it depends on the observations. No continuity assumption on $F$ is needed: an observation equal to $x$ is included in the count.

Definition: Let $X_1,\ldots,X_n$ be independent and identically distributed random variables with distribution function $F$. Their empirical distribution function is $$ F_n(t)=\frac{1}{n}\sum_{i=1}^n \mathbf{1}\{X_i\le t\},\qquad t\in\mathbb{R}, $$ where $\mathbf{1}\{A\}$ equals 1 when the event $A$ occurs and 0 otherwise. At a fixed threshold $x$, write $p=F(x)=P(X_i\le x)$. Then $nF_n(x)$ is the number of sample observations at or below $x$.

The Exact Pointwise Distribution

For each $i$, the indicator $I_i=\mathbf{1}\{X_i\le x\}$ is Bernoulli with success probability $p$. Independence of the observations implies independence of these indicators. Thus the number $K_x=nF_n(x)=\sum_{i=1}^n I_i$ is a count of $n$ independent successes, each with probability $p$.

Theorem: If $X_1,\ldots,X_n$ are independent and identically distributed with distribution function $F$, then, for every fixed $x$, $$ nF_n(x)\sim \operatorname{Binomial}(n,F(x)). $$ Consequently, for $k=0,1,\ldots,n$, $$ P\left(F_n(x)=\frac{k}{n}\right) =\binom{n}{k}F(x)^k\{1-F(x)\}^{n-k}. $$ The empirical CDF at $x$ has no other possible values. At $F(x)=0$ or $F(x)=1$, the count is respectively 0 or $n$ with probability 1.

To prove the probability formula, consider any particular set of $k$ sample indices that fall at or below $x$. Independence gives probability $p^k(1-p)^{n-k}$ for that set to be exactly the indices counted. There are $\binom{n}{k}$ possible sets, and the corresponding events are disjoint. Adding their probabilities gives the stated mass. The endpoint cases follow directly: if $p=0$, every indicator is zero almost surely; if $p=1$, every indicator is one almost surely. This also proves the theorem when $F$ has atoms, since the success event is precisely $X_i\le x$. $\square$

The support is a grid: $F_n(x)$ can only be $0,1/n,2/n,\ldots,1$. In particular, a proposed value $y$ has probability zero unless $ny$ is an integer from 0 through $n$. When $k=ny$ is such an integer, the point probability is the binomial mass at $k$.

Exact Probabilities and Moments

The binomial law gives exact probabilities for thresholds as well as single values. For any real $a$, the event $F_n(x)\le a$ means that the integer count $K_x$ is at most $\lfloor na\rfloor$. The analogous upper-tail event uses the smallest integer at least $na$. These floor and ceiling rules matter because the empirical CDF lies on a discrete grid.

Pointwise probabilities: With $p=F(x)$ and $K_x=nF_n(x)$, $$ P\{F_n(x)\le a\} =\sum_{k=0}^{\min(n,\lfloor na\rfloor)} \binom{n}{k}p^k(1-p)^{n-k} $$ when $a\ge 0$; the probability is 0 when $a<0$. For $a\ge 0$, $$ P\{F_n(x)\ge a\} =\sum_{k=\max(0,\lceil na\rceil)}^n \binom{n}{k}p^k(1-p)^{n-k}, $$ with the sum interpreted as 0 if its lower limit exceeds $n$.

The first two moments follow from the indicator representation. Each indicator has expectation $p$ and variance $p(1-p)$. Because the indicators are independent, the variance of their sum is the sum of their variances. Dividing the count by $n$ scales its expectation by $1/n$ and its variance by $1/n^2$.

Theorem: Under the same independent, identically distributed sampling assumptions, for every fixed $x$, with $p=F(x)$, $$ E[F_n(x)]=p,\qquad \operatorname{Var}(F_n(x))=\frac{p(1-p)}{n}, $$ and $$ E[F_n(x)^2]=p^2+\frac{p(1-p)}{n}. $$

For a complete derivation, write $F_n(x)=n^{-1}\sum_{i=1}^n I_i$. Since $E[I_i]=p$, linearity of expectation gives $$ E[F_n(x)]=\frac{1}{n}\sum_{i=1}^n E[I_i] =\frac{np}{n}=p. $$ Also, $I_i^2=I_i$, so $E[I_i^2]=p$ and $\operatorname{Var}(I_i)=p-p^2=p(1-p)$. Independence gives $$ \operatorname{Var}(F_n(x)) =\frac{1}{n^2}\sum_{i=1}^n\operatorname{Var}(I_i) =\frac{np(1-p)}{n^2} =\frac{p(1-p)}{n}. $$ Finally, the identity $E[Y^2]=\operatorname{Var}(Y)+\{E[Y]\}^2$, applied to $Y=F_n(x)$, gives the stated second moment. $\square$

Worked Exact Calculations

Worked Example: A Point Probability for Eight Observations

Suppose $X_1,\ldots,X_8$ are independent and identically distributed and $F(x)=0.3$. Find the probability that exactly one quarter of the observations are at or below $x$.

The requested event is $F_8(x)=1/4$, so its count is $8F_8(x)=2$. The count has a binomial distribution with $n=8$ and $p=0.3$. Therefore, $$ P\{F_8(x)=1/4\} =\binom{8}{2}(0.3)^2(0.7)^6. $$

Here $\binom{8}{2}=28$, $(0.3)^2=0.09$, and $(0.7)^6=0.117649$. Thus $$ P\{F_8(x)=1/4\} =28(0.09)(0.117649) =0.29647548. $$

The result is a point probability on the empirical CDF's grid. For example, $F_8(x)=0.3$ is impossible because $8(0.3)=2.4$ is not an integer.

Worked Example: A Threshold at an Atom

Let each observation equal 0 with probability $0.4$ and 2 with probability $0.6$. For $x=0$ and $n=5$, find $P\{F_5(0)\ge 0.8\}$, and calculate the expectation and variance of $F_5(0)$.

Since $F(0)=P(X_i\le 0)=0.4$, the number at or below 0 has a $\operatorname{Binomial}(5,0.4)$ distribution. The event $F_5(0)\ge 0.8$ is the event that this count is at least 4. Therefore, $$ P\{F_5(0)\ge 0.8\} =\binom{5}{4}(0.4)^4(0.6)+(0.4)^5 =5(0.0256)(0.6)+0.01024 =0.0768+0.01024 =0.08704. $$

The pointwise moments are $$ E[F_5(0)]=0.4,\qquad \operatorname{Var}(F_5(0))=\frac{(0.4)(0.6)}{5}=0.048. $$

This example has an atom at the threshold itself. The mass at 0 is included because the empirical CDF uses $X_i\le 0$; replacing that condition with $X_i<0$ would change the success probability and answer a different question.

Worked Example: A Lower-Tail Probability Under a Normal Model

Suppose $X_1,\ldots,X_{10}$ are independent $N(0,1)$ observations, and take $x=0$. Find $P\{F_{10}(0)\le 0.1\}$, then give the mean and variance of $F_{10}(0)$.

By symmetry of the standard normal distribution, $F(0)=P(X_i\le 0)=0.5$. The inequality $F_{10}(0)\le 0.1$ is equivalent to a count of at most $10(0.1)=1$. Hence $$ P\{F_{10}(0)\le 0.1\} =\binom{10}{0}(0.5)^{10}+\binom{10}{1}(0.5)^{10} =\frac{1+10}{1024} =\frac{11}{1024} \approx 0.0107422. $$

The pointwise mean and variance are $$ E[F_{10}(0)]=0.5,\qquad \operatorname{Var}(F_{10}(0))=\frac{(0.5)(0.5)}{10}=0.025. $$

Although the observations have a continuous distribution, the empirical CDF value is still discrete: it can only be one of $0,0.1,0.2,\ldots,1$. Continuity of the sampling distribution does not make the finite-sample empirical CDF value continuous.

Why the Pointwise Law Is Useful

The exact law turns a question about a random fraction into a finite sum of binomial probabilities. It can be used to assess how likely a specified empirical proportion is, to calculate the chance that a sample fraction exceeds a threshold, or to describe the pointwise variability of the empirical CDF. The moments show that the empirical value is centered at the population probability $F(x)$, with variance decreasing as the sample size increases.

The calculation is pointwise: it concerns one fixed $x$. It does not by itself give the joint distribution of empirical CDF values at several thresholds. Nor does the formula require ranks, order statistics, or a continuous population distribution. Its essential assumptions are independent observations with a common distribution and a threshold fixed independently of the realized sample.

Common Mistakes

Using a density value instead of a probability. The binomial success probability is $F(x)=P(X_i\le x)$, not the density evaluated at $x$.
Ignoring the grid. A value $y$ is possible only if $ny$ is an integer between 0 and $n$. Otherwise its point probability is zero.
Dropping equality at an atom. The event in the empirical CDF is $X_i\le x$, so probability mass at $x$ is included in $F(x)$.
Assuming continuity is required. The count argument works for discrete, continuous, and mixed distributions alike.
Changing the sampling assumptions. The binomial result uses independence and a common success probability. It need not hold for dependent observations or observations with different distributions.

Key takeaway: At any fixed threshold $x$, the empirical CDF is a binomial count divided by sample size: $$ F_n(x)=\frac{K_x}{n},\qquad K_x\sim\operatorname{Binomial}(n,F(x)). $$ This gives its exact finite-sample point probabilities, tail probabilities, expectation, variance, and second moment.

Check Your Understanding

Use the fixed-threshold binomial law to answer each question.

If $n=12$ and $F(x)=0.25$, which values can $F_n(x)$ take?
For $n=6$ and $F(x)=0.5$, write an exact expression for $P\{F_n(x)=2/6\}$.
If $n=20$ and $F(x)=0.7$, express $P\{F_n(x)\ge 0.8\}$ as a finite binomial sum, with the correct lower count.
For arbitrary $n$ and $F(x)=p$, state the mean and variance of $F_n(x)$.
Why does the binomial result remain valid when the population distribution has positive probability mass at $x$?