Tutorials › Statistical Theory I - Probability and Distribution Theory › The Empirical Distribution Function as an Indicator Average

Home  /  Tutorials  /  Ranks, Spacings, and Empirical Laws

Ranks, Spacings, and Empirical Laws · Tutorial 484 of 994

The Empirical Distribution Function as an Indicator Average

Learn how each observed value contributes a step to the empirical distribution function and how to verify its defining samplewise properties, even when observations are tied.

Advanced 9 min read

What You'll Learn

  • Express the empirical distribution function as an average of indicator variables.
  • Verify its bounds and monotonicity directly from a fixed sample.
  • Determine its left and right limits at every real number.
  • Compute jump locations and jump sizes when observations are tied.
  • Describe its endpoint limits and the intervals on which it is constant.
  • Distinguish samplewise properties from probabilistic claims about the empirical CDF.

From a Sample to a Distribution Function

A distribution function summarizes how much probability lies at or below each threshold. Given observed data, the empirical distribution function makes the same kind of summary using the fraction of observations at or below the threshold. Its key feature is simple but useful: each observation contributes an indicator that switches from zero to one at that observation’s value.

The properties developed here are samplewise: they hold for every finite list of real observations. No assumption of independence, identical distributions, or continuity is needed. Probability enters only if we later treat the observations as random variables and ask how the empirical function behaves across repeated samples.

Definition: For a fixed sample \(x_1,\ldots,x_n\) of real numbers, where \(n\geq 1\), the empirical distribution function is
$$ F_n(x)=\frac{1}{n}\sum_{i=1}^{n}\mathbf{1}\{x_i\leq x\},\qquad x\in\mathbb{R}. $$
Here \(\mathbf{1}\{x_i\leq x\}\) is one if \(x_i\leq x\), and zero otherwise. If the observations are random variables \(X_1,\ldots,X_n\), the same definition applied to their realized values gives the random function \(F_n(x)=n^{-1}\sum_{i=1}^{n}\mathbf{1}\{X_i\leq x\}\).

At a given threshold \(x\), every observation no greater than \(x\) contributes one, and every observation greater than \(x\) contributes zero. Thus \(nF_n(x)\) is the number of sample observations no greater than \(x\), and \(F_n(x)\) is the fraction of the sample that is no greater than \(x\). Repeated values count repeatedly: an observation is counted once for each index at which it appears.

Worked Example: Evaluating the Indicator Average

Take the sample \(2,5,5,9\), so \(n=4\). Evaluate \(F_n(x)\) at \(x=4\), \(x=5\), and \(x=8\).

At \(x=4\), only the observation 2 is at most 4. The four indicators are \(1,0,0,0\), so \(F_n(4)=(1+0+0+0)/4=1/4\).

At \(x=5\), the observations 2, 5, and 5 are at most 5. The indicators are \(1,1,1,0\), so \(F_n(5)=3/4\). Both copies of 5 count.

At \(x=8\), the same three observations are at most 8, while 9 is not. Therefore \(F_n(8)=3/4\). The function does not change between sample values 5 and 9.

Basic Properties for Every Fixed Sample

For each fixed \(x\), \(F_n(x)\) is an average of \(n\) numbers, each equal to zero or one. This immediately bounds its value between zero and one. More importantly, increasing the threshold cannot make any indicator change from one back to zero. That observation gives monotonicity.

Theorem: For any fixed real sample \(x_1,\ldots,x_n\), \(F_n\) takes values in \([0,1]\) and is nondecreasing. It is right-continuous, has a left limit at every real \(x\), and satisfies
$$ \lim_{x\to-\infty}F_n(x)=0,\qquad \lim_{x\to+\infty}F_n(x)=1. $$ At any \(x\), its jump size is
$$ F_n(x)-F_n(x-)=\frac{1}{n}\#\{i:x_i=x\}, $$ where \(F_n(x-)=\lim_{t\uparrow x}F_n(t)\). In particular, there is a positive jump at \(x\) exactly when \(x\) occurs in the sample.

Proof. Each indicator in the defining sum is either zero or one. Hence \(0\leq F_n(x)\leq 1\). If \(a\leq b\), then \(x_i\leq a\) implies \(x_i\leq b\). Consequently \(\mathbf{1}\{x_i\leq a\}\leq\mathbf{1}\{x_i\leq b\}\) for every \(i\); summing and dividing by \(n\) gives \(F_n(a)\leq F_n(b)\). This proves monotonicity.

To establish the remaining claims, list the distinct observed values in increasing order as \(y_1<\cdots<y_k\), and let \(m_j=\#\{i:x_i=y_j\}\) be the multiplicity of \(y_j\). Each \(m_j\) is a positive integer and \(\sum_{j=1}^{k}m_j=n\). For \(y_j\leq x<y_{j+1}\), the observations counted are precisely those at \(y_1,\ldots,y_j\), so

$$ F_n(x)=\frac{m_1+\cdots+m_j}{n}. $$

For \(x<y_1\), the value is zero; for \(x\geq y_k\), it is one. This finite step-function description proves that \(F_n\) is constant between consecutive distinct observations and gives both endpoint limits: it is exactly zero below the sample minimum and exactly one at or above the sample maximum.

At \(x=y_j\), the definition includes every observation equal to \(y_j\), so \(F_n(y_j)=(m_1+\cdots+m_j)/n\). At thresholds immediately below \(y_j\), but above \(y_{j-1}\) when \(j>1\), the value is \((m_1+\cdots+m_{j-1})/n\); for \(j=1\), it is zero. The left limit therefore exists, and the difference between the value at \(y_j\) and that limit is \(m_j/n\). If \(x\) is not an observed value, there is an interval around \(x\) containing no observed values, and \(F_n\) is constant there. Its left limit then equals its value, so the jump is zero.

Finally, right-continuity follows from the same description. At an observed value \(y_j\), the function has already included all \(m_j\) observations at that value, and it remains at that level for thresholds from \(y_j\) up to the next distinct value. At a point not in the sample, the function is locally constant. In either case the limit from the right equals the function’s value. \(\square\)

Reading Jumps and Ties

The jump at an observed value is its relative frequency in the sample, not necessarily \(1/n\). If there are no ties, each observed value has multiplicity one, so each jump is \(1/n\). If a value occurs several times, its indicator terms switch together and create one larger jump. The total of all jump sizes is one, since the multiplicities sum to \(n\).

Worked Example: Finding Jump Sizes With Ties

Consider the sample \(-1,2,2,2,6\). Its distinct values are \(-1,2,6\), with multiplicities \(1,3,1\), respectively. The jumps are therefore \(1/5\) at \(-1\), \(3/5\) at 2, and \(1/5\) at 6. Their sum is \(1/5+3/5+1/5=1\).

For thresholds below \(-1\), \(F_n(x)=0\). From \(-1\) up to, but not including, 2, it equals \(1/5\). At 2 it becomes \(4/5\), since four observations are at most 2, and it stays \(4/5\) until 6. At 6 it becomes 1 and remains 1 thereafter.

In particular, \(F_n(2)=4/5\), whereas \(F_n(2-)=1/5\). Their difference is \(3/5\), exactly the fraction of sample values equal to 2.

The convention \(x_i\leq x\) matters at a jump: the empirical CDF includes observations equal to the threshold. This is why the function takes its new, higher value at the observed point itself. A related way to express the definition is to place mass \(1/n\) at each observation; tied observations then contribute combined mass at their shared value. This describes the same step function without changing how the sample is counted.

Worked Example: Describing the Entire Function

Let the sample be \(0,3,7,7\). The distinct values are \(0,3,7\), with multiplicities \(1,1,2\). Reading from left to right, the empirical CDF is

$$ F_n(x)= \begin{cases} 0, & x<0,\\ 1/4, & 0\leq x<3,\\ 1/2, & 3\leq x<7,\\ 1, & x\geq 7. \end{cases} $$

For example, at \(x=3\), the values 0 and 3 are counted, giving \(F_n(3)=2/4=1/2\). At \(x=7\), both copies of 7 are counted, giving \(F_n(7)=4/4=1\). Thus the jump at 7 is \(1-1/2=1/2\), which agrees with its multiplicity \(2\) divided by \(n=4\). The function is right-continuous at 0, 3, and 7 because its displayed value at each point is the level it takes immediately to the right.

Why the Indicator Formulation Matters

Writing \(F_n(x)\) as an indicator average exposes its structure without requiring the data to be sorted. For each fixed threshold, the function is a proportion; across thresholds, each individual indicator contributes a single step. This representation is also the natural starting point when studying the empirical CDF as a random function. The samplewise facts proved here do not, by themselves, give a probability distribution for \(F_n(x)\), an expectation, or a covariance. Those are separate probabilistic questions.

A frequent error is to say that every jump must have size \(1/n\). That is true only when the sample has no ties. If an observed value occurs \(m\) times, then all \(m\) corresponding indicators switch from zero to one at that threshold, so the jump is \(m/n\). Another error is to use a strict inequality in the definition without noticing its effect: replacing \(x_i\leq x\) by \(x_i<x\) changes the value at each observed point and gives a different endpoint convention. For the empirical distribution function defined here, equality is included.

Key takeaway: For any finite real sample, the empirical CDF is a nondecreasing, right-continuous step function. It starts at zero and ends at one, with a jump of size equal to the observed relative frequency at each distinct sample value.

Check Your Understanding

Use the indicator definition and samplewise properties to answer the following.

  1. For the sample \(1,1,4,8\), calculate \(F_n(1)\), \(F_n(3)\), and \(F_n(8)\).
  2. For the sample in question 1, list every jump location and its jump size. Do the sizes sum to one?
  3. Explain why \(F_n(a)\leq F_n(b)\) whenever \(a\leq b\), using the individual indicators.
  4. For any finite sample, identify thresholds below which \(F_n\) is zero and thresholds at or above which it is one.
  5. Why does a tie produce a jump larger than \(1/n\), and what is the jump size if a value occurs \(m\) times?

Previous

Next

Back to Tutorials