Generalized Inverses and Empirical Quantile Conventions

Why Quantile Conventions Need a Definition

A quantile is often described informally as a point at which a distribution reaches a specified probability level. That description can be ambiguous: a CDF may jump over the level, or it may remain constant across an interval. For empirical distributions, a further ambiguity appears when the probability level multiplied by the sample size is an integer. In that case, two natural inverse conventions can select adjacent order statistics.

The distinction is not merely notation. A quantile convention determines which sample value is reported, especially in small samples and at exact probability levels such as one-half. We begin with definitions that apply to any CDF, then derive their empirical forms.

The weak inequality in the lower inverse and the strict inequality in the upper inverse are both essential. An infimum is a greatest lower bound; it need not belong to the set whose infimum is being taken. In particular, the upper inverse is not necessarily a point where \(F(x)>u\).

Basic Properties of the Two Inverses

For \(0<u<1\), both inverse values are finite. The CDF tends to zero as \(x\) tends to \(-\infty\), and tends to one as \(x\) tends to \(+\infty\). Consequently, the sets in both definitions are nonempty and bounded below. The lower inverse is attained: \(F(F^{-1}_{\mathrm{L}}(u))\geq u\). The upper inverse need not be attained, although its value satisfies \(F(F^{-1}_{\mathrm{U}}(u))\geq u\).

Here are the attainment details. Write \(a=F^{-1}_{\mathrm{L}}(u)\). By the definition of infimum, for every positive integer \(m\), there is a point \(y_m\) with \(F(y_m)\geq u\) and \(a\leq y_m<a+1/m\). Monotonicity gives \(F(a+1/m)\geq u\); right-continuity then gives \(F(a)\geq u\). For \(b=F^{-1}_{\mathrm{U}}(u)\), choose \(z_m\) with \(F(z_m)>u\) and \(b\leq z_m<b+1/m\). Right-continuity and monotonicity imply \(F(b+1/m)\geq F(z_m)>u\), hence \(F(b)\geq u\). This last conclusion does not imply \(F(b)>u\).

To prove the inequality, note that \(\{x:F(x)>u\}\subseteq\{x:F(x)\geq u\}\). The infimum of the larger set cannot exceed the infimum of the smaller set. For the characterization, if \(F(x)\geq u\), then \(x\) belongs to the set defining the lower inverse, so its infimum is at most \(x\). Conversely, if the lower inverse is at most \(x\), its attainment and monotonicity give \(F(x)\geq u\). This proves the theorem. \(\square\)

The two inverses agree at many levels, but not all. A jump can make them differ at precisely the probability level reached by the jump: the lower inverse includes the point where the CDF first reaches that level, while the upper inverse seeks a location where the CDF exceeds it.

Empirical Inverses and Order Statistics

Let \(X_1,\ldots,X_n\) be a sample, and write \(X_{(1)}\leq\cdots\leq X_{(n)}\) for the observations arranged in nondecreasing order. Repeated values are retained in this list. The empirical CDF is $$ F_n(x)=\frac{1}{n}\sum_{i=1}^{n}I(X_i\leq x), $$ where \(I(A)\) is one if \(A\) holds and zero otherwise. Its generalized inverses are defined by the same infimum rules as for a population CDF.

To prove the lower formula, let \(j=\lceil nu\rceil\). At \(X_{(j)}\), at least \(j\) observations are no larger than \(X_{(j)}\), so \(F_n(X_{(j)})\geq j/n\geq u\). For every \(x<X_{(j)}\), at most \(j-1\) observations are no larger than \(x\), so \(F_n(x)\leq(j-1)/n<u\). Therefore the infimum where \(F_n\geq u\) is \(X_{(j)}\).

For the upper formula, let \(r=\lfloor nu\rfloor+1\). At \(X_{(r)}\), at least \(r\) observations are no larger, and \(r>nu\), so \(F_n(X_{(r)})\geq r/n>u\). For every \(x<X_{(r)}\), at most \(r-1=\lfloor nu\rfloor\) observations are no larger, giving \(F_n(x)\leq\lfloor nu\rfloor/n\leq u\). Hence the first location in the sense of an infimum where \(F_n\) exceeds \(u\) is \(X_{(r)}\). The proof also covers ties: the count bounds remain valid for repeated order-statistic values. \(\square\)

When the Probability Level Times the Sample Size Is an Integer

If \(nu\) is not an integer, then \(\lceil nu\rceil=\lfloor nu\rfloor+1\), so the two empirical inverses select the same indexed order statistic. If \(nu=k\) is an integer with \(1\leq k<n\), the indices differ: $$ F^{-1}_{n,\mathrm{L}}(u)=X_{(k)} \qquad\text{and}\qquad F^{-1}_{n,\mathrm{U}}(u)=X_{(k+1)}. $$ This is the precise source of a common index ambiguity.

When \(nu\) is not an integer, the two inverse rules agree. For example, in the same sample, at \(u=0.4\), \(nu=2.4\), so both indices are three and both inverses equal \(X_{(3)}=7\). This comparison shows why it is useful to check \(nu\) before translating a probability level into an index.

Ties do not change the formulas, but they can make two different indices yield the same numerical value. For instance, if \(X_{(k)}=X_{(k+1)}\), the lower and upper inverse values at \(u=k/n\) are equal despite the difference in indices. The formulas describe positions in the ordered list; they do not guarantee distinct observed values.

Common Mistakes and Why the Convention Matters

• Calling the upper inverse the first point where \(F(x)>u\). A strict-exceedance set need not have a smallest element. Define the upper inverse using an infimum, which may not be attained.
• Using the same empirical index at an integer level. If \(nu=k\), the lower inverse uses \(X_{(k)}\) and the upper inverse uses \(X_{(k+1)}\), for \(1\leq k<n\).
• Assuming distinct observations. The order-statistic formulas remain valid with ties; adjacent indexed order statistics may simply have the same value.
• Treating interpolation as a generalized inverse. A midpoint or another interpolated value requires an additional convention. It does not follow from either infimum definition.

These distinctions matter whenever a quantile is reported from a finite sample, compared across software or studies, or used to describe a threshold. A report should state the convention when the exact choice can affect the result. The definition gives the rule; the integer case reveals where alternative rules can diverge.