Weighted Quantiles and Median Estimation

Why Weighted Quantiles Matter

A population median describes the middle of a target distribution, but a survey sample may represent different numbers of population members through different analysis weights. A median calculated from the raw sample values gives each responding record equal influence. A weighted median instead uses the survey weights to estimate the point where half of the target population’s represented weight lies at or below the value. The same idea estimates quartiles, percentiles, and other quantiles.

The central calculation is an inverse of a cumulative distribution: choose a probability, such as 0.50, and find the smallest value at which the weighted cumulative proportion reaches or exceeds it. Specifying this convention matters because a finite weighted distribution usually has jumps rather than a smooth curve. In this tutorial, quantiles use the lower-inverse convention, which returns a value in the support of the distribution.

Population Quantiles and the Lower-Inverse Convention

Let a finite target population contain units $i=1,\ldots,N$, with measured values $Y_i$ and nonnegative target masses $a_i$, at least one of which is positive. The target mass may be one for every population unit, or it may describe a target distribution in which units have different contributions. Let $A=\sum_{i=1}^{N}a_i$. The weighted population cumulative distribution function (CDF) at a value $y$ is the proportion of target mass attached to values at most $y$.

Definition: The weighted population CDF is $$ F_a(y)=\frac{\sum_{i=1}^{N}a_i I(Y_i\le y)}{A}, \qquad A=\sum_{i=1}^{N}a_i. $$ For $0<p<1$, the lower $p$-quantile is $$ Q_a(p)=\inf\{y:F_a(y)\ge p\}. $$ The weighted population median under this convention is $Q_a(0.50)$. For example, $Q_a(0.25)$ is the lower first quartile and $Q_a(0.90)$ is the lower 90th percentile.

The indicator $I(Y_i\le y)$ equals one when unit $i$'s value is at most $y$, and zero otherwise. Thus the numerator adds the target mass at or below $y$, while the denominator is the total target mass. The CDF is nondecreasing, remains between zero and one, and can jump at observed population values. The quantile definition selects the first value at which the CDF reaches the requested probability, not a value obtained by interpolating between observations.

When all units have equal target mass, this definition is the ordinary finite-population quantile under the same lower-inverse convention. More generally, weighted population quantiles apply when the target distribution itself assigns different masses to units. Survey sampling usually presents a related estimation problem: the population values are not all observed, so the population CDF and its quantiles must be estimated from sampled records and their weights.

Estimating Quantiles From Survey Data

Suppose the survey provides values $x_i$ and positive analysis weights $w_i$ for eligible sampled records $i=1,\ldots,n$. The weights should be the weights intended to represent the target population for the quantity being estimated. The weighted empirical CDF, as introduced in the earlier material on Weighted Empirical Cumulative Distribution Functions, is the weighted proportion of eligible sample weight attached to values at most $y$.

Definition: For a weighted sample with total weight $W=\sum_{i=1}^{n}w_i>0$, define $$ \widehat F_w(y)=\frac{\sum_{i=1}^{n}w_i I(x_i\le y)}{W}. $$ The estimated lower $p$-quantile is $$ \widehat Q_w(p)=\inf\{y:\widehat F_w(y)\ge p\}, \qquad 0<p<1. $$ In particular, $\widehat Q_w(0.50)$ is the estimated weighted median.

To calculate this estimate, order the distinct observed values from smallest to largest. For each distinct value, add together the weights of all records with that value. Then accumulate those combined weights in order, divide by $W$, and select the first distinct value whose cumulative proportion is at least $p$. Aggregating weights at equal values ensures that the cumulative proportion at a value includes every record with that value.

\widehat Q_w(p)=x_{(j)} \quad\text{where }j\text{ is the first distinct-value index for which } \frac{\sum_{\ell:\,x_\ell\le x_{(j)}}w_\ell}{W}\ge p.

The weights need not be integers or literal counts. Their relative sizes determine the weighted distribution: multiplying every weight by the same positive constant leaves $\widehat F_w(y)$, and therefore the estimated quantile, unchanged. The quantile estimate has the same units as the measured variable. A weighted median of 18 minutes, for example, is an estimated value in minutes, not a percentage or a count of respondents.

Worked Example: Estimating a Weighted Median

In a hypothetical survey, four sampled adults report weekly hours of exercise. Their values and positive analysis weights are: 2 hours with weight 2, 4 hours with weight 1, 7 hours with weight 4, and 10 hours with weight 3. Estimate the weighted median and the weighted first quartile using the lower-inverse convention.

The total weight is $W=2+1+4+3=10$. The values are already ordered and distinct. The cumulative weights and proportions are:

At 2 hours: cumulative weight $2$, proportion $2/10=0.20$.
At 4 hours: cumulative weight $2+1=3$, proportion $3/10=0.30$.
At 7 hours: cumulative weight $3+4=7$, proportion $7/10=0.70$.
At 10 hours: cumulative weight $7+3=10$, proportion $10/10=1.00$.

For the median, the target probability is $p=0.50$. The cumulative proportion is 0.30 at 4 hours and 0.70 at 7 hours, so the first value reaching at least 0.50 is 7:

\widehat Q_w(0.50)=7\text{ hours}.

For the first quartile, $p=0.25$. The cumulative proportion at 2 hours is 0.20, below 0.25, while at 4 hours it is 0.30. Therefore $\widehat Q_w(0.25)=4$ hours. These are estimates for the target population represented by the weights; they are not simply the unweighted median and quartile of the four records.

Why Tied Values Must Be Accumulated Together

A cumulative distribution at $y$ includes every record whose value is at most $y$. Consequently, if multiple records have the same value, the CDF at that value includes all of their weights, regardless of their row order in the data file. One can calculate the estimate by grouping equal values first, or by using the defining indicator sum directly at each distinct value.

Worked Example: Correct Cumulative Weight With Tied Values

Consider a hypothetical survey sample of four households with monthly energy use of 5, 5, 8, and 12 units, and analysis weights 1, 3, 2, and 4, respectively. Estimate the weighted median. The two records with energy use of 5 must both be included when evaluating the CDF at 5.

The total weight is $W=1+3+2+4=10$. Combine the weights for each distinct value:

At 5 units, combined weight is $1+3=4$, so the cumulative proportion is $4/10=0.40$.
At 8 units, cumulative weight is $4+2=6$, so the cumulative proportion is $6/10=0.60$.
At 12 units, cumulative weight is $6+4=10$, so the cumulative proportion is $10/10=1.00$.

The first cumulative proportion at least 0.50 occurs at 8 units. Thus $\widehat Q_w(0.50)=8$ units. In particular, evaluating the CDF at 5 gives $4/10$, not $1/10$: both records at 5 satisfy $x_i\le 5$. The answer does not depend on which of those tied records happens to appear first in the file.

A Threshold Characterization

The lower-inverse definition has a useful direct interpretation. At the estimated quantile, at least the requested fraction of total weight is at or below the estimate. At any smaller observed value, the cumulative fraction is below the requested probability. This characterization follows from the first-crossing rule.

Result: Let the distinct sample values be $z_1<z_2<\cdots<z_m$, with combined weights $A_j=\sum_{i:x_i=z_j}w_i$. Define $C_j=\sum_{\ell=1}^{j}A_\ell$. For $0<p<1$, if $j$ is the first index satisfying $C_j/W\ge p$, then $$ \frac{C_j}{W}\ge p \quad\text{and}\quad \frac{C_{j-1}}{W}<p, $$ where $C_0=0$. The estimated lower $p$-quantile is $z_j$.

To justify the result, each $A_j$ is the total weight at the distinct value $z_j$, and all weights are positive. Therefore $C_j$ is the total weight of records with values at most $z_j$, so $\widehat F_w(z_j)=C_j/W$. By the choice of $j$, this proportion is at least $p$. If $j>1$, the first-crossing rule also gives $C_{j-1}/W<p$; for $j=1$, the same statement holds because $C_0/W=0<p$. For any $y<z_j$, the total weight with values at most $y$ cannot exceed $C_{j-1}$, so $\widehat F_w(y)<p$. Thus $z_j$ is exactly the smallest value at which the weighted empirical CDF reaches $p$.

This argument also explains why the cumulative total must be defined at distinct values, or equivalently as all weight with $x_i\le y$. A record-by-record running total can stop partway through a group of tied observations. That partial total is not the CDF at the tied value and cannot establish the first-crossing property.

Worked Example: Estimating a Weighted 75th Percentile

A hypothetical travel survey records commute times of 10, 15, 20, and 30 minutes, with analysis weights 2, 1, 5, and 2. Estimate the lower 75th percentile.

The total weight is $W=2+1+5+2=10$. The cumulative proportions are $2/10=0.20$ at 10 minutes, $3/10=0.30$ at 15 minutes, $8/10=0.80$ at 20 minutes, and $10/10=1.00$ at 30 minutes. The target probability $p=0.75$ has not been reached at 15 minutes, since $0.30<0.75$, but it has been reached at 20 minutes, since $0.80\ge0.75$. Therefore

\widehat Q_w(0.75)=20\text{ minutes}.

This estimate means that the weighted empirical distribution places at least 75% of its total represented weight on commute times of 20 minutes or less, while the weighted proportion at or below any smaller observed value is below 75%. It does not mean that exactly 75% of weighted records have a commute time of 20 minutes.

Interpretation, Assumptions, and Scope

A weighted quantile is meaningful only in relation to the population and variable the weights are meant to represent. The records must refer to the relevant eligible units, the measured values must use a consistent scale, and the weights must be positive for records contributing to the weighted distribution. If the analysis concerns a particular domain, the numerator and denominator must use records in that domain; otherwise the result describes a different target. Records with missing values also require a clearly defined handling rule, because excluding them changes which records and total weight contribute to the estimated distribution.

Under these conditions, the formula produces a descriptive point estimate from the observed weighted empirical distribution. It does not by itself quantify sampling uncertainty, establish that the estimate is close to the population quantile, or correct for a mismatch between the survey weights and the intended target population. Those questions require additional design and data considerations. The purpose here is to define and calculate the weighted quantile estimate, not to attach a confidence interval to it.

Key takeaway: A lower weighted quantile is the first value at which the weighted cumulative proportion reaches the requested probability. For tied observations, include the combined weight of every record at that value before deciding whether the threshold has been reached.

Common Mistake

A tempting but incorrect approach is to sort individual records and treat the running weight after each row as the CDF at that row’s value. With tied values, this can produce a partial total for a value that the CDF must include in full. For example, two records with value 5 and weights 1 and 3 give a record-by-record partial proportion of $1/4$ after the first row, but the CDF at 5 is $4/4=1$, because both records satisfy the condition $x_i\le5$. The correction is to combine tied weights first, or to calculate the weighted proportion using all records with values at most the value being evaluated. Also avoid describing a weighted quantile as an interpolated average unless a different quantile convention has been explicitly specified.

Check Your Understanding

Use the lower-inverse convention and show how the cumulative weights support each answer.

A sample has values 3, 6, and 9, with weights 1, 5, and 4. What is the estimated weighted median?
In a weighted sample, two records both have value 12 and weights 2 and 6. What total weight must be included in the numerator of $\widehat F_w(12)$ from these two records?
Explain why a row-by-row cumulative weight may not equal the weighted empirical CDF at a value that occurs on several rows.
If the cumulative weighted proportion is 0.48 at one distinct value and 0.63 at the next, which is the estimated lower median?
What does an estimated weighted 90th percentile say about the weighted empirical distribution, and what does it not say about sampling uncertainty?