Deriving Effective Sample Size From Weights

Why Weight Concentration Reduces Effective Information

A weighted analysis may include $n$ observations, yet those observations need not contribute equally. If a small number carry most of the total weight, the weighted estimate can behave, in an important variance calculation, like an average based on fewer equally weighted observations. The effective sample size summarizes this concentration in a single number.

Here the target is the standard weight-based effective sample size for one set of nonnegative weights. We derive it from normalized weights, establish its bounds, and explain the assumptions behind its variance interpretation. It is a diagnostic summary of weight concentration; it does not by itself establish that the weights or the causal analysis are valid.

Definition and Basic Properties

Definition: Let $w_1,\ldots,w_n$ be finite, nonnegative weights with $\sum_{i=1}^n w_i>0$. Define the normalized weights $a_i=w_i/\sum_{j=1}^n w_j$, so that $\sum_{i=1}^n a_i=1$. The weight-based effective sample size is $$ n_{\mathrm{eff}}=\frac{1}{\sum_{i=1}^n a_i^2} =\frac{\left(\sum_{i=1}^n w_i\right)^2}{\sum_{i=1}^n w_i^2}. $$

The two expressions are identical because $a_i^2=w_i^2/(\sum_j w_j)^2$. The normalized-weight expression makes the interpretation especially clear: squaring gives greater influence to large shares of total weight. Multiplying every $w_i$ by the same positive constant changes none of the $a_i$, so it also changes neither the effective sample size nor the weighted mean.

An observation with $a_i=0$ contributes nothing to the weighted mean or to $\sum_i a_i^2$. Thus the number of observations with positive weight is a relevant upper limit, even when the data set contains more rows. The definition requires at least one positive weight; no assumption that all weights are strictly positive is needed.

Deriving the Variance-Equivalent Interpretation

The formula has a precise variance interpretation under a simple reference model. Suppose $Y_1,\ldots,Y_n$ are independent random variables with a common finite variance $\sigma^2>0$. Their means need not be specified for this variance calculation. For fixed normalized weights, the weighted average is $\hat\mu_w=\sum_i a_iY_i$. Independence gives its variance as follows.

\operatorname{Var}(\hat\mu_w) =\operatorname{Var}\left(\sum_{i=1}^n a_iY_i\right) =\sum_{i=1}^n a_i^2\operatorname{Var}(Y_i) =\sigma^2\sum_{i=1}^n a_i^2 =\frac{\sigma^2}{n_{\mathrm{eff}}}.

An ordinary mean of $m$ independent observations with this same variance has variance $\sigma^2/m$ when $m$ is a positive integer. To define a variance-equivalent benchmark even when the resulting effective size is not an integer, extend the expression $\sigma^2/m$ to positive real $m$. Matching that benchmark to the weighted-mean variance yields $m=n_{\mathrm{eff}}$.

Theorem: Under fixed normalized weights and independent $Y_1,\ldots,Y_n$ with a common finite variance $\sigma^2>0$, the variance of $\sum_i a_iY_i$ equals the ordinary-mean variance benchmark $\sigma^2/m$ at the unique positive real value $m=n_{\mathrm{eff}}$. This is a variance-equivalent effective sample size, not necessarily a literal sample count.

Proof. By independence, the variance of a sum is the sum of the variances of its terms, and $\operatorname{Var}(a_iY_i)=a_i^2\sigma^2$. Therefore $\operatorname{Var}(\sum_i a_iY_i)=\sigma^2\sum_i a_i^2$. Equating this with the positive-real benchmark $\sigma^2/m$, and using $\sigma^2>0$, gives $\sum_i a_i^2=1/m$. Since $\sum_i a_i^2>0$, the unique solution is $m=1/\sum_i a_i^2=n_{\mathrm{eff}}$. If instead $\sigma^2=0$, both variances are zero for every positive $m$; that equality would not identify a unique benchmark. $\square$

Worked Example: Equal Weights Preserve the Full Sample Size

Suppose four observations have weights $2,2,2,2$. Their sum is $8$, so each normalized weight is $2/8=1/4$. Consequently, $$ \sum_{i=1}^4 a_i^2=4\left(\frac14\right)^2=\frac14, \qquad n_{\mathrm{eff}}=\frac{1}{1/4}=4. $$

For a variance check, let the four outcomes be independent with common variance $9$. The weighted mean has variance $9(1/4)=9/4$, equal to the ordinary-mean benchmark $9/m$ at $m=4$. Equal weights therefore retain all four observations’ worth of information under this reference model.

Weight Variability and the Effective-Size Formula

The definition can also be written in terms of the ordinary coefficient of variation of the weights. Let $\bar w=n^{-1}\sum_iw_i>0$, and define their empirical variance using divisor $n$ by $s^2_{w,n}=n^{-1}\sum_i(w_i-\bar w)^2$. Define $\mathrm{CV}_w^2=s^2_{w,n}/\bar w^2$. With this divisor convention, the identity below is exact:

\sum_{i=1}^n w_i^2 =\sum_{i=1}^n\{(w_i-\bar w)+\bar w\}^2 =\sum_{i=1}^n(w_i-\bar w)^2+n\bar w^2 =n\bar w^2(1+\mathrm{CV}_w^2).

The cross-product vanishes because $\sum_i(w_i-\bar w)=0$. Also, $\sum_iw_i=n\bar w$. Substituting both expressions into the definition gives

n_{\mathrm{eff}} =\frac{n^2\bar w^2}{n\bar w^2(1+\mathrm{CV}_w^2)} =\frac{n}{1+\mathrm{CV}_w^2}.

This is an algebraic identity for the observed weights under the stated divisor convention. Greater weight variability means greater $\mathrm{CV}_w^2$, and therefore a smaller effective size for a fixed $n$. A software-reported variance using divisor $n-1$ has a different numerical convention; when applying this identity, use the variance with divisor $n$, or convert it first.

Worked Example: Unequal Weights and a Fractional Effective Size

Take four raw weights $1,1,1,5$, with total $8$. The normalized weights are $1/8,1/8,1/8,5/8$. Thus $$ \sum_i a_i^2 =3\left(\frac18\right)^2+\left(\frac58\right)^2 =\frac{3+25}{64} =\frac{7}{16}, \qquad n_{\mathrm{eff}}=\frac{16}{7}\approx2.286. $$

Check the coefficient-of-variation calculation. The raw-weight mean is $\bar w=2$, and the deviations are $-1,-1,-1,3$. Their squared values sum to $12$, so $s^2_{w,n}=12/4=3$ and $\mathrm{CV}_w^2=3/4$. Hence $n/(1+\mathrm{CV}_w^2)=4/(1+3/4)=16/7$, matching the direct calculation.

If the outcomes are independent with variance $9$, the weighted mean has variance $9(7/16)=63/16=3.9375$. The positive-real ordinary-mean variance benchmark $9/m$ equals this at $m=16/7$, since $9/(16/7)=63/16$. This does not describe an ordinary mean of $16/7$ observations: a literal sample count must be an integer.

Bounds and When Equality Occurs

Theorem: If $k$ of the $n$ weights are positive, then $1\leq n_{\mathrm{eff}}\leq k$. The upper bound is attained exactly when the $k$ positive normalized weights are equal. The lower bound is attained exactly when one normalized weight is $1$ and all others are zero.

Proof. The normalized weights are nonnegative and sum to one. Therefore $$ 1=\left(\sum_i a_i\right)^2 =\sum_i a_i^2+2\sum_{i<j}a_ia_j \geq \sum_i a_i^2. $$ This gives $n_{\mathrm{eff}}\geq1$, with equality exactly when all products $a_ia_j$ for $i<j$ are zero. Because the weights sum to one, that happens exactly when one weight is one and the rest are zero. For the upper bound, apply Cauchy–Schwarz to the $k$ positive normalized weights: $1=(\sum_{i:a_i>0}a_i)^2\leq k\sum_i a_i^2$. Thus $n_{\mathrm{eff}}\leq k$. Equality in Cauchy–Schwarz holds exactly when those $k$ weights are equal. $\square$

Worked Example: A Zero Weight and the Active Count

Consider weights $0,1,1,2$. There are four records but only three positive weights. Their total is $4$, and their normalized values are $0,1/4,1/4,1/2$. Therefore $$ \sum_i a_i^2=0^2+\left(\frac14\right)^2+\left(\frac14\right)^2+\left(\frac12\right)^2 =\frac{1+1+4}{16}=\frac38, \qquad n_{\mathrm{eff}}=\frac83\approx2.667. $$

The result lies between $1$ and $3$, the count of positive weights, as the bounds require. Counting all four records as though each contributed information would miss the fact that the zero-weight record makes no contribution, while treating the three positive weights as equal would overstate the effective size.

Using the Diagnostic Carefully

The effective sample size is useful for comparing how concentrated two weight sets are, or for noticing that a nominal sample size overstates the number of equally weighted observations represented by a weighted mean. It complements distribution summaries such as the maximum and tail quantiles: those describe particular parts of the weight distribution, whereas $n_{\mathrm{eff}}$ combines all normalized weights through their squared shares.

Its variance-equivalence interpretation has specific limits. It is exact for the fixed-weight, independent, common-variance reference model above. If outcomes have different variances, are dependent, or have associations with the weights, the variance of the weighted mean need not equal $\sigma^2\sum_i a_i^2$. If weights are estimated from data, uncertainty in their estimation also is not captured by this fixed-weight calculation. The effective size is consequently not, in general, the sample size to insert into a standard error formula, and it is not a substitute for an analysis that accounts for the actual design and estimation.

Nor does a large effective size validate a causal analysis. It summarizes weight concentration, not whether the weighting model is appropriate or whether required assumptions hold. Conversely, a small value is a warning that the weighted estimate may be relying heavily on a limited number of observations; it should motivate closer inspection of the weights and the analysis rather than a mechanical rule that determines whether to proceed.

Common Mistakes

Using raw squared weights without the total-weight correction. The correct expression is $(\sum_iw_i)^2/\sum_iw_i^2$, or equivalently $1/\sum_i a_i^2$ after normalization.
Calling a fractional result a literal number of observed people. A value such as $16/7$ is a variance-equivalent benchmark under a stated model, not a sample count that can be collected.
Assuming the variance interpretation needs no conditions. The derivation uses fixed weights, independence, a common finite variance, and a positive variance to identify the unique benchmark.
Using effective size as a universal standard-error correction. It is a weight-concentration diagnostic. Dependence, unequal outcome variances, and estimated weights can change the relevant variance calculation.

Key takeaway: Normalize the weights, square their shares, and take the reciprocal. The resulting effective size equals the nominal size for equal weights and decreases as weight concentration increases; its variance interpretation is conditional, not universal.

Check Your Understanding

Answer each question using the definition and assumptions above. Brief solutions follow the questions.

For normalized weights $1/2,1/4,1/4$, calculate $n_{\mathrm{eff}}$.
Four raw weights have coefficient of variation squared equal to $1$, using divisor $n$. What is their effective size?
Can $n_{\mathrm{eff}}=5$ when only four weights are positive? State the relevant bound.
Why does the variance-equivalent benchmark not identify a unique $m$ when the common outcome variance is zero?

Solutions. (1) The squared shares sum to $1/4+1/16+1/16=3/8$, so $n_{\mathrm{eff}}=8/3$. (2) The identity gives $n_{\mathrm{eff}}=4/(1+1)=2$. (3) No. With $k=4$ positive weights, the upper bound is $n_{\mathrm{eff}}\leq4$. (4) If $\sigma^2=0$, both the weighted-mean variance and the benchmark $\sigma^2/m$ are zero for every positive $m$; equating them cannot select a unique value.