The CLT for Weighted Averages

Why Weights Change the Sampling Distribution

A simple average gives every observation the same influence. Many summaries instead assign different weights: a combined estimate may give more influence to a more precise measurement, or a population summary may give groups influence according to their population shares. Even when the observations are independent, unequal weights change how much the average varies from sample to sample. They also affect whether a normal approximation is appropriate.

The key is to keep two questions separate. First, what are the exact mean and variance of the weighted average? These follow directly from the weights and the observations’ means and variances. Second, is the standardized weighted average approximately normal? That depends on whether its variation is spread across many small contributions or dominated by a few large ones.

Definition: For independent observations $X_1,\ldots,X_n$ and fixed nonnegative weights $w_1,\ldots,w_n$ that sum to 1, the weighted average is $$ \bar{X}_w=\sum_{i=1}^{n}w_iX_i. $$ The weights determine each observation’s contribution. Equal weights $w_i=1/n$ give the ordinary sample mean.

Mean and Variance

Suppose each $X_i$ has mean $\mu_i$ and variance $\sigma_i^2$. Linearity of expectation gives the mean of the weighted average. Independence means the variance of a sum is the sum of the variances of its terms; multiplying $X_i$ by $w_i$ multiplies its variance by $w_i^2$. Thus:

E(\bar{X}_w)=\sum_{i=1}^{n}w_i\mu_i, \qquad \operatorname{Var}(\bar{X}_w)=\sum_{i=1}^{n}w_i^2\sigma_i^2.

The squared weights in the variance are important. A weight of 0.4 does not contribute 40% of an observation’s variance; it contributes $0.4^2$ times that variance. If all observations have the same mean $\mu$ and variance $\sigma^2$, then the weights summing to 1 give $E(\bar{X}_w)=\mu$, and

\operatorname{Var}(\bar{X}_w)=\sigma^2\sum_{i=1}^{n}w_i^2, \qquad SE(\bar{X}_w)=\sigma\sqrt{\sum_{i=1}^{n}w_i^2}.

Here the standard error is the standard deviation of the weighted average across repetitions of the sampling process. It has the same units as the observations. The formula relies on independence; for dependent observations, covariance terms are also needed.

Worked Example: A Weighted Average of Two Measurements

Suppose two independent, hypothetical measurements $X_1$ and $X_2$ have means 100 and 120 units and standard deviations 4 and 6 units. Consider $\bar{X}_w=0.7X_1+0.3X_2$. Find its mean and standard deviation.

Independence and the weight sum $0.7+0.3=1$ justify using the weighted-average formulas. The mean is

E(\bar{X}_w)=0.7(100)+0.3(120)=70+36=106\text{ units}.

The variances of the measurements are $4^2=16$ and $6^2=36$ square units. Therefore,

\operatorname{Var}(\bar{X}_w) =0.7^2(16)+0.3^2(36) =0.49(16)+0.09(36) =7.84+3.24 =11.08\text{ square units}.

The standard deviation is $\sqrt{11.08}=3.329$ units. The weighted average has expected value 106 units and standard deviation about 3.329 units. If both measurements are normally distributed, their independent weighted sum is exactly normal; no CLT approximation is needed. Without normal component distributions, these moment calculations remain valid, but they do not by themselves establish normality.

Unequal Weights and Effective Sample Size

When the observations are identically distributed with standard deviation $\sigma$, equal weights give variance $\sigma^2/n$. Unequal weights generally increase variance relative to that benchmark. To see why, the Cauchy–Schwarz inequality gives

\left(\sum_{i=1}^{n}w_i\right)^2 \leq n\sum_{i=1}^{n}w_i^2.

Since the weights sum to 1, this means $\sum_i w_i^2\geq 1/n$. Equality holds only when all weights are equal. Consequently, among weights summing to 1, equal weights produce the smallest variance when the observations have a common variance and are independent. Unequal weights can still be appropriate for the purpose of the summary, but they should not be treated as if they were equal weights when measuring variability.

For independent, identically distributed observations, the quantity $1/\sum_iw_i^2$ is often called the effective sample size. It describes the number of equally weighted observations that would give the same variance under the same common variance $\sigma^2$. It is a variance comparison, not necessarily a literal count of independent observations.

Worked Example: Comparing Unequal Weights With Equal Weights

Four independent hypothetical observations have a common standard deviation of 5 units. Compare equal weights with weights 0.4, 0.3, 0.2, and 0.1.

With equal weights, each weight is 0.25. The sum of squared weights is $4(0.25^2)=0.25$, so the average has variance $5^2(0.25)=6.25$ square units and standard deviation 2.5 units.

For the unequal weights, the squared-weight sum is

$$ 0.4^2+0.3^2+0.2^2+0.1^2 =0.16+0.09+0.04+0.01 =0.30. $$

The variance is $5^2(0.30)=7.5$ square units, and the standard deviation is $\sqrt{7.5}=2.739$ units. The effective sample size is $1/0.30=3.333$, smaller than four. Thus these unequal weights give more variability than four equal weights, despite using four observations.

When Is a Weighted Average Approximately Normal?

The weighted-average CLT concerns a sequence of weighted averages as the number of independent contributions grows. Having many observations is helpful, but it is not enough by itself: one observation might still account for a large share of the total variation. A useful general statement uses the Lindeberg condition. For a sequence of problems indexed by $n$, let $X_{ni}$ be independent, with mean $\mu_{ni}$ and variance $\sigma_{ni}^2$, and define

S_n^2=\sum_{i=1}^{n}w_{ni}^2\sigma_{ni}^2.

Assume $S_n^2>0$. The Lindeberg condition requires that, for every fixed $\varepsilon>0$, the total variance contributed by terms whose centered, weighted contributions exceed $\varepsilon S_n$ becomes negligible compared with $S_n^2$:

\frac{1}{S_n^2} \sum_{i=1}^{n} E\!\left[ \bigl(w_{ni}(X_{ni}-\mu_{ni})\bigr)^2 \mathbf{1}\!\left\{ |w_{ni}(X_{ni}-\mu_{ni})|>\varepsilon S_n \right\} \right] \longrightarrow 0.

Here $\mathbf{1}\{\cdot\}$ equals 1 when the condition inside braces holds and 0 otherwise. Under independence and this condition, the standardized weighted sum converges in distribution to a standard normal random variable:

\frac{\bar{X}_w-E(\bar{X}_w)}{S_n} \ \xrightarrow{d}\ N(0,1).

The condition formalizes the idea that no small number of unusually large contributions should control the variation. A convenient sufficient condition applies when the observations are independent and identically distributed with finite, positive variance, and the weights are nonnegative and sum to 1: if

\frac{\max_i w_{ni}}{\sqrt{\sum_{i=1}^{n}w_{ni}^2}} \longrightarrow 0,

then the standardized weighted average is asymptotically normal. The ratio compares the largest weight with the scale of the combined standard deviation. It must tend to zero as the number of observations grows. This condition is useful, but it is not a universal substitute for the Lindeberg condition when the observations have different distributions or variances.

Key result: For independent observations, calculate the weighted average’s exact mean and variance from the individual means, variances, and squared weights. Use a normal approximation only when the independent contributions are sufficiently spread out; a dominant contribution can prevent the weighted-average CLT from being a good guide.

Worked Example: A CLT Approximation With Unequal Weights

Suppose 200 independent hypothetical outcomes $X_i$ are Bernoulli with $P(X_i=1)=0.30$ and $P(X_i=0)=0.70$. Give 40 outcomes weight 0.01 each, and give the other 160 outcomes weight 0.00375 each. Estimate $P(\bar{X}_w>0.36)$.

The weights sum to $40(0.01)+160(0.00375)=0.4+0.6=1$. The observations are independent and identically distributed, with mean 0.30 and variance $0.30(0.70)=0.21$. The weighted average therefore has mean 0.30. Its squared-weight sum is

$$ 40(0.01^2)+160(0.00375^2) =0.004+0.00225 =0.00625. $$

Its variance and standard error are

\operatorname{Var}(\bar{X}_w)=0.21(0.00625)=0.0013125, \qquad SE(\bar{X}_w)=\sqrt{0.0013125}=0.03623.

For these weights, the largest-weight diagnostic is $0.01/\sqrt{0.00625}=0.1265$. No single observation accounts for most of the standard deviation, and there are 200 independent contributions. These features support using a CLT approximation, though the diagnostic is not a guarantee of a particular numerical error. Standardizing the cutoff gives

z=\frac{0.36-0.30}{0.03623}=1.656.

The standard normal upper-tail probability at 1.656 is about 0.0489. Thus the CLT estimates that the probability the weighted average exceeds 0.36 is approximately 0.049. This is an approximation, not an exact probability; the weighted sum of binary outcomes is discrete.

Why a Large Weight Can Defeat the Approximation

Consider instead one observation with weight 0.9 and many other observations sharing the remaining 0.1. Even if the total number of observations is large, the weighted average still depends heavily on that single observation. For identically distributed observations, the largest-weight diagnostic need not approach zero. In that setting, simply saying “the sample size is large” does not justify a normal approximation. The distribution of the dominant observation can remain visible in the distribution of the weighted average.

A related mistake is to calculate variability using $\sigma^2/n$ regardless of the weights. That expression applies to an equally weighted mean of independent, identically distributed observations. For a weighted average, use $\sigma^2\sum_iw_i^2$ under those same distribution and independence assumptions. If observations have different variances, use $\sum_iw_i^2\sigma_i^2$ instead.

Check the setup.
Identify the observations, their weights, and whether they are independent. Confirm that the weights sum to 1 if the statistic is intended to be a weighted average.

Calculate the moments.
Use $E(\bar{X}_w)=\sum_iw_i\mu_i$ and, under independence, $\operatorname{Var}(\bar{X}_w)=\sum_iw_i^2\sigma_i^2$.

Assess the normal approximation.
Check that the weighted contributions are not dominated by a few observations. For identically distributed observations, the maximum-weight diagnostic provides a useful guide as the number of contributions grows.

Standardize and interpret.
If the CLT approximation is justified, subtract the weighted mean and divide by the weighted standard error. Translate the resulting probability back into the original context and units.

Common pitfall: “There are many observations, so their weighted average must be approximately normal.” This ignores the weights. If one or a few observations dominate the total variation, a large count alone does not establish a good CLT approximation. Examine the weight structure as well as the number of observations.

Check Your Understanding

Use the formulas and conditions from this tutorial to answer the following questions.

For independent observations with means $\mu_i$, variances $\sigma_i^2$, and weights summing to 1, write the formulas for the mean and variance of their weighted average.
Four independent observations have a common variance of 9 and weights 0.5, 0.25, 0.15, and 0.10. Calculate the variance of their weighted average.
Why does the variance formula use $w_i^2$, rather than $w_i$, for each independent observation?
What does effective sample size describe when observations are independent and identically distributed?
Why can a large number of observations still be insufficient justification for a normal approximation to a weighted average?

What You'll Learn