Confidence Intervals for Weighted Totals and Means

Why Intervals Need the Survey Design

A weighted estimate describes a population quantity, but it is still based on a sample. A confidence interval expresses the sampling uncertainty around that estimate. For a complex survey, the interval must reflect how the sample was actually selected: unequal selection probabilities, stratification, clustering, and any other features used in the design-based standard error. An interval calculated as if the weighted records were a simple random sample can give misleading precision.

This tutorial uses design-consistent standard errors for descriptive estimates. The standard error is the estimated standard deviation of an estimator over repeated samples selected under the survey design. Here, the focus is how to turn that standard error into an interval for a population total or mean, and how to report the result with its population-scale units.

Definition: A confidence interval for a population quantity is an interval produced by a procedure that, under its assumptions, covers that fixed quantity at a stated rate over repeated samples. A design-consistent interval uses an estimate and a standard error that account for the sampling design. The population quantity is fixed; it is the interval procedure that varies from sample to sample.

Intervals for Totals and Means

Let $y_i$ be the value of an outcome for sampled unit $i$, and let $w_i$ be its survey weight. A weighted estimate of the population total of $y$ is commonly formed as $\hat{T}_y=\sum_{i\in s}w_i y_i$, where $s$ is the set of sampled units. Its units are the outcome units summed over the population. For example, if $y_i$ is annual health-care spending in dollars per person, the total is in dollars for the target population.

A population mean is an average per population unit. When the population size $N$ is known, the mean is $\mu_y=T_y/N$. When the population size is estimated from the survey, a commonly used weighted mean is the ratio $\hat{\mu}_y=\hat{T}_y/\hat{N}$, where $\hat{N}=\sum_{i\in s}w_i$. The mean and its standard error are expressed in the outcome’s units, such as dollars per person or days per patient.

Formula: Let $\hat{\theta}$ denote the weighted estimate of a total or mean, and let $\widehat{SE}(\hat{\theta})$ be its design-consistent standard error. A two-sided confidence interval has the form

\hat{\theta}\ \pm\ c\,\widehat{SE}(\hat{\theta}),

where $c$ is a critical value chosen for the confidence level and the sampling distribution approximation. For an approximate normal interval, $c=z_{1-\alpha/2}$. When a suitable design-based degrees of freedom $df$ are used, a t interval uses $c=t_{1-\alpha/2,df}$.

For a 95% interval, $\alpha=0.05$, so the critical value is the 97.5th percentile of the reference distribution. The normal critical value is approximately 1.96. A t critical value depends on $df$ and is generally larger when the degrees of freedom are limited. The degrees of freedom should come from the survey variance procedure; they are not automatically the sample size minus one. For example, with Taylor linearization, degrees of freedom may be based on the number of sampled primary sampling units and strata. The appropriate calculation depends on the design and variance method.

The standard error in the formula must be for the same estimator and target as the interval. For a total, use the design-based standard error of the total estimate. For a ratio mean, use the design-based standard error of that ratio estimate. Under standard ratio linearization, uncertainty in the mean is assessed through the weighted residual $y_i-\hat{\mu}_y$, with the residual-total standard error scaled by the estimated denominator. In practice, use the variance procedure appropriate to the survey design and estimator rather than substituting an ordinary unweighted standard deviation.

Why the Formula Gives a Confidence Interval

The interval is justified by the sampling distribution of the estimator, not by treating the observed estimate as a random population value. If the design-based standard error estimates the sampling variability well and the standardized estimator is approximately normal (or approximately t with the specified degrees of freedom), then the interval procedure has approximately the stated coverage.

P\left( -c \leq \frac{\hat{\theta}-\theta}{\widehat{SE}(\hat{\theta})} \leq c \right) \approx 1-\alpha .

Rearranging the inequalities gives

P\left( \hat{\theta}-c\,\widehat{SE}(\hat{\theta}) \leq \theta \leq \hat{\theta}+c\,\widehat{SE}(\hat{\theta}) \right) \approx 1-\alpha .

Thus, over repeated samples drawn using the design, approximately $100(1-\alpha)\%$ of intervals constructed this way cover the fixed population quantity. The approximation relies on an appropriate design-based standard error and a sufficiently accurate reference distribution. It is not a claim that the particular interval already calculated has a $1-\alpha$ probability of containing the fixed quantity.

Worked Example: A Confidence Interval for a Population Total

Suppose a hypothetical probability survey estimates the total number of annual clinic visits made by adults in a region. The weighted total is 12,400 visits, its design-consistent standard error is 620 visits, and the survey variance procedure supplies $df=24$. Construct a 95% t interval. Assume the total estimator is approximately t-distributed after standardization using these degrees of freedom, and that the design-based standard error is appropriate for this survey.

For $df=24$, the two-sided 95% t critical value is approximately 2.064. The margin of error is

2.064(620)=1{,}279.68\text{ visits}.

Therefore,

12{,}400\pm1{,}279.68 = (11{,}120.32,\ 13{,}679.68)\text{ visits}.

Rounded suitably, the 95% interval is approximately 11,120 to 13,680 annual clinic visits in the adult population. Under the stated design and distribution assumptions, the procedure used to construct such intervals would cover the fixed population total in about 95% of repeated samples. The endpoints are population totals, not counts of visits observed in the sample.

Weighted Means and Their Units

A weighted mean is not just a weighted total with a different label. Its target is an average per population unit, and the interval must use the standard error of that mean. If a mean is estimated as a ratio of weighted totals, its sampling variability depends on both numerator and denominator. In general, it is not correct to divide the standard error of the numerator by the estimated denominator and ignore uncertainty in the denominator.

A design-based variance procedure for the ratio accounts for the ratio structure, often by linearizing the estimator. Conceptually, the residual $y_i-\hat{\mu}_y$ measures how each sampled unit’s outcome differs from the estimated population mean. The design-based variability of the weighted residual total, scaled by the estimated population size, provides an approximate standard error for the mean. Use the survey procedure’s standard error for the ratio mean, rather than calculating one from an unweighted formula.

Worked Example: A Confidence Interval for a Weighted Mean

In a hypothetical survey of adults, the weighted total of annual out-of-pocket spending is estimated as $84,200, and the estimated population size is 10,000 adults. The ratio mean estimate is $8.42 per adult. Suppose the design-based standard error for this mean is $0.31 per adult, with $df=24$. Assume the design-based standard error and t reference approximation are suitable. Find and interpret a 95% interval.

The mean estimate is the ratio

\hat{\mu}_y=\frac{\$84{,}200}{10{,}000\text{ adults}} =\$8.42\text{ per adult}.

Using $t_{0.975,24}=2.064$, the margin of error is

2.064(\$0.31\text{ per adult}) =\$0.63984\text{ per adult}.

The interval is

\$8.42\pm\$0.63984 = (\$7.78016,\ \$9.05984)\text{ per adult}.

Rounded to cents, the 95% interval is $7.78 to $9.06 in annual out-of-pocket spending per adult. It concerns the population mean, not the spending of every adult or the range of individual spending values.

When a Known Population Size Connects Means and Totals

If the population size $N$ is known and treated as fixed, a mean estimate can be defined as $\hat{\mu}_y=\hat{T}_y/N$. Multiplication by a fixed positive constant transforms both the estimate and its standard error: $\widehat{SE}(\hat{T}_y)=N\widehat{SE}(\hat{\mu}_y)$. Consequently, an interval for the mean can be multiplied by $N$ to obtain the corresponding interval for the total, provided both intervals use the same critical value and the same underlying estimator.

Worked Example: Converting a Mean Interval to a Total Interval

Suppose a hypothetical survey estimates a mean of 3.60 days of missed work per person in a population of exactly 18,000 people. The design-based standard error of the mean is 0.22 days per person, and the variance procedure gives $df=18$. Assume the design-based t approximation is suitable. Construct a 95% interval for the population total of missed workdays.

The estimated total is

\hat{T}_y=N\hat{\mu}_y =18{,}000(3.60) =64{,}800\text{ person-days}.

The standard error of the total is the mean standard error multiplied by the fixed population size:

\widehat{SE}(\hat{T}_y)=18{,}000(0.22) =3{,}960\text{ person-days}.

For $df=18$, $t_{0.975,18}$ is approximately 2.101. The margin of error is

2.101(3{,}960)=8{,}319.96\text{ person-days}.

Thus the interval is

64{,}800\pm8{,}319.96 = (56{,}480.04,\ 73{,}119.96)\text{ person-days}.

Rounded to the nearest person-day, the 95% interval is 56,480 to 73,120 person-days. This conversion depends on $N$ being known and fixed. If the denominator is estimated from the survey, the mean is a ratio and the interval for the total cannot generally be obtained by multiplying the mean interval by an estimated population size while ignoring uncertainty in that denominator.

Choosing and Reporting the Interval

A normal interval can be appropriate when the design-based estimator is well approximated by a normal distribution and the degrees of freedom are large. A t interval is often used when the variance estimation procedure provides limited design degrees of freedom. The choice is a property of the inference procedure; it is not decided by whether the number of sampled records seems large. In particular, many records from a small number of sampled clusters do not necessarily provide many independent pieces of information for variance estimation.

A clear report identifies the estimate, the confidence level, the interval endpoints, the population target, and the units. It should also make clear whether the estimate is a total or a mean. For example: “The estimated population total was 12,400 annual clinic visits (95% design-based CI: 11,120 to 13,680 visits).” For a mean, report the per-unit scale: “The estimated annual mean was $8.42 per adult (95% design-based CI: $7.78 to $9.06 per adult).” A statement that gives endpoints without units or target can leave the reader unsure what population quantity is being estimated.

Common Mistakes

Using an ordinary standard error. Treating weighted records as a simple random sample can ignore clustering, stratification, and unequal selection probabilities. Use a standard error calculated for the survey design and the specific total or mean estimator.
Using the wrong degrees of freedom. The sample size minus one is not automatically the design-based $df$. Use the degrees of freedom supplied or justified by the survey variance procedure.
Confusing total and mean units. A total of annual spending is in dollars for the population; a mean is in dollars per person. Keep the target and units explicit in the calculation and report.
Interpreting the interval as a probability statement about this fixed parameter. A 95% confidence level describes the long-run coverage of the procedure under its assumptions. It does not mean there is a 95% probability that the fixed population quantity lies in the calculated interval.
Assuming a symmetric interval must be plausible at every boundary. The estimate plus or minus a standard error gives a symmetric approximation. If that interval conflicts with known limits of the target, the approximation may be unsuitable; do not silently truncate the endpoints and claim the original coverage without a justified method.

Key takeaway: For a weighted total or mean, form the interval from the estimate and its matching design-consistent standard error, using a justified normal or design-based t critical value. Interpret the result as a repeated-sampling procedure for a fixed population quantity, and report the target and its population-scale units.

Check Your Understanding

Use the design-based interval principles to answer these questions.

A weighted total has estimate 5,600 units and design-based standard error 280 units. Write the formula for its 95% normal interval before calculating it.
Why might a t critical value based on design degrees of freedom be larger than 1.96?
A ratio mean has a weighted total of $45,000 and an estimated population size of 6,000. What is the mean estimate, and what are its units?
A mean interval is based on a known population size $N$. How do the total estimate and its standard error relate to the mean estimate and its standard error?
Explain why “there is a 95% probability that the fixed population mean is in this calculated interval” is not the standard confidence-interval interpretation.