Why Reporting Choices Matter
A weighted estimate is only useful if readers can tell what it describes and how it was obtained. A percentage without its denominator may refer to all sampled people, eligible respondents, or only those with a nonmissing answer. A population estimate beside an unlabelled count may be mistaken for the number of people observed. And a precise-looking number can conceal substantial sampling uncertainty.
Publication-ready reporting makes these distinctions visible. A reader should be able to identify the target population, the measure and its units, the denominator, the unweighted sample count, and the uncertainty attached to the estimate. Tables and figures should also disclose whether any results were suppressed or judged too unstable to report.
What Every Display Should Make Clear
Start by defining the analysis population and the denominator. For a percentage, state which records are eligible and whether records with missing responses are excluded from the denominator or shown as a separate category. If a table reports results for several groups, make clear whether each percentage uses its group's records as the denominator or whether all percentages use a common denominator.
Report the unweighted number of records contributing to each estimate. It describes the observed sample basis; it is not a substitute for the weighted population denominator and is not necessarily an effective sample size. If a result depends on a complex sample design, report uncertainty from an appropriate design-based variance procedure. A standard error or confidence interval computed as though observations were a simple random sample may not represent that uncertainty.
For each estimate, use a label that identifies its measure and units. “Estimated percentage of eligible respondents reporting a regular visit” is more informative than “Rate.” For means, name the quantity and units, such as “Mean travel time, minutes.” Put the confidence level on confidence intervals and the uncertainty measure on standard errors. If the interval method or design-based variance approach requires explanation, state it in a note.
| Reporting item | What to specify | Why it matters |
|---|---|---|
| Population and eligibility | Who is represented and which records qualify | Defines the scope of the estimate |
| Measure and units | For example, percentage, persons, or minutes | Prevents ambiguity about what a number means |
| Denominator | Eligible population or records used for the calculation | Makes percentages and rates interpretable |
| Unweighted sample count | Number of observed records contributing to the estimate | Shows the sample basis without confusing it with a population total |
| Uncertainty | Design-based standard error or confidence interval, clearly named | Shows the precision supported by the survey design |
| Limitations | Missing data, suppression, or instability notes | Prevents readers from treating a restricted result as an ordinary estimate |
Worked Example: A Weighted Percentage in a Table
Worked Example: A Weighted Percentage in a Table
Hypothetical data. A survey has 480 eligible respondents with nonmissing answers to a yes-or-no question about having a regular source of care. Their survey weights sum to 12,000, interpreted here as an estimated eligible population of 12,000 people. The weights for respondents answering “Yes” sum to 1,860. Suppose the survey analysis supplies a design-based standard error of 0.018 for the weighted proportion.
The weighted proportion is the weighted “Yes” total divided by the sum of weights for all eligible respondents with an answer:
For a 95% confidence interval using the normal critical value 1.96, the margin of error is \(1.96(0.018)=0.03528\), or 3.528 percentage points. Thus,
The displayed sample count is 480, not 12,000. The latter is the sum of weights and, in this example, estimates the eligible population size. A publication-ready row could read:
| Measure | Weighted estimate | Unweighted n | Estimated eligible population | Design-based SE | 95% confidence interval |
|---|---|---|---|---|---|
| Has a regular source of care | 15.5% | 480 | 12,000 people | 1.8 percentage points | 12.0% to 19.0% |
A note should say that the denominator comprises eligible respondents with nonmissing answers, and that the standard error and interval account for the survey design. In context, the interval describes uncertainty for the population percentage of eligible people with a regular source of care; it does not say that 95% of individuals have this characteristic.
Figures for Weighted Estimates
A figure should communicate the same information as a table, not hide important qualifications in a caption that readers may miss. Give the figure a title or caption naming the population and outcome. Label axes with the measure and units; for percentages, show a percentage scale. Identify the denominator and unweighted sample count, either in the figure or in a clear note. If uncertainty is shown, explain whether the bars represent standard errors, confidence intervals, or another quantity.
For categorical estimates, bars can show weighted percentages or totals. State whether categories are mutually exclusive and exhaustive, and identify how missing responses were handled. If categories are displayed as percentages that sum to 100%, specify the denominator used to form that distribution. Error bars should correspond to the stated uncertainty measure. If they show 95% confidence intervals, say so directly rather than using the vague label “error bars.”
Worked Example: Labelling a Weighted Figure
Hypothetical data. In a survey of 400 respondents with nonmissing answers about usual transportation to a clinic, the survey weights sum to an estimated population of 1,000,000 people. The weighted percentages are 48% public transit, 32% private vehicle, and 20% walking or cycling. The categories are mutually exclusive and include every nonmissing answer. Suppose the design-based standard errors are 3.0, 2.5, and 2.0 percentage points, respectively.
For a public-transit estimate, a normal-based 95% interval is \(48\%\pm1.96(3.0\text{ percentage points})\), or \(48\%\pm5.88\) percentage points. Its endpoints are 42.12% and 53.88%, which can be reported as 42.1% to 53.9%. The corresponding intervals for private vehicle and walking or cycling are \(32\%\pm1.96(2.5\%)\), or 27.1% to 36.9%, and \(20\%\pm1.96(2.0\%)\), or 16.1% to 23.9%.
A clear figure would use three bars with heights 48%, 32%, and 20%, a vertical axis labelled “Weighted percentage of respondents,” and 95% confidence-interval bars. Its caption could read: “Usual transportation to a clinic among respondents with a nonmissing answer; weighted percentages use the estimated population of 1,000,000 eligible people. Unweighted n=400. Error bars show design-based 95% confidence intervals.” The estimates add to 100% because the categories partition the stated denominator; their separate confidence intervals are not required to add to 100%.
The plotted intervals describe uncertainty for each category estimate. They should not be used alone as a test of whether two categories differ: that question concerns uncertainty in a contrast, which is not represented simply by inspecting whether two separate intervals overlap.
Suppression and Unstable Results
Small unweighted counts can make estimates unstable or create disclosure risks. Survey weights do not remove these concerns: a weighted estimate can look smooth while depending on very few observed records. A publication should state its suppression and instability rules rather than implying that an omitted value is zero, unavailable by accident, or equal to a neighbouring value.
Rules depend on the purpose of the report and the applicable disclosure standards; there is no single count threshold that is appropriate in every setting. A rule might suppress a cell when its unweighted sample count is below a prespecified minimum. A separate instability rule might flag an estimate when its relative standard error is large or its confidence interval is too wide for the intended use. State the criterion and how the result is marked. If revealing a total and all but one of its component cells would disclose the suppressed value by subtraction, consider complementary suppression or another disclosure-safe presentation.
Worked Example: Marking a Sparse Estimate
Hypothetical data and reporting policy. A subgroup has 18 respondents with a nonmissing answer. Their weights sum to an estimated eligible population of 1,200 people, and the weights for “Yes” sum to 240. The estimated percentage is \(240/1{,}200=20.0\%\). Suppose the design-based standard error is 9.0 percentage points. The report’s stated policy suppresses estimates based on fewer than 30 unweighted records.
The estimate is below the sample-count threshold, so the published cell should not show 20.0%, its standard error, or its confidence interval. The correct presentation is a suppression label such as “Suppressed (unweighted n<30),” accompanied by a note stating the rule. This label does not mean that the estimate is zero; it means the report withholds the numerical result under its prespecified policy.
For clarity, the underlying uncertainty can still be checked during analysis. A normal-based interval would be \(20.0\%\pm1.96(9.0\text{ percentage points})\), giving 2.36% to 37.64%. Its breadth illustrates the limited precision, while the unweighted count triggers suppression under the policy. The published display should follow the policy consistently, including checking that other published totals do not reveal the withheld cell by subtraction.
A Short Derivation: What a Weighted Percentage Uses
Let \(w_i\) be the survey weight for eligible record \(i\), and let \(y_i=1\) if that record has the characteristic and \(y_i=0\) otherwise. For the records included in the denominator, the weighted proportion is the weighted sum of the indicator divided by the sum of weights:
Since \(y_i\) is one for a “Yes” record and zero otherwise, \(\sum_i w_i y_i\) is the sum of weights for “Yes” records. The denominator is the sum of weights for all included records. If the analyst instead excludes missing answers, both sums must be formed over the eligible nonmissing records. Changing that denominator can change the reported percentage, so the table or figure must identify the rule. The formula defines the point estimate; it does not by itself provide a design-based standard error. That uncertainty must be estimated with a method appropriate to the survey design.
Common Reporting Mistakes
- Calling the sum of weights the sample size. The sum of weights may estimate a population total; the unweighted count records how many observations contributed. Label both correctly when both are reported.
- Leaving the denominator implicit. A percentage is incomplete without its reference population or eligible records, especially when missing answers are excluded.
- Reporting uncertainty without naming it. State whether a displayed number is a standard error or a confidence interval, give the confidence level where relevant, and use units consistent with the estimate.
- Using a simple-random-sample uncertainty calculation by default. For a complex survey, use an appropriate design-based method and say so. The weighted point estimate alone does not determine its standard error.
- Treating a suppressed estimate as zero. Mark it as suppressed, explain the policy, and check whether other displayed totals could reveal it.
- Rounding inconsistently. Use precision that reflects the estimate's uncertainty and apply a consistent rule across comparable rows. Rounding should not make a highly uncertain estimate look exact.
Check Your Understanding
Use these questions to check whether a weighted display would be interpretable to someone who did not conduct the analysis.
- A survey percentage is calculated among eligible respondents with nonmissing answers. What should the report state about its denominator?
- Why should a weighted population total and an unweighted sample count not both be labelled “n” without explanation?
- A figure shows vertical bars above weighted percentages. What information should its caption provide about the bars and the survey design?
- A report suppresses cells with fewer than 25 observed records. What should it display for a suppressed cell, and why is a zero an incorrect substitute?
- Why can a weighted estimate with a large estimated population denominator still be unstable?