Why Denominator Audits Matter
A weighted descriptive estimate is a ratio of weighted totals. Its denominator therefore does more than supply a scale: it states which records define the group being described. A weighted mean of an outcome among eligible adults, a weighted share among respondents with a recorded outcome, and an event rate per person-year have different denominators because they describe different quantities.
An estimate can be arithmetically correct and still answer the wrong question. This happens when an eligibility filter excludes records that belong in the target group, includes records that do not belong, or applies different record rules to the numerator and denominator. A reliable audit begins by stating who or what is eligible, then checking exactly which weighted records enter each total.
The distinction between eligibility and observation is essential. A record with a missing outcome may still belong to the eligible target group. Excluding it from a calculation because its outcome is unavailable does not make it ineligible; it changes the records represented by the calculation. Unless a suitable missing-data or nonresponse adjustment is used, a weighted estimate based only on observed outcomes describes the eligible records with observed outcomes, not automatically every eligible record in the target population.
Write the Target Before the Formula
For an outcome \(y_i\), the weighted mean among eligible records with observed outcomes is
Both the numerator and denominator use the same eligible, outcome-observed records. The denominator is the sum of their weights, not the number of rows in the file and not the sum of weights for every eligible record regardless of whether the outcome is observed. The expression is defined only when \(\sum_i w_i A_i R_i>0\).
If every eligible record has an observed outcome, then \(R_i=1\) for all eligible records and the formula reduces to the weighted mean over eligible records. If some eligible records have missing outcomes, the complete-record mean remains a well-defined descriptive ratio, but its direct interpretation is narrower: it is a mean among eligible records with observed outcomes. Extending it to all eligible records requires additional information or assumptions; silently placing missing outcomes in the denominator is not a remedy.
For a binary outcome \(y_i\), where \(y_i=1\) denotes the event and \(y_i=0\) denotes no event, the weighted proportion among eligible records with observed outcomes is
The denominator includes eligible observed records with either outcome value. Using only event records in the denominator would produce an event share of 100% by construction, not an estimate of the event proportion.
Worked Example: Missing Outcomes and the Weighted-Mean Denominator
In hypothetical survey data, five adults are eligible for a blood-pressure summary. One additional record is ineligible. The table gives each record’s weight and systolic blood pressure in millimetres of mercury (mmHg).
| Record | Eligible? | Weight \(w_i\) | Blood pressure (mmHg) |
|---|---|---|---|
| A | Yes | 2 | 120 |
| B | Yes | 3 | 140 |
| C | Yes | 1 | 130 |
| D | Yes | 4 | Missing |
| E | Yes | 5 | 150 |
| F | No | 10 | 200 |
The complete-record mean among eligible adults excludes D because the outcome is missing, and excludes F because the record is ineligible. Its weighted numerator is \(2(120)+3(140)+1(130)+5(150)=1{,}540\) mmHg-weight units. Its denominator is \(2+3+1+5=11\) weight units. Therefore,
This is the weighted mean for eligible records with an observed blood pressure. The total weight of all five eligible records is 15, but using 15 as the denominator while retaining the same numerator would give \(1{,}540/15=102.7\) mmHg. That calculation implicitly gives the missing record a zero blood pressure contribution while still counting its weight; it is not a valid complete-record mean and does not recover the mean for all eligible adults.
Exposure Denominators Must Match the Rate
A weighted rate per unit of exposure has a different denominator from a weighted proportion. If \(d_i\) is the observed number of events for record \(i\), and \(e_i\) is its observed exposure, the rate among eligible records with both quantities available is
Here \(R_i=1\) means that the event count and its corresponding exposure are observed and usable for the rate calculation. The numerator is a weighted event total; the denominator is a weighted exposure total. Consequently, \(\hat{r}\) has units of events per exposure unit. The records contributing exposure without events must remain in the denominator when they belong to the defined eligible group and their exposure is observed. A record with zero events can still contribute substantial exposure.
The eligibility rule should be based on the quantity being described, not on whether a record had an event. If the rate concerns events per person-year among all eligible participants, filtering the data to event-positive participants before summing person-years changes the denominator and no longer estimates that rate.
Worked Example: Keeping Zero-Event Records in the Exposure Total
Consider hypothetical follow-up data for five eligible participants. Event counts and exposure are observed for all five, so the rate calculation uses the same five records in both totals.
| Record | Weight | Events | Exposure (person-years) |
|---|---|---|---|
| A | 2 | 1 | 2 |
| B | 3 | 0 | 1 |
| C | 1 | 2 | 4 |
| D | 4 | 1 | 3 |
| E | 5 | 0 | 5 |
The weighted event total is \(2(1)+3(0)+1(2)+4(1)+5(0)=8\) events. The weighted exposure total is \(2(2)+3(1)+1(4)+4(3)+5(5)=48\) person-years. Thus,
Records B and E have no events, but contribute \(3(1)+5(5)=28\) weighted person-years to the denominator. If an analyst first filtered to participants with at least one event, the denominator would be \(4+4+12=20\) weighted person-years, while the event total would remain 8. The resulting \(8/20=0.4000\) events per person-year describes a filtered group and is not the rate among all five eligible participants.
Audit Record Membership, Not Just Arithmetic
A practical denominator audit can be done by listing the records that should enter each total and comparing that list with the records actually used. For each estimate, write down the target group, eligibility rule, outcome-observation rule, and any exposure-observation rule. Then verify that the numerator and denominator agree with those definitions.
Describe the population or subgroup and the quantity being summarized, including the outcome or exposure units.
Write an explicit rule for \(A_i=1\). Check that it is about membership in the target group, not whether the outcome happened or has a convenient value.
Identify which eligible records have the outcome, event count, and exposure information required for the stated calculation.
List or count the records and weights entering the numerator and denominator. For a mean or proportion, these should be the same eligible observed records; for a rate, event and exposure records must correspond to the defined rate population.
Recalculate after correcting suspected filters and report which records or weights changed. Interpret each resulting ratio according to its actual denominator.
The basic arithmetic explains why extra denominator weight can materially alter an estimate. Suppose a correct weighted mean is \(N/W\), with weighted outcome total \(N\) and eligible observed weight total \(W\). If an analyst adds records with total weight \(q>0\) to the denominator but adds no corresponding outcome total, the displayed value becomes \(N/(W+q)\). For positive \(N\), this is smaller than \(N/W\), since
This algebra does not make the altered ratio a valid estimate; it shows that changing denominator membership changes the result, even if the numerator is untouched. Conversely, dropping eligible records from both totals changes the ratio according to the outcomes and weights of the dropped records. The direction of that change cannot be known without examining them.
Worked Example: An Accidental Age Cutoff Changes the Mean
Suppose a hypothetical report summarizes a measurement \(y\), in units, among adults aged 65 years or older. The correct eligibility rule is age \(\geq 65\). The records are:
| Age (years) | Weight | Measurement (units) | Eligible under age \(\geq 65\)? |
|---|---|---|---|
| 64 | 2 | 10 | No |
| 65 | 3 | 20 | Yes |
| 66 | 1 | 40 | Yes |
| 70 | 4 | 30 | Yes |
| 65 | 2 | 50 | Yes |
The correct weighted numerator for eligible adults is \(3(20)+1(40)+4(30)+2(50)=320\) weighted units, and the denominator is \(3+1+4+2=10\). The correct mean is \(320/10=32.0\) units.
Now suppose a filter is mistakenly coded as age \(>65\), excluding both records aged 65. The resulting numerator is \(1(40)+4(30)=160\), and the denominator is \(1+4=5\). That calculation gives \(160/5=32.0\) in this particular data set, so the error happens not to change the estimate. This does not make the filter correct: it still excludes eligible records and changes the represented weighted population.
To see why the estimate can change, consider the same records with the 65-year-old’s measurement of 50 replaced by 80 units, with all weights and eligibility rules unchanged. The correct numerator becomes \(3(20)+1(40)+4(30)+2(80)=380\), giving \(380/10=38.0\) units. The erroneous \(>65\) calculation remains \(160/5=32.0\) units. The mistaken cutoff has excluded eligible weight of \(3+2=5\) and its weighted outcomes, producing a different summary. The first dataset’s equality was coincidental, not evidence that the denominator was audited successfully.
Common Denominator Pitfalls
- Counting all eligible weight despite missing outcomes. For a complete-record mean or proportion, include only the eligible records with observed outcomes in both totals. Do not treat missing values as zero. Also do not claim that the observed-record estimate represents all eligible records without an appropriate basis.
- Using different filters for numerator and denominator. A weighted proportion requires the event and non-event records in the same eligible observed group. A rate requires a denominator that matches the event-count target, with the corresponding exposure included.
- Filtering on the outcome itself. Keeping only event-positive records before calculating an event proportion or a population rate excludes relevant non-events or exposure and changes the quantity.
- Assuming a small or zero numerical change means no error occurred. An incorrect rule can happen to yield the same ratio in a particular dataset. Eligibility is a definition of the target, not a choice made by checking whether the answer changes.
- Confusing eligibility with response. A missing outcome does not erase a record’s eligibility. Keep those two conditions separate when describing the target and the observed-data calculation.
Check Your Understanding
For each question, identify the records that belong in the relevant denominator and explain what the resulting estimate would describe.
- A weighted mean is intended for all eligible records, but one eligible record has a missing outcome. What does a complete-record weighted mean directly describe, and why should the missing record’s weight not simply be added to its denominator?
- In a weighted binary-outcome proportion, should eligible records with an observed outcome of zero enter the denominator? Explain.
- A rate is defined as events per person-year among all eligible participants. Why must participants with observed exposure but zero events remain in the exposure denominator?
- An age rule defines eligibility as age \(\geq 65\), but a data filter uses age \(>65\). What audit should be performed even if the estimate is numerically unchanged?
- Give one reason why an estimate based on observed outcomes might not represent the full eligible target group.