Aggregating Counts Across Exposure Records

Why Exposure-Weighted Aggregation Matters

When records have different amounts of exposure, a count of 10 events over 100 person-years does not carry the same rate information as 10 events over 1,000 person-years. If the records are governed by one common event rate, the natural pooled estimate uses the total count divided by the total exposure. This tutorial derives that rule and clarifies both its assumptions and the information that pooling retains.

The key assumptions are that exposure is known, the records share one rate per exposure unit, and their counts are independent Poisson variables conditional on the exposures. The records must also represent distinct exposure contributions: counting the same person-time twice would not be legitimate aggregation. If these assumptions fail, adding the numbers is still possible arithmetically, but the pooled calculation need not estimate the intended rate under a valid common-rate model.

Definition: For record \(i\), let \(E_i\) be its known, nonnegative exposure and \(Y_i\) its count. A common-rate Poisson model assumes that, for every record with \(E_i>0\), \(Y_i\) is Poisson with mean \(\lambda E_i\), where \(\lambda\geq 0\) is the common rate per exposure unit. Conditional on the exposures, the counts are independent. A zero-exposure record has \(Y_i=0\) with probability one and contributes no information about \(\lambda\).

For example, if \(E_i\) is measured in person-years, then \(\lambda\) is an expected event count per person-year. The exposure units must agree across records, or be converted to a common unit before pooling. In the derivations below, we omit zero-exposure records and assume that at least one record has positive exposure. Define the total count and total exposure by \(Y_+=\sum_iY_i\) and \(E_+=\sum_iE_i\).

What Distribution Does the Total Count Have?

Under the common-rate model, the total count is itself Poisson, with mean equal to the rate multiplied by the total exposure. Independence is an essential condition for this distributional statement.

Theorem: Suppose \(Y_1,\ldots,Y_m\) are conditionally independent and \(Y_i\sim\operatorname{Poisson}(\lambda E_i)\), with fixed \(E_i>0\) and \(\lambda\geq0\). Then \(Y_+=\sum_{i=1}^mY_i\sim\operatorname{Poisson}(\lambda E_+)\), where \(E_+=\sum_{i=1}^m E_i\).

Proof. For \(\lambda>0\), take any nonnegative integer \(s\). Independence gives the joint probability of a particular set of counts as the product of their Poisson probabilities. Sum over every set of nonnegative counts whose total is \(s\):

\begin{aligned} P(Y_+=s) &=\sum_{\substack{y_1+\cdots+y_m=s\\y_i\geq0}} \prod_{i=1}^m e^{-\lambda E_i} \frac{(\lambda E_i)^{y_i}}{y_i!}\\ &=e^{-\lambda E_+}\lambda^s \sum_{\substack{y_1+\cdots+y_m=s\\y_i\geq0}} \prod_{i=1}^m\frac{E_i^{y_i}}{y_i!}\\ &=e^{-\lambda E_+}\lambda^s\frac{E_+^s}{s!}\\ &=e^{-\lambda E_+}\frac{(\lambda E_+)^s}{s!}. \end{aligned}

The third equality is the multinomial theorem: expanding \((E_1+\cdots+E_m)^s\) gives \(s!\) times the sum in the preceding line. The final expression is the Poisson probability with mean \(\lambda E_+\). If \(\lambda=0\), every \(Y_i\) equals zero with probability one, so the same conclusion holds for the degenerate Poisson distribution at zero. \(\square\)

This result justifies treating the pooled count as a Poisson count over the pooled exposure when the common-rate and independence assumptions hold. It does not say that any collection of counts becomes a valid common-rate model merely because its counts and exposures can be added.

The Pooled Maximum Likelihood Estimate

The same aggregation rule follows directly from the likelihood. For positive exposures, the joint likelihood of the records is

\begin{aligned} L(\lambda) &=\prod_{i=1}^m e^{-\lambda E_i} \frac{(\lambda E_i)^{Y_i}}{Y_i!}\\ &=e^{-\lambda E_+}\lambda^{Y_+} \prod_{i=1}^m\frac{E_i^{Y_i}}{Y_i!}, \qquad \lambda>0. \end{aligned}

The final product depends on the observed exposures and counts but not on \(\lambda\). Thus, as a function of the common rate, the likelihood depends on the records through \(Y_+\) and \(E_+\). This is why the pooled counts and exposures retain the likelihood information needed to estimate this one common rate.

Result: If \(E_+>0\) and \(Y_+>0\), the maximum likelihood estimate of the common rate is \(\hat{\lambda}=Y_+/E_+\). If \(Y_+=0\), the likelihood is maximized at the boundary \(\hat{\lambda}=0\).

To verify the result when \(Y_+>0\), discard the likelihood factor that does not involve \(\lambda\). The log-likelihood, up to an additive constant, is \(Y_+\log(\lambda)-\lambda E_+\). Its first and second derivatives are

\frac{d\ell}{d\lambda}=\frac{Y_+}{\lambda}-E_+, \qquad \frac{d^2\ell}{d\lambda^2}=-\frac{Y_+}{\lambda^2}<0.

Setting the first derivative to zero gives \(\lambda=Y_+/E_+\), and the negative second derivative shows this is the unique interior maximum. When \(Y_+=0\), the likelihood as a function of \(\lambda\geq0\) is proportional to \(e^{-\lambda E_+}\), which decreases as \(\lambda\) increases, so its maximum is at zero. This boundary case is consistent with the zero-count likelihood behavior considered in Poisson models.

Worked Example: Pooling Three Person-Time Records

Three clinics observe 4, 9, and 7 events over 200, 300, and 500 person-years, respectively. Under a common-rate model, find the pooled maximum likelihood estimate and compare it with the simple average of the three observed rates.

The total event count and total person-time are \(Y_+=4+9+7=20\) and \(E_+=200+300+500=1{,}000\) person-years. Therefore,

\hat{\lambda}_{\mathrm{pooled}} =\frac{Y_+}{E_+} =\frac{20}{1{,}000} =0.020\text{ events per person-year}.

The three row-specific rates are \(4/200=0.020\), \(9/300=0.030\), and \(7/500=0.014\) events per person-year. Their unweighted average is \((0.020+0.030+0.014)/3=0.064/3\approx0.02133\), which is not the pooled estimate. The pooled value instead weights these rates by their shares of total exposure:

\frac{200}{1{,}000}(0.020) +\frac{300}{1{,}000}(0.030) +\frac{500}{1{,}000}(0.014) =0.004+0.009+0.007 =0.020.

The third clinic supplies half of all the exposure, so its observed rate receives half of the weight. Counting each clinic equally would ignore this difference in exposure.

Why the Exposure Weights Appear

For each record with \(E_i>0\), define its observed rate as \(R_i=Y_i/E_i\). The pooled estimate can be written exactly as an exposure-weighted average:

\hat{\lambda}_{\mathrm{pooled}} =\frac{\sum_iY_i}{\sum_iE_i} =\sum_i\frac{E_i}{E_+}\frac{Y_i}{E_i} =\sum_i\frac{E_i}{E_+}R_i.

The weights \(E_i/E_+\) are nonnegative and sum to one. Records with larger exposure therefore contribute more to the pooled rate. The algebra also shows why row rates cannot be averaged this way when some exposure is zero: \(Y_i/E_i\) would be undefined. Under the model, such a record has count zero almost surely and can simply be left out of the rate calculation.

An unweighted average is not automatically biased under the common-rate model. In fact, \(E[R_i]=\lambda\) for each positive-exposure record, so their unweighted average is also unbiased for \(\lambda\). But it is generally not the maximum likelihood estimate, and it typically has larger variance. To see the variance difference, let \(m\) be the number of positive-exposure records. Independence and the Poisson variance give

\begin{aligned} \operatorname{Var}(\hat{\lambda}_{\mathrm{pooled}}) &=\frac{\operatorname{Var}(Y_+)}{E_+^2} =\frac{\lambda}{E_+},\\ \operatorname{Var}\left(\frac{1}{m}\sum_{i=1}^mR_i\right) &=\frac{1}{m^2}\sum_{i=1}^m\operatorname{Var}(R_i) =\frac{\lambda}{m^2}\sum_{i=1}^m\frac{1}{E_i}. \end{aligned}

For positive \(E_i\), the Cauchy–Schwarz inequality gives \((\sum_i E_i)(\sum_i1/E_i)\geq m^2\). Consequently, the unweighted average's variance is at least \(\lambda/E_+\), the variance of the pooled estimator. Equality holds when all positive exposures are equal. Thus even though both estimates are unbiased under the common-rate model, pooling is at least as precise in this comparison.

Worked Example: Different Observed Averages and Variances

Suppose two independent records have exposures \(E_1=1\) and \(E_2=9\), and a common rate \(\lambda=0.5\) per exposure unit. Consider observed counts \(Y_1=0\) and \(Y_2=9\). The row rates are \(0/1=0\) and \(9/9=1\), so their unweighted average is \((0+1)/2=0.5\). The pooled rate is

\hat{\lambda}_{\mathrm{pooled}} =\frac{0+9}{1+9} =\frac{9}{10} =0.9.

These estimates differ because the unweighted average gives the one-unit record the same influence as the nine-unit record, while the pooled estimate weights them in proportion to exposure. This observed difference does not contradict unbiasedness: under the stated common-rate model, each row rate has expectation \(0.5\), and the pooled rate also has expectation \(0.5\).

Their model-based variances further illustrate the role of exposure. The pooled count has mean and variance \(\lambda E_+=0.5(10)=5\), so the pooled rate has variance \(5/10^2=0.05\). The unweighted average has variance

\frac{0.5}{2^2}\left(\frac{1}{1}+\frac{1}{9}\right) =\frac{0.5}{4}\left(\frac{10}{9}\right) =\frac{5}{36} \approx0.1389.

The pooled estimate uses the full total exposure without treating a small-exposure row as equally informative as a large-exposure row.

What Pooling Can Conceal

The pooled estimate is appropriate as the common-rate maximum likelihood estimate only when a common rate is a reasonable model for the records. If instead record \(i\) has its own rate \(\lambda_i\), with \(E[Y_i]=E_i\lambda_i\), then the expected pooled count-to-exposure ratio is

E\left[\frac{\sum_iY_i}{E_+}\right] =\frac{\sum_iE_i\lambda_i}{E_+}.

This is an exposure-weighted average of the record-specific rates. By contrast, the expected unweighted average of row rates is \(m^{-1}\sum_i\lambda_i\), assuming all \(m\) exposures are positive. These are generally different targets. Pooling can therefore conceal rate variation across records; it does not establish that the common-rate assumption is true. Moreover, if the counts are dependent, the sum still has expectation equal to the sum of its expected counts, but it need not have the Poisson distribution used in the likelihood derivation.

Common mistake: “Each record is one row, so I should give every row rate equal weight.” This reasoning confuses equal numbers of records with equal amounts of exposure. The correction is to use \(\sum_iY_i/\sum_iE_i\) for the common-rate Poisson maximum likelihood estimate, equivalently weighting each row rate by \(E_i/E_+\). An unweighted average answers a different averaging question and is not the pooled likelihood estimate.

Before aggregating, check that the exposures use compatible units, the records do not double-count exposure, the positive-exposure records are plausibly governed by one rate, and conditional independence is defensible for the Poisson likelihood. If the purpose is specifically to estimate one common rate, the pooled count and exposure provide the direct estimate. If the records may have different rates, preserving their separate counts and exposures is important for investigating that possibility.

Check Your Understanding

Use the common-rate Poisson model and show the calculations that support your answers.

Two independent records have counts 3 and 8 and exposures 150 and 400 person-years. Find the pooled estimate and the unweighted average of the row rates.
State the assumptions needed for the sum of the counts to have a Poisson distribution with mean \(\lambda E_+\).
Explain why an unweighted average of row rates can be unbiased for a common rate but still differ from its maximum likelihood estimate.
Under record-specific rates \(\lambda_i\), what weighted average of those rates is the expected pooled count-to-exposure ratio?
A record has zero exposure. Under the stated model, what count can it contribute, and why is its row-specific rate not included in the weighted-average expression?

Aggregating Counts Across Exposure Records

What You'll Learn