Uncertainty In Standardized Rates

Why Standardized Estimates Have Uncertainty

A standardized rate combines stratum-specific rates using weights chosen to represent a target population. Even when those weights are fixed, the rates are usually estimated from counts and exposure, so the resulting standardized rate is uncertain. A standardized ratio has an additional source of uncertainty when its expected-count denominator is itself calculated using estimated reference rates. Correct uncertainty calculations must identify which quantities are estimated and how their estimates vary together.

Write $\lambda_j$ for the rate in stratum $j$, and $w_j$ for its standard-population weight, with $w_j\geq 0$ and $\sum_{j=1}^J w_j=1$. The direct standardized rate is the weighted average of the stratum-specific rates. Hats denote estimates. The calculations below distinguish uncertainty in estimated rates from uncertainty in estimated weights.

Definition: For stratum-specific rate estimates $\hat{\lambda}_1,\ldots,\hat{\lambda}_J$ and standard weights $w_1,\ldots,w_J$, the directly standardized rate estimator is $$ \hat{D}=\sum_{j=1}^J w_j\hat{\lambda}_j. $$ If the weights are fixed, its variance is determined by the variance and covariance of the estimated rates. If the weights are also estimated, their uncertainty and covariance with the rates may contribute as well.

Direct Standardization With Fixed Weights

Suppose stratum $j$ has observed count $Y_j$ and exposure $T_j>0$. Under a Poisson rate model, $Y_j\sim\operatorname{Poisson}(T_j\lambda_j)$, and the rate estimate is $\hat{\lambda}_j=Y_j/T_j$. Its expectation is $\lambda_j$ and its variance is $\lambda_j/T_j$. If the stratum counts are independent and the standard weights are fixed, the variance of $\hat{D}$ follows by adding the variances of the weighted rate estimates.

Result: If the weights are fixed and the rate-estimate vector has covariance matrix $\Sigma$, then $$ \operatorname{Var}(\hat{D})=w^\mathsf{T}\Sigma w =\sum_{j=1}^J\sum_{k=1}^J w_jw_k\operatorname{Cov}(\hat{\lambda}_j,\hat{\lambda}_k). $$ For independent Poisson stratum counts, this reduces to $$ \operatorname{Var}(\hat{D})=\sum_{j=1}^J w_j^2\frac{\lambda_j}{T_j}. $$

To verify the general formula, express the estimator as the scalar product $w^\mathsf{T}\hat{\lambda}$. Its variance is $\operatorname{Var}(w^\mathsf{T}\hat{\lambda})=w^\mathsf{T}\operatorname{Cov}(\hat{\lambda})w=w^\mathsf{T}\Sigma w$. Expanding this matrix product gives the double sum. If the stratum estimates are independent, all terms with $j\ne k$ are zero. For a Poisson rate estimate, $\operatorname{Var}(Y_j/T_j)=\operatorname{Var}(Y_j)/T_j^2=T_j\lambda_j/T_j^2=\lambda_j/T_j$. Substituting gives the stated independent-strata expression.

In practice, the unknown $\lambda_j$ in the variance is replaced by $\hat{\lambda}_j$. This gives an estimated variance, not an exact known variance. A standard error is the square root of that estimated variance. A normal-approximation interval may be useful when the rate estimates are sufficiently well behaved, but it can be poor with sparse counts or near a boundary such as a rate of zero.

Worked Example: A Directly Standardized Rate

Two independent strata have counts $Y_1=20$ and $Y_2=18$, with exposures $T_1=1000$ and $T_2=600$ person-years. Use fixed standard weights $w_1=0.70$ and $w_2=0.30$. Calculate the standardized rate and its estimated standard error.

The stratum-specific estimates are $\hat{\lambda}_1=20/1000=0.020$ and $\hat{\lambda}_2=18/600=0.030$ events per person-year. Therefore,

\hat{D}=0.70(0.020)+0.30(0.030) =0.014+0.009=0.023

The plug-in variance is

\widehat{\operatorname{Var}}(\hat{D}) =(0.70)^2\frac{0.020}{1000} +(0.30)^2\frac{0.030}{600} =0.0000098+0.0000045=0.0000143.

Thus the estimated standard error is $\sqrt{0.0000143}\approx0.00378$ events per person-year. For illustration, a normal-approximation interval is $0.023\pm1.96(0.00378)$, or approximately $(0.0156,0.0304)$. This interval is an approximation; the variance calculation itself uses the Poisson model and the stated independence assumption.

When the Weights Are Estimated

Weights can be treated as fixed when they are known by design or defined as the target of inference—for example, when standard-population proportions are taken from a complete, external population and the analysis is conditional on that standard. The weights then specify which weighted rate is being estimated, rather than adding sampling uncertainty to the study’s rate estimates.

If the weights are estimated from a sample, their variability may matter. A first-order calculation treats the standardized rate as the function $g(w,\lambda)=w^\mathsf{T}\lambda$. Its gradient with respect to $w$ is $\lambda$, and its gradient with respect to $\lambda$ is $w$. The joint covariance of the estimated weights and rates must be used if both are random. In particular, simply adding a weight-variance term while omitting nonzero covariance terms need not be valid.

Result: Suppose $(\hat{w},\hat{\lambda})$ consistently estimates $(w,\lambda)$, and $\sqrt{n}\{(\hat{w},\hat{\lambda})-(w,\lambda)\}\xrightarrow{d}N(0,V)$ for a finite covariance matrix $V$. Then the delta method gives a first-order asymptotic variance for $\hat{w}^{\mathsf{T}}\hat{\lambda}$ equal to $$ \frac{1}{n} \begin{pmatrix}\lambda\\w\end{pmatrix}^{\mathsf{T}} V \begin{pmatrix}\lambda\\w\end{pmatrix}, $$ where the coordinate order in $V$ is $(w,\lambda)$. The corresponding plug-in calculation replaces $w,\lambda,V$ by consistent estimates.

This result follows by applying the multivariate delta method to the differentiable function $g(w,\lambda)=w^\mathsf{T}\lambda$, whose gradient in the stated coordinate order is $(\lambda^\mathsf{T},w^\mathsf{T})^\mathsf{T}$. The asymptotic covariance of a differentiable function of a consistent, asymptotically normal estimator is the gradient multiplied on both sides of the estimator’s asymptotic covariance matrix. The factor $1/n$ expresses the usual covariance scale when $V$ is the limiting covariance of the $\sqrt n$-scaled estimator.

Worked Example: Sampling Uncertainty in Standard Weights

Suppose a standard population is sampled, and the estimated proportions in two strata are $\hat{w}_1=120/200=0.60$ and $\hat{w}_2=80/200=0.40$. Treat the stratum rates $0.010$ and $0.040$ as known for this calculation. Find the standardized rate and the estimated variance due to sampling the weights.

The estimate is $\hat{D}=0.60(0.010)+0.40(0.040)=0.006+0.016=0.022$. For a two-category sample of size $n=200$, the estimated variance of the first proportion is $\hat{w}_1(1-\hat{w}_1)/n=0.60(0.40)/200=0.0012$; the second proportion is $1-\hat{w}_1$, so its variance is also $0.0012$, and their covariance is $-0.0012$.

Because the weights sum to one, the standardized rate can be written as $\hat{D}=0.040+\hat{w}_1(0.010-0.040)$. Therefore its estimated variance is $(0.010-0.040)^2(0.0012)=0.00000108$, and its estimated standard error is approximately $0.00104$. This calculation isolates weight uncertainty; if the rates were also estimated, their variances and any covariance with the weights would also be needed.

Delta-Method Uncertainty for a Standardized Ratio

An indirectly standardized ratio commonly has the form $\hat{R}=\hat{O}/\hat{E}$, where $\hat{O}$ is an observed count or count estimate and $\hat{E}$ is an estimated expected count. For $E>0$, the gradient of $g(O,E)=O/E$ is $(1/E,-O/E^2)$. That derivative alone is not enough to justify a variance approximation: the delta method is an asymptotic result requiring regularity conditions.

Result: Suppose $(\hat{O},\hat{E})$ consistently estimates $(O,E)$, with $E>0$, and $$ \sqrt{n}\begin{pmatrix}\hat{O}-O\\\hat{E}-E\end{pmatrix} \xrightarrow{d}N\left(0, \begin{pmatrix}V_{OO}&V_{OE}\\V_{OE}&V_{EE}\end{pmatrix}\right). $$ Then, for $\hat{R}=\hat{O}/\hat{E}$, the first-order asymptotic variance is $$ \operatorname{Var}(\hat{R})\ \approx\ \frac{1}{n}\left( \frac{V_{OO}}{E^2} +\frac{O^2V_{EE}}{E^4} -\frac{2O V_{OE}}{E^3} \right). $$

To derive the expression, differentiability of $g$ at $(O,E)$, with $E>0$, gives the first-order Taylor expansion $\hat{R}-R=g_O(\hat{O}-O)+g_E(\hat{E}-E)+r_n$, where $R=O/E$ and the remainder $r_n$ is of smaller order than the estimation error in probability under the stated asymptotic normality and consistency conditions. The gradient is $a=(1/E,-O/E^2)^\mathsf{T}$. The limiting variance is $a^\mathsf{T}Va$. Multiplying out yields the displayed expression. Replacing unknown quantities by consistent estimates gives a first-order plug-in variance estimate.

When $\hat{O}$ and $\hat{E}$ are independent, $V_{OE}=0$, so the covariance term vanishes. If they are not independent, omitting that term can overstate or understate uncertainty. The approximation also requires the denominator to remain away from zero asymptotically; a positive observed denominator alone does not establish that condition.

Worked Example: A Ratio With an Estimated Expected Count

An observed count is $O=48$, with a Poisson variance estimate of 48. The expected count uses study exposures $T_1=1000$, $T_2=500$, and independent reference rate estimates $0.020$ and $0.040$. Those reference rates were estimated from independent Poisson counts with exposures $S_1=S_2=10000$. Calculate the ratio and its first-order standard error, treating the observed count and reference data as independent.

The estimated expected count is $\hat{E}=1000(0.020)+500(0.040)=20+20=40$, so the standardized ratio is $\hat{R}=48/40=1.20$.

For independent Poisson reference counts, the estimated rate variances are $0.020/10000$ and $0.040/10000$. Since $\hat{E}=T_1\hat{\lambda}_1+T_2\hat{\lambda}_2$, its estimated variance is

\widehat{\operatorname{Var}}(\hat{E}) =1000^2\frac{0.020}{10000} +500^2\frac{0.040}{10000} =2+1=3.

The independent-numerator-and-denominator delta calculation is

\widehat{\operatorname{Var}}(\hat{R}) \approx \frac{48}{40^2} +\frac{48^2(3)}{40^4} =0.0300+0.0027=0.0327.

The estimated standard error is $\sqrt{0.0327}\approx0.181$. The first term reflects uncertainty in the observed count; the second reflects uncertainty in the estimated reference rates used to form the expected count. This is a first-order approximation under the stated independence and asymptotic conditions.

Common Mistake and Practical Interpretation

A common mistake is to claim that finite covariance by itself makes a delta-method variance valid. For example, let $\hat{O}=1$ and let $\hat{E}$ equal 1 or 2 with probability $1/2$ each. Then $E[\hat{E}]=1.5$, and $\operatorname{Var}(\hat{E})=0.25$. The ratio is 1 or 0.5 with equal probability, so its mean is 0.75 and its variance is $0.0625$. A derivative calculation at the means instead gives $0.25/1.5^4\approx0.04938$. Finite covariance does not make that approximation exact or guarantee its accuracy.

The correction is to present the delta-method variance as a first-order approximation under an appropriate asymptotic regime: the estimators must be consistent, have a suitable joint asymptotic normal limit, and the function must be differentiable at the limiting parameter value. For ratios, the limiting denominator must be positive. In applications, also state whether weights and reference rates are fixed or estimated, and whether the quantities entering the ratio are independent.

Treating weights as fixed is appropriate when inference is conditional on a known, prespecified standard population. If the standard weights come from a sample and the target includes their sampling variability, include that variability. For standardized ratios, uncertainty in estimated reference rates can matter even when the study’s standard weights are fixed. These distinctions determine what uncertainty an interval describes.

Key takeaway: With fixed weights, propagate rate uncertainty through the weighted sum using the full covariance matrix. For ratios, use the joint covariance of numerator and denominator and require the delta method’s asymptotic conditions; finite covariance alone is not enough.

Check Your Understanding

Use the assumptions stated in each question, and distinguish exact variance identities from first-order approximations.

For independent Poisson stratum counts and fixed standard weights, why does the variance of a directly standardized rate involve squared weights?
Two estimated stratum-specific rates are correlated. Which part of the fixed-weight variance formula accounts for that dependence?
A set of standard weights is estimated from a sample. What additional information is needed to propagate its uncertainty into the standardized rate?
For a standardized ratio, what covariance term appears in the delta-method variance when numerator and denominator are not independent?
Why does finite covariance of a numerator and denominator not, by itself, justify a delta-method variance approximation?