Tutorials › Exam PA - Predictive Analytics › Fitting Poisson GLMs with Offsets in R

Exam PA · Model Comparison, Offsets, and Weights · Tutorial 379 of 966

Fitting Poisson GLMs with Offsets in R

Fit a Poisson GLM with an exposure offset in R and use its linear predictor to obtain expected counts for different exposure amounts.

Advanced 11 min read

What You'll Learn

  • Specify a Poisson GLM with a log-exposure offset in R
  • Distinguish the offset from an estimated regression coefficient
  • Calculate an intercept-only rate estimate from counts and exposure
  • Interpret fitted linear predictors and expected counts
  • Use predict() with new exposure values and check common offset mistakes

Why an Exposure Offset Is Needed

A count is difficult to compare across observations when the observations cover different amounts of time, policy exposure, or population at risk. A count of four events over a short period may indicate a higher underlying rate than a count of six events over a much longer period. A Poisson rate model accounts for this by including exposure in the expected count, rather than treating every observed count as if it came from the same amount of risk.

This tutorial focuses on fitting that model in R and using its fitted values. The earlier material on exposure as a known rate multiplier and on deriving the log-exposure offset establishes why the exposure enters with coefficient fixed at one. Here the practical task is to specify that offset correctly, read the fitted linear predictor, and turn it into expected counts.

Definition: Let \(Y_i\) be the event count for observation \(i\), and let \(e_i>0\) be its known exposure in a consistent unit. A Poisson rate model assumes that, conditional on predictors, \(Y_i\) has a Poisson distribution with mean \(\mu_i\). With a log link and predictor vector \(x_i\), the model is $$ Y_i \mid x_i,e_i \sim \operatorname{Poisson}(\mu_i), \qquad \log(\mu_i)=\log(e_i)+x_i^\mathsf{T}\beta. $$ The term \(\log(e_i)\) is an offset: it is included in the linear predictor with its coefficient fixed at one, not estimated from the data.

The model implies \(\mu_i=e_i\exp(x_i^\mathsf{T}\beta)\). Thus, \(\exp(x_i^\mathsf{T}\beta)\) is the modeled event rate per unit of exposure, while \(\mu_i\) is the expected count over the actual exposure \(e_i\). These are different quantities. The Poisson assumption also entails \(\operatorname{Var}(Y_i\mid x_i,e_i)=\mu_i\); observations are ordinarily assumed conditionally independent. The offset specification is appropriate when exposure is known and proportional to the expected count. It does not estimate the exposure effect from the data.

Specifying the Model in R

In R, put the offset directly in the model formula using offset(log(exposure)). The response is the observed count, the predictors describe the rate, and exposure is a positive variable measured in a consistent unit. For example, if the data frame is named claims, a model with a categorical rating group is fitted as follows:

R code:
fit <- glm(count ~ rating_group + offset(log(exposure)),
    family = poisson(link = "log"), data = claims)

The formula tells R that the outcome is count and that rating_group contributes estimated coefficients. The offset is calculated row by row from exposure, but its coefficient is fixed at one. In contrast, writing exposure as an ordinary predictor would ask R to estimate a coefficient for it; that is a different model. A rate model requires the exposure column to be positive because its logarithm must be defined.

1
Check the response and exposure.
Confirm that the response is a count and that exposure represents the amount of risk associated with that count. Check that exposure is positive and recorded consistently.
2
Specify the rate predictors.
Include the variables whose association with the event rate is to be estimated. Do not add exposure as an ordinary predictor when the intended assumption is a proportional rate multiplier.
3
Fit and calculate predictions.
Use a Poisson family with a log link and the log-exposure offset. For a new observation, supply its exposure as well as the predictor values when requesting a prediction.

Worked Examples

Worked Example: Fitting Rates for Two Rating Groups

A hypothetical insurer records event counts and exposure for two rating groups. The data are: group A has counts 1 and 2 with exposures 2 and 4; group B has counts 3 and 6 with exposures 2 and 4. Fit a Poisson model with rating group and a log-exposure offset. Then find the expected count for a group A observation with exposure 3 and a group B observation with exposure 1.5.

Method. Fit the model with rating group as a factor. With this two-level factor model, each group has its own fitted rate. For a group with total count \(Y_+\) and total exposure \(e_+\), the rate estimate is \(Y_+/e_+\). This follows by writing the Poisson log-likelihood for the group, apart from terms that do not involve its rate \(\lambda\):

$$ \ell(\lambda)=Y_+\log(\lambda)-\lambda e_+ + \text{constant}, \qquad \frac{\partial \ell}{\partial \lambda}=\frac{Y_+}{\lambda}-e_+. $$

Setting the derivative equal to zero gives \(\widehat{\lambda}=Y_+/e_+\). The second derivative is \(-Y_+/\lambda^2<0\), so this stationary point maximizes the log-likelihood when \(Y_+>0\).

Step 1: Find each group’s fitted rate. For group A, total count is \(1+2=3\) and total exposure is \(2+4=6\), giving rate \(3/6=0.5\). For group B, total count is \(3+6=9\) and total exposure is \(2+4=6\), giving rate \(9/6=1.5\).

Step 2: Express the fitted rates through R’s coefficients. With A as the reference level, the fitted intercept is \(\log(0.5)=-0.693147\ldots\). The group B coefficient is the log rate ratio, \(\log(1.5/0.5)=\log(3)=1.098612\ldots\). In R, the specification is:

fit <- glm(count ~ rating_group + offset(log(exposure)),
    family = poisson(link = "log"), data = claims)

The fitted rate for B is obtained by adding the intercept and group coefficient: \(\log(0.5)+\log(3)=\log(1.5)\). The offset is then added for each observation; it is not part of either estimated coefficient.

Step 3: Predict counts at the requested exposures. For group A with exposure 3, the linear predictor is \(\log(3)+\log(0.5)=\log(1.5)=0.4055\) to four decimal places. Exponentiating gives expected count \(1.5\). For group B with exposure 1.5, the linear predictor is \(\log(1.5)+\log(1.5)=\log(2.25)=0.8109\) to four decimal places. Exponentiating gives expected count \(2.25\).

Check. Each expected count equals exposure times the fitted group rate: \(3(0.5)=1.5\) and \(1.5(1.5)=2.25\). The common mistake avoided is treating the fitted group rate itself as the expected count.

Worked Example: Fitting an Intercept-Only Rate in R

A hypothetical monitoring program records counts \(2,5,8\) for three observation periods, with exposures \(1,4,5\), respectively. Fit an intercept-only Poisson rate model with an offset. Find the model’s expected count for new observations with exposures \(0.5\) and \(2.5\).

Step 1: Calculate the fitted rate. The total count is \(2+5+8=15\); the total exposure is \(1+4+5=10\). The intercept-only fitted rate is therefore \(15/10=1.5\) events per exposure unit. Equivalently, the intercept is \(\log(1.5)=0.405465\ldots\).

Step 2: Fit the R model. The corresponding specification is:

fit0 <- glm(count ~ 1 + offset(log(exposure)),
    family = poisson(link = "log"), data = monitoring)

The term \(1\) requests an intercept but no other estimated predictors. For a new observation, R needs its exposure so it can add the offset. For example:

new_data <- data.frame(exposure = c(0.5, 2.5))
predict(fit0, newdata = new_data, type = "link")
predict(fit0, newdata = new_data, type = "response")

Step 3: Calculate and interpret the predictions. At exposure \(0.5\), the linear predictor is \(\log(0.5)+\log(1.5)=\log(0.75)=-0.2877\), to four decimal places. The expected count is \(\exp(-0.287682\ldots)=0.75\). At exposure \(2.5\), it is \(\log(2.5)+\log(1.5)=\log(3.75)=1.3218\), to four decimal places, and the expected count is \(3.75\).

Check. Multiplying each exposure by the fitted rate gives \(0.5(1.5)=0.75\) and \(2.5(1.5)=3.75\). Fractional expected counts are valid: an expectation need not be an integer, even though each observed Poisson count is an integer.

Worked Example: Reading Link-Scale and Response-Scale Predictions

A hypothetical equipment service dataset has two categories. Category standard has counts 2 and 4 at exposures 2 and 4; category priority has counts 3 and 9 at exposures 2 and 6. Fit a category model with a log-exposure offset. For a new priority observation with exposure 2.4, state the fitted linear predictor and expected count, and identify which R prediction scale gives each.

Step 1: Estimate the rates. Standard has total count \(2+4=6\) and total exposure \(2+4=6\), so its fitted rate is \(1\). Priority has total count \(3+9=12\) and total exposure \(2+6=8\), so its fitted rate is \(1.5\).

Step 2: Obtain the fitted coefficient representation. Taking standard as reference, the intercept is \(\log(1)=0\), and the priority coefficient is \(\log(1.5/1)=\log(1.5)=0.405465\ldots\). The model can be fit using:

fit_service <- glm(count ~ category + offset(log(exposure)),
    family = poisson(link = "log"), data = service)

Step 3: Calculate the new observation’s predictor. For priority and exposure \(2.4\), the predictor includes both the category contribution and the offset: \(\eta=\log(2.4)+\log(1.5)=\log(3.6)=1.280934\ldots\), or \(1.2809\) to four decimal places. Its expected count is \(\exp(\eta)=3.6\).

In R, `type = "link"` returns the linear predictor \(\eta\), including the offset for the supplied exposure. `type = "response"` returns \(\exp(\eta)\), the expected count. The new data must include both category and exposure; omitting exposure would not provide the offset needed for this prediction.

Check. The prediction agrees with exposure times the priority rate: \(2.4(1.5)=3.6\). The common mistake avoided is to report the link-scale value \(1.2809\) as a count; exponentiation is required to return to the response scale.

What the Fitted Linear Predictor Represents

For an observation with predictors \(x_i\) and exposure \(e_i\), the fitted linear predictor is \(\widehat{\eta}_i=\log(e_i)+x_i^\mathsf{T}\widehat{\beta}\). It is the logarithm of that observation’s fitted mean count. Applying the inverse log link gives \(\widehat{\mu}_i=\exp(\widehat{\eta}_i)\). Equivalently, the fitted rate is \(\exp(x_i^\mathsf{T}\widehat{\beta})\), and multiplying it by exposure gives the fitted mean count. These two routes must give the same result.

For R predictions, this distinction is controlled by the type argument. With type = "link", the result is on the linear-predictor scale; with type = "response", the result is on the mean-count scale. For a model containing an offset, include the exposure column in newdata so R can calculate the offset for each prediction. A predicted mean of, for example, 0.75 is not a claim that an individual observation will have a fractional event count; it is the model’s expected count under the specified predictors and exposure.

Common Mistakes and a Practical Check

A frequent error is to omit the offset and fit an ordinary intercept-only Poisson model to counts \(2,5,8\). That model estimates a common mean count of \((2+5+8)/3=5\), regardless of the exposures \(1,4,5\). The rate model instead estimates \(15/10=1.5\) events per exposure unit, so its fitted means at the observed exposures are \(1.5,6,7.5\). The second and third fitted means differ from the observed counts; the model is estimating a common rate, not forcing each observed count to equal its fitted mean.

A related error is to put exposure in the formula as an ordinary predictor, such as count ~ exposure. That estimates an exposure coefficient and does not impose the proportional relationship \(\mu_i=e_i\lambda_i\). When proportionality is the modeling assumption, use offset(log(exposure)). The offset’s coefficient is fixed at one; it is not an extra coefficient to interpret or test.

A useful arithmetic check after fitting is to confirm that each response-scale prediction equals its exposure multiplied by the fitted rate for its predictor values. If predictions for otherwise identical records do not scale proportionally with exposure under the specified model, check that the offset and prediction data have been supplied as intended. This check does not establish that the Poisson assumptions are adequate; it verifies the relationship specified by the model.

Key takeaway: In a Poisson rate GLM, R’s offset(log(exposure)) contributes the known log exposure to the linear predictor with its coefficient fixed at one. The linear predictor is the log expected count; exponentiating it gives the expected count, equal to exposure times the fitted rate.

Check Your Understanding

For each problem, identify the fitted rate or predictor as requested and check the expected count by multiplying the rate by exposure.

  1. An intercept-only Poisson rate model is fitted to total count 9 over total exposure 6. What is the fitted rate, and what expected count does the model assign to an observation with exposure 3? (Answer: fitted rate \(1.5\); expected count \(4.5\))
  2. A two-group offset model has group A total count 6 over exposure 6, and group B total count 12 over exposure 6. With A as the reference group, what are the intercept and group B coefficient? (Answer: intercept \(0\); group B coefficient \(\log(2)=0.6931\) to four decimal places)
  3. An intercept-only model estimates a rate of 1.5 per exposure unit. For an exposure of 0.8, what are the linear predictor and expected count? Give the predictor to four decimal places. (Answer: linear predictor \(\log(1.2)=0.1823\); expected count \(1.2\))
  4. A model has fitted rate 2 for a specified predictor profile. What are the linear predictor and expected count for exposure 1.5? (Answer: linear predictor \(\log(3)=1.0986\); expected count \(3\))
  5. A researcher fits an intercept-only Poisson model without an offset to counts 4 and 8, observed over exposures 2 and 6. What common mean count does that model estimate, and what rate does the offset model estimate? (Answer: no-offset mean count \(6\); offset-model rate \((4+8)/(2+6)=1.5\) per exposure unit)
Clinical Biostats