A Worked Poisson Regression With A Binary Predictor

Why a Binary Predictor Gives a Two-Group Model

A binary predictor divides observations into two groups. In a Poisson regression with a log link, the model assigns one conditional mean to the group coded 0 and another to the group coded 1. This makes the model especially transparent: the likelihood can be written using the observations, but its fitted means are determined by the count total and sample size in each group.

The coefficient for the binary predictor compares those two means on the logarithmic scale. Working through the likelihood shows both why this interpretation holds and when it can fail to give a finite estimate. Throughout, assume the responses are independent Poisson random variables and there is no exposure offset.

Definition: Let \(Y_i\) be the count response for observation \(i\), and let \(x_i\) be a predictor taking values 0 or 1. The binary-predictor Poisson regression model is

Y_i \mid x_i \sim \operatorname{Poisson}(\mu_i), \qquad \log(\mu_i)=\beta_0+\beta_1 x_i.

Here \(\mu_i=E[Y_i\mid x_i]\). Thus the group with \(x_i=0\) has conditional mean \(e^{\beta_0}\), and the group with \(x_i=1\) has conditional mean \(e^{\beta_0+\beta_1}\).

Write \(n_0\) and \(n_1\) for the numbers of observations in groups 0 and 1, and write \(S_0\) and \(S_1\) for their respective response totals. Both groups must be represented to estimate two separate group means. When each group total is positive, the model has a finite maximum-likelihood estimate. The calculations below derive that estimate rather than assuming the group averages are the answer.

The Likelihood and Its Group Structure

For a Poisson response with mean \(\mu_i\), the probability of observing \(y_i\) is \(e^{-\mu_i}\mu_i^{y_i}/y_i!\). Independence makes the likelihood the product of these contributions. Under the binary-predictor model, every observation in a given group has the same fitted mean. Define \(\lambda_0=e^{\beta_0}\) and \(\lambda_1=e^{\beta_0+\beta_1}\); these are positive group means.

L(\beta_0,\beta_1) = \prod_{i:x_i=0}\frac{e^{-\lambda_0}\lambda_0^{y_i}}{y_i!} \prod_{i:x_i=1}\frac{e^{-\lambda_1}\lambda_1^{y_i}}{y_i!} = \frac{e^{-n_0\lambda_0}\lambda_0^{S_0}}{\prod_{i:x_i=0}y_i!} \frac{e^{-n_1\lambda_1}\lambda_1^{S_1}}{\prod_{i:x_i=1}y_i!}.

The second equality collects terms within each group: there are \(n_g\) exponential factors \(e^{-\lambda_g}\), and the powers of \(\lambda_g\) add to the group total \(S_g\). The factorial terms depend on the observed counts but not on the model parameters. Consequently, maximizing the likelihood over the two group means separates into maximizing one function for group 0 and one for group 1.

Theorem: Suppose \(n_0>0\) and \(n_1>0\). If \(S_0>0\) and \(S_1>0\), the binary-predictor Poisson model has a unique finite maximum-likelihood estimate. Its fitted means and coefficients are

\hat{\lambda}_0=\frac{S_0}{n_0},\qquad \hat{\lambda}_1=\frac{S_1}{n_1},\qquad \hat{\beta}_0=\log\left(\frac{S_0}{n_0}\right),\qquad \hat{\beta}_1=\log\left(\frac{S_1/n_1}{S_0/n_0}\right).

Proof. Since both groups are observed, the transformation from coefficients to group means, \(\lambda_0=e^{\beta_0}\) and \(\lambda_1=e^{\beta_0+\beta_1}\), is a one-to-one correspondence between finite coefficient pairs and positive mean pairs. Its inverse is \(\beta_0=\log(\lambda_0)\) and \(\beta_1=\log(\lambda_1)-\log(\lambda_0)\).

Ignoring terms constant in the means, the log-likelihood is the sum over \(g=0,1\) of \(S_g\log(\lambda_g)-n_g\lambda_g\). For either group, its derivative with respect to its positive mean is \(S_g/\lambda_g-n_g\), which is zero only at \(\lambda_g=S_g/n_g\). Its second derivative is \(-S_g/\lambda_g^2<0\) when \(S_g>0\), so this stationary point is the unique maximum. Substituting these two means into the inverse transformation gives the stated coefficient estimates. \(\square\)

This result also identifies a boundary case. If \(S_g=0\), that group’s mean-dependent log-likelihood is \(-n_g\lambda_g\), which increases toward its supremum as \(\lambda_g\) decreases to zero. A finite coefficient cannot produce a mean of zero because the exponential of a finite number is positive. Thus a group with total zero does not have a finite fitted coefficient pair under this model.

Worked Examples

Worked Example: Fitting the Model to Individual Counts

Suppose five observations have \(x_i=0\) and counts \(1,0,2,1,1\), while four observations have \(x_i=1\) and counts \(2,3,1,2\). Find the likelihood, fitted group means, and coefficient estimates.

Step 1: Calculate the group sizes and totals. For group 0, \(n_0=5\) and \(S_0=1+0+2+1+1=5\). For group 1, \(n_1=4\) and \(S_1=2+3+1+2=8\).

Step 2: Write the likelihood. Using the group likelihood form,

L(\lambda_0,\lambda_1) = \frac{e^{-5\lambda_0}\lambda_0^5}{1!0!2!1!1!} \frac{e^{-4\lambda_1}\lambda_1^8}{2!3!1!2!} = \frac{e^{-5\lambda_0}\lambda_0^5}{2} \frac{e^{-4\lambda_1}\lambda_1^8}{24}.

The factorial products are \(1!0!2!1!1!=2\) and \(2!3!1!2!=24\). The group means are \(\hat{\lambda}_0=5/5=1\) and \(\hat{\lambda}_1=8/4=2\). Both totals are positive, so the finite-estimate result applies.

Step 3: Convert the fitted means to coefficients. The intercept is \(\hat{\beta}_0=\log(1)=0\). The binary-predictor coefficient is \(\hat{\beta}_1=\log(2)-\log(1)=\log(2)\), approximately \(0.6931\). Checking the model predictions gives \(e^{\hat{\beta}_0}=1\) for group 0 and \(e^{\hat{\beta}_0+\hat{\beta}_1}=e^{\log 2}=2\) for group 1.

At the maximum, the likelihood is \(L(1,2)=e^{-5}/2 \times 2^8e^{-8}/24=(16/3)e^{-13}\). Equivalently, the maximized log-likelihood is \(\log(16/3)-13\). The two expressions agree because \(\log((16/3)e^{-13})=\log(16/3)-13\).

Worked Example: A Lower Mean in the Group Coded One

A study records \(n_0=10\) observations with total count \(S_0=12\) and \(n_1=8\) observations with total count \(S_1=6\). Calculate the fitted means and coefficients, then check the predictions.

The fitted group means are the group totals divided by their sizes:

\hat{\lambda}_0=\frac{12}{10}=1.2,\qquad \hat{\lambda}_1=\frac{6}{8}=0.75.

The coefficient estimates are \(\hat{\beta}_0=\log(1.2)\), approximately \(0.1823\), and

\hat{\beta}_1 = \log\left(\frac{0.75}{1.2}\right) = \log(0.625) \approx -0.4700.

The fitted mean for group 0 is \(e^{0.1823}\), approximately \(1.2\). For group 1, the linear predictor is \(\log(1.2)+\log(0.625)=\log(1.2\times0.625)=\log(0.75)\), so its fitted mean is \(0.75\). The negative coefficient is consistent with the group 1 mean being smaller than the group 0 mean.

Worked Example: A Group With No Observed Counts

Suppose group 0 contains counts \(0,0,1\), and group 1 contains counts \(0,0,0\). Find the likelihood and determine whether the usual finite coefficient estimates exist.

Here \(n_0=3\), \(S_0=1\), \(n_1=3\), and \(S_1=0\). The likelihood is

L(\lambda_0,\lambda_1) = \left(\frac{e^{-3\lambda_0}\lambda_0}{1!}\right) \left(\frac{e^{-3\lambda_1}}{0!0!0!}\right) = e^{-3\lambda_0}\lambda_0 e^{-3\lambda_1}.

For group 0, the likelihood-dependent log term is \(\log(\lambda_0)-3\lambda_0\), which is maximized at \(\lambda_0=1/3\). For group 1, the term is \(-3\lambda_1\). It increases as \(\lambda_1\) approaches zero, so its supremum occurs at the boundary \(\lambda_1=0\), not at a positive mean. Since \(\lambda_1=e^{\beta_0+\beta_1}\) is positive for every finite pair of coefficients, there is no finite maximum-likelihood estimate for \((\beta_0,\beta_1)\).

Along a sequence with \(\lambda_0=1/3\) and \(\lambda_1\) decreasing toward zero, the likelihood approaches \((1/3)e^{-1}\). The group 1 mean tends to zero relative to the positive group 0 mean, so the coefficient \(\log(\lambda_1/\lambda_0)\) decreases without bound. Reporting a finite log-ratio estimate in this example would not match the likelihood.

What the Binary Coefficient Compares

Key result: When both fitted group means are positive, \(e^{\hat{\beta}_1}=\hat{\lambda}_1/\hat{\lambda}_0\). The exponentiated coefficient is the fitted conditional mean for \(x=1\) divided by the fitted conditional mean for \(x=0\); the coefficient itself is the logarithm of that ratio.

The relationship follows directly from the model: \(\mu(0)=e^{\beta_0}\) and \(\mu(1)=e^{\beta_0+\beta_1}\). Dividing gives \(\mu(1)/\mu(0)=e^{\beta_1}\); taking logarithms gives \(\beta_1=\log(\mu(1)/\mu(0))\). At the maximum-likelihood estimates, the same identity holds for the fitted means. In a two-group model, the ratio is specifically the group 1 sample mean divided by the group 0 sample mean.

This is a comparison of model-based conditional means for the two predictor values. It does not, by itself, establish that changing the predictor would cause a change in a response. The group coding also matters: reversing which group is coded 0 changes the reference mean and reverses the direction of the log-ratio.

Common Mistake and Practical Checks

A common mistake is to treat \(\hat{\beta}_1\) itself as the ratio of the group means. It is not: it is the logarithm of the ratio. For example, in the first worked example the group means are 2 and 1, so the ratio is 2, while the coefficient is \(\log(2)\), approximately \(0.6931\). Exponentiating the coefficient recovers the ratio: \(e^{0.6931}\) is approximately 2.

Another important check is whether both groups have positive totals before applying the finite-estimate formulas. If a group total is zero, its sample mean is zero, and taking the logarithm of that mean is not a finite operation. In that case the likelihood approaches its supremum at a boundary mean; a finite coefficient estimate does not exist. With positive totals, verify the fit in three steps:

Compute each group mean as its response total divided by its group size.
Set the intercept equal to the log of the mean in the group coded 0.
Set the binary coefficient equal to the log of the group 1 mean divided by the group 0 mean, and check both fitted means by exponentiating the linear predictors.

These calculations matter because they connect the regression parameterization to the actual group summaries. They also provide a direct way to detect a coding error or an incorrectly reversed ratio without relying only on the sign of a coefficient.

Check Your Understanding

Use the model and coding defined above to answer the following questions.

For group totals \(S_0=9\), \(n_0=6\), \(S_1=12\), and \(n_1=8\), calculate the fitted means, intercept, and binary-predictor coefficient.
Why can the likelihood be factored into one mean-dependent contribution for each group?
If both group totals are positive, what does \(e^{\hat{\beta}_1}\) equal in terms of the fitted group means?
What happens to the likelihood-based estimate when one group has a positive sample size but a total count of zero?
If the group labels are recoded so that the original group 1 becomes the reference group, how does the direction of the coefficient comparison change?