Why Likelihood Contributions Matter
A generalized linear model specifies a conditional distribution for each response, given its predictors. The likelihood records how well a proposed set of model parameters accounts for the observed responses. To construct it correctly, work observation by observation: write the response’s probability mass or density, substitute the model’s parameterization, and then identify which terms vary with the parameters of interest.
This distinction is useful because a likelihood may contain factors that are constant with respect to regression coefficients but still belong to the exact likelihood. Some such factors also depend on a dispersion parameter. Whether a term can be omitted therefore depends on which parameters are being treated as variable and what comparison is being made.
Here \(\mathbf{x}_i\) denotes the fixed predictor information for observation \(i\). A model uses a linear predictor, commonly \(\eta_i=\mathbf{x}_i^{\mathsf T}\boldsymbol{\beta}\), and a link to connect that predictor to the conditional mean \(\mu_i=E[Y_i\mid\mathbf{x}_i]\). The response distribution then determines the contribution. This tutorial focuses on writing that contribution, not on finding parameter estimates.
The Exponential-Family Contribution
Assume each response has a distribution in the exponential family and that its observed value lies in the model’s support. Use the standard representation
The natural parameter is \(\theta_i\), the dispersion parameter is \(\phi\), and \(a\), \(b\), and \(c\) are functions determined by the response distribution. In this representation, the individual likelihood contribution is the expression on the right evaluated at the observed \(y_i\). The mean and variance, when the usual regularity conditions hold, are \(E[Y_i\mid\mathbf{x}_i]=b'(\theta_i)\) and \(\operatorname{Var}(Y_i\mid\mathbf{x}_i)=a(\phi)b''(\theta_i)\). The model links the mean to the predictors; a chosen link and response family determine how \(\eta_i\) corresponds to \(\theta_i\).
For independent observations, multiplying the contributions adds their exponents. Thus the log-likelihood is
The terms involving \(\theta_i\) generally vary with the regression parameters because the model connects \(\theta_i\) to the predictors. The term \(c(y_i,\phi)\) does not involve \(\boldsymbol{\beta}\) directly in this representation, but it can involve \(\phi\). Similarly, \(a(\phi)\) matters if \(\phi\) is being estimated. Therefore, “constant” must always mean constant with respect to a specified parameter, not constant in every possible model calculation.
Why Independent Contributions Multiply
The product form follows directly from the conditional independence assumption. Given the predictor information, if the joint conditional distribution factors into the individual response distributions, then the probability or density of the observed response vector is their product. Taking logarithms turns that product into a sum.
Assume \(Y_1,\ldots,Y_n\) are conditionally independent given their predictor information and the model parameters.
Conditional independence gives \(f(y_1,\ldots,y_n\mid\mathbf{x}_1,\ldots,\mathbf{x}_n)=\prod_{i=1}^n f(y_i\mid\mathbf{x}_i)\).
The resulting product, viewed as a function of the model parameters with the observed \(y_i\) fixed, is the likelihood.
Since \(\log\prod_i L_i=\sum_i\log L_i\), the log-likelihood is often easier to write and compare term by term.
This justification also identifies an important limit: if responses are dependent after conditioning on the predictors, their joint likelihood need not be the product of the individual contributions. The independence assumption must not be inferred merely from having several rows of data.
Worked Examples for Common Response Distributions
Worked Example: Two Poisson Contributions
Suppose two conditionally independent counts follow Poisson distributions with log links. Their linear predictors are \(\eta_1=\beta_0\) and \(\eta_2=\beta_0+\beta_1\), so \(Y_i\mid\mathbf{x}_i\sim\operatorname{Poisson}(\mu_i)\) with \(\log\mu_i=\eta_i\). The observed counts are \(y_1=2\) and \(y_2=0\). Write the likelihood as a function of \(\beta_0,\beta_1\), then evaluate it at \(\beta_0=0,\ \beta_1=\log 2\).
Method. For a Poisson response with mean \(\mu_i\), the mass at \(y_i\) is \(e^{-\mu_i}\mu_i^{y_i}/y_i!\). The log link gives \(\mu_1=e^{\beta_0}\) and \(\mu_2=e^{\beta_0+\beta_1}\). Conditional independence allows us to multiply the two masses.
The likelihood as a function of the coefficients is
At the specified values, \(\mu_1=e^0=1\) and \(\mu_2=e^{\log 2}=2\). The first contribution is \(e^{-1}1^2/2!=1/(2e)\); the second is \(e^{-2}2^0/0!=e^{-2}\). Therefore
The likelihood and log-likelihood values are consistent: exponentiating \(-3-\log 2\) gives \(e^{-3}/2\). The factor \(1/2\) comes from \(2!\); it does not depend on the coefficients, but it is part of the exact likelihood. The common mistake is to forget that factorial and report \(e^{-3}\), twice the correct value.
Worked Example: Bernoulli Contributions With a Logit Link
A hypothetical study records three binary outcomes, assumed conditionally independent given a predictor \(x\). Let \(Y_i\mid x_i\) be Bernoulli with success probability \(p_i\), with \(\log\{p_i/(1-p_i)\}=\beta_0+\beta_1x_i\). The data are \((x_1,y_1)=(0,1)\), \((x_2,y_2)=(1,0)\), and \((x_3,y_3)=(2,1)\). Write the log-likelihood and evaluate it at \(\beta_0=0,\ \beta_1=\log 3\).
Method. A Bernoulli observation has contribution \(p_i^{y_i}(1-p_i)^{1-y_i}\), where \(y_i\) is either 0 or 1. Under the logit link, if \(\eta_i=\beta_0+\beta_1x_i\), then \(p_i=e^{\eta_i}/(1+e^{\eta_i})\). Substitution gives the individual log contribution \(y_i\eta_i-\log(1+e^{\eta_i})\), so
At the stated coefficients, the three linear predictors are \(0,\log 3,\log 9\). The corresponding probabilities are \(1/2,\ 3/4,\ 9/10\). The observed outcomes are success, failure, success, so the three contributions are \(1/2,\ 1-3/4=1/4,\ 9/10\). Their product is
The check is direct: the log contributions are \(-\log 2\), \(-\log 4\), and \(\log 9-\log 10\); their sum is \(\log(9/80)\). A common error is to use \(p_i\) for every observation. A failure contributes \(1-p_i\), not \(p_i\), which is why the second factor is \(1/4\), not \(3/4\).
Worked Example: Gaussian Contributions and a Dispersion Parameter
Assume two independent responses satisfy \(Y_1\sim N(\beta,4)\) and \(Y_2\sim N(\beta+2,4)\), where the second argument of \(N(\mu,\sigma^2)\) is the variance. The observations are \(y_1=3,\ y_2=5\), and the variance \(4\) is known. Write the likelihood in terms of \(\beta\), identify the terms constant in \(\beta\), and evaluate at \(\beta=2\).
Method. A normal response with mean \(\mu_i\) and known variance \(\sigma^2\) has density \((2\pi\sigma^2)^{-1/2}\exp\{-(y_i-\mu_i)^2/(2\sigma^2)\}\). Conditional independence gives the product of the two densities. Here \(\sigma^2=4\), \(\mu_1=\beta\), and \(\mu_2=\beta+2\), so
The factor \(1/(8\pi)\) is the product of the two normalizing constants and does not depend on \(\beta\), since the variance is fixed. The squared residual terms do depend on \(\beta\). At \(\beta=2\), the fitted means are \(2\) and \(4\); the residuals are \(3-2=1\) and \(5-4=1\), giving a sum of squared residuals \(1^2+1^2=2\). Thus
This is a joint density value, not the probability that the continuous response vector equals exactly \((3,5)\); exact points have probability zero for a continuous distribution. The common mistake is to call \(1/(8\pi)\) constant without specifying the parameter. It is constant in \(\beta\) here, but if the variance were unknown, the normalizing factor would depend on that variance and could not be discarded when considering it.
When May a Term Be Dropped?
For an exact likelihood, retain every factor specified by the probability mass function or density. For a calculation concerned only with how the likelihood varies in \(\boldsymbol{\beta}\), a term independent of \(\boldsymbol{\beta}\) may be omitted if the other parameters are held fixed. This creates a likelihood proportional to the exact likelihood, not a new exact likelihood.
The distinction is particularly important in the exponential-family form. The response-only part \(c(y_i,\phi)\) can be omitted when working only with regression parameters and \(\phi\) is fixed. It cannot automatically be omitted when \(\phi\) is also under consideration. Likewise, factorials in Poisson contributions and normalizing constants in Gaussian densities may be constant in the regression coefficients but still matter to the full likelihood value. When comparing likelihoods, ensure that omitted terms are genuinely common to the comparison.
Check Your Understanding
For each problem, write the requested likelihood contribution or likelihood value and check which terms depend on the stated parameter.
- A Poisson count has mean \(\mu=2\), and the observed count is \(y=3\). Write its contribution, including the factorial, and give the log contribution. (Answer: \(L=4e^{-2}/3\approx0.1804\); \(\ell=\log(4/3)-2\approx-1.7123\).)
- A Bernoulli response has logit \(\eta=\log 4\) and observed value \(y=0\). Find its contribution and explain why it is not the success probability. (Answer: \(p=4/5\), so \(L=1-p=1/5=0.2000\).)
- A normal observation has mean \(4\), known variance \(9\), and observed value \(7\). Write its density contribution. (Answer: \(L=(3\sqrt{2\pi})^{-1}e^{-1/2}\approx0.0807\); this is a density, not a point probability.)
- Five independent Bernoulli trials are summarized by two successes. If their common success probability is \(p=0.3\), calculate the grouped binomial likelihood contribution, including the combinatorial factor. (Answer: \(L={5\choose2}(0.3)^2(0.7)^3=0.3087\).)
- For a Poisson observation \(y=0\) with log mean \(\eta=\beta\), write the exact contribution and identify its coefficient-dependent part. (Answer: \(L(\beta)=e^{-e^\beta}\); the full contribution depends on \(\beta\), and there is no additional factorial factor because \(0!=1\).)