Likelihood Inference for Negative-Binomial Regression

Likelihood Inference in a Negative-Binomial Regression

A negative-binomial regression describes counts whose variability exceeds that of a Poisson model while relating their mean to predictors. Once the likelihood is fitted, inference can address both the regression coefficients and the dispersion parameter. Wald, score, and likelihood-ratio procedures use the same likelihood but evaluate evidence in different ways: at the estimate, at the null model, or by comparing the two.

This tutorial uses the NB2 mean–size parameterization. Let $Y_i$ be independent counts, with mean $\mu_i$ and size $r=1/\alpha$, where $\alpha>0$ is the dispersion parameter. The regression model has $\log(\mu_i)=o_i+x_i^\mathsf{T}\beta$, where $o_i$ is a known offset, $x_i$ is a predictor vector, and $\beta$ is the coefficient vector. Thus $\operatorname{Var}(Y_i)=\mu_i+\alpha\mu_i^2$. Inference below assumes a correctly specified model, identifiable parameters, and an interior true parameter value with $\alpha>0$.

Definition: Under this parameterization, the probability mass function for observation $i$ is $$ P(Y_i=y_i)= \frac{\Gamma(y_i+1/\alpha)}{\Gamma(1/\alpha)\,y_i!} \left(\frac{1}{1+\alpha\mu_i}\right)^{1/\alpha} \left(\frac{\alpha\mu_i}{1+\alpha\mu_i}\right)^{y_i}, \qquad y_i=0,1,2,\ldots. $$ The log-likelihood is the sum of the log probabilities over the independent observations. Write $\theta=(\beta^\mathsf{T},\alpha)^\mathsf{T}$, and denote its maximum-likelihood estimate by $\hat\theta$.

The likelihood supplies both an estimate and a measure of local curvature. Let $J(\theta)=-\partial^2\ell(\theta)/\partial\theta\partial\theta^\mathsf{T}$ be the observed information matrix. Under standard regularity conditions, the covariance matrix of the maximum-likelihood estimate is estimated by $J(\hat\theta)^{-1}$; expected Fisher information can be used instead when specified. This matrix includes uncertainty in dispersion as well as in the regression coefficients.

Three Likelihood Procedures

Consider testing $H_0:C\beta=c$, where $C$ has rank $q$. The full model estimates all parameters without this restriction; the restricted model estimates parameters subject to it. For testing dispersion, a hypothesis such as $H_0:\alpha=\alpha_0$ can be handled in the same way when $\alpha_0>0$ is an interior value. These are asymptotic procedures: under the null and regularity conditions, their reference distribution is chi-squared with $q$ degrees of freedom.

Definition: The Wald statistic measures the distance between the unrestricted estimate and the null restriction in estimated covariance units. For $H_0:C\beta=c$, it is $$ W=(C\hat\beta-c)^\mathsf{T} \left\{C\widehat{\operatorname{Var}}(\hat\beta)C^\mathsf{T}\right\}^{-1} (C\hat\beta-c). $$ For one restriction, $W$ is the squared Wald $z$-statistic.

Wald inference is convenient because it uses the unrestricted fit and its estimated covariance matrix. But it relies on a local approximation around that fit. For a dispersion interval, a Wald interval directly on $\alpha$ can extend below zero, which is outside the parameter space. A common alternative is to form an interval for $\log(\alpha)$, then exponentiate its endpoints.

Definition: A score test evaluates the likelihood’s slope and curvature at the restricted fit. Partition the parameters as $(\delta,\lambda)$, where $\delta=0$ represents the null restriction and $\lambda$ contains the nuisance parameters. At the restricted maximum-likelihood estimate, let $U_\delta$ be the score for $\delta$, and let $I$ be the expected information, partitioned into $I_{\delta\delta}, I_{\delta\lambda}, I_{\lambda\lambda}$. The score statistic is $$ S=U_\delta^\mathsf{T} \left(I_{\delta\delta}-I_{\delta\lambda}I_{\lambda\lambda}^{-1}I_{\lambda\delta}\right)^{-1}U_\delta. $$ The matrix in parentheses is the information for $\delta$ after accounting for nuisance parameters.

The nuisance-parameter adjustment matters: the score for the tested coefficient alone does not generally have the right variance when other parameters are estimated. In a score test, those nuisance parameters are fitted under the null, not under the unrestricted alternative. The statistic is especially useful when the full model is costly to fit or when the null model is the natural starting point.

Definition: The likelihood-ratio statistic compares the maximized log-likelihoods of the unrestricted and restricted models: $$ \Lambda=2\{\ell(\hat\theta)-\ell(\tilde\theta)\}, $$ where $\tilde\theta$ is the restricted maximum-likelihood estimate. Under the null and regularity conditions, $\Lambda$ is asymptotically $\chi^2_q$.

All three methods target the same null hypothesis, but their finite-sample results need not agree. Wald uses the unrestricted estimate and its curvature; the score test uses the restricted fit; and the likelihood-ratio test measures the loss in maximized log-likelihood imposed by the restriction.

Why the Test Statistics Have Their Reference Distributions

The score statistic follows from the large-sample behavior of the score under the null. Suppose the model is regular, the null is interior, and the information matrix is nonsingular. At the restricted estimate, the nuisance score is zero. The remaining score for $\delta$, after accounting for nuisance estimation, has asymptotic mean zero and covariance equal to the efficient information $I_{\delta\delta}-I_{\delta\lambda}I_{\lambda\lambda}^{-1}I_{\lambda\delta}$. Standardizing a mean-zero multivariate normal vector by its covariance matrix gives a chi-squared statistic.

Account for nuisance estimation.
At the restricted maximum, the nuisance score is zero. The score for $\delta$ is adjusted for its information relationship with $\lambda$, giving efficient information $I_{\delta\delta}-I_{\delta\lambda}I_{\lambda\lambda}^{-1}I_{\lambda\delta}$.

Standardize the score.
Under the null, the adjusted score is asymptotically normal with mean zero and the efficient information as its covariance. Multiplying by the inverse square root of that covariance gives an asymptotic standard normal vector.

Take its squared length.
The sum of squares of $q$ asymptotically independent standard normal components has a $\chi^2_q$ distribution. This squared length is exactly the score statistic $S$.

The likelihood-ratio result can be seen from the same local likelihood approximation. Near the true parameter, the log-likelihood is approximately a quadratic function with curvature given by information. Maximizing this quadratic without the restriction and with the restriction gives a difference equal to the squared standardized displacement in the $q$ restricted directions. Under the null that displacement is asymptotically standard normal in each direction, so twice the log-likelihood difference is asymptotically $\chi^2_q$. The Wald statistic uses that same local quadratic approximation, but estimates the displacement from the unrestricted fit. Consequently, the three statistics are asymptotically equivalent under the null in regular, large samples, although they can differ in a particular data set.

Worked Examples

Worked Example: A Wald Interval for a Rate Ratio

Suppose an NB2 regression with log link has estimated exposure coefficient $\hat\beta_1=0.40$, with estimated standard error $0.16$. The coefficient compares the expected count rates for a one-unit increase in exposure, holding the other predictors and any offset structure fixed. Consider the two-sided test $H_0:\beta_1=0$.

The Wald statistic on the $z$ scale is $z=0.40/0.16=2.50$. Its two-sided normal-approximation $p$-value is approximately $0.0124$. The 95% Wald interval for the coefficient is $0.40\pm1.96(0.16)=0.40\pm0.3136=[0.0864,0.7136]$.

Exponentiating gives a rate ratio of $\exp(0.40)\approx1.492$, with interval $[\exp(0.0864),\exp(0.7136)]\approx[1.090,2.041]$. Thus the fitted rate is about 1.49 times as large per one-unit increase, and this curvature-based interval excludes a rate ratio of 1.

Worked Example: A Score Test at the Restricted Fit

In the same type of NB2 regression, suppose the restricted model sets $\beta_1=0$ and estimates the intercept, remaining coefficients, and dispersion under that restriction. At this restricted fit, suppose the score for $\beta_1$ is $U_{\beta_1}=2.10$ and its efficient information, after adjustment for nuisance parameters, is $1.44$. These are likelihood derivatives evaluated at the restricted fit.

For this single restriction, $S=U_{\beta_1}^2/1.44=2.10^2/1.44=4.41/1.44=3.0625$. The equivalent signed score statistic is $2.10/\sqrt{1.44}=2.10/1.20=1.75$. The two-sided normal-approximation $p$-value is about $0.080$, equivalently the upper-tail probability from $\chi^2_1$ at 3.0625.

This calculation uses no unrestricted estimate of $\beta_1$. It asks whether the likelihood is still increasing strongly in the tested direction when the null model is fitted, relative to the information available in that direction.

Worked Example: Likelihood-Ratio Test and a Profile Interval

Suppose the unrestricted NB2 fit has $\ell(\hat\theta)=-128.40$, while the fit with $\beta_1=0$ has $\ell(\tilde\theta)=-131.05$. For the one-coefficient null, the likelihood-ratio statistic is $2\{ -128.40-(-131.05)\}=2(2.65)=5.30$. Against $\chi^2_1$, this gives a $p$-value of approximately $0.021$.

A profile-likelihood interval for $\beta_1$ is found by fixing $\beta_1$ at trial values, maximizing over the other coefficients and $\alpha$ at each value, and retaining values for which $2\{\ell(\hat\theta)-\ell_{\mathrm{prof}}(\beta_1)\}\leq3.841$, the 95% one-degree-of-freedom cutoff. Suppose the full estimate is $0.40$, with maximized log-likelihood $-128.40$, and profile calculations give:

Fixed $\beta_1$	Profile log-likelihood	Twice the log-likelihood loss
0.09	-130.34	3.88
0.10	-130.25	3.70
0.71	-130.35	3.90
0.72	-130.28	3.76

The lower endpoint lies between 0.09 and 0.10, and the upper endpoint between 0.71 and 0.72. Linear interpolation of the displayed likelihood losses gives approximate endpoints $0.09+(3.88-3.841)(0.01)/(3.88-3.70)\approx0.092$ and $0.71+(3.90-3.841)(0.01)/(3.90-3.76)\approx0.714$. The profile interval is therefore about $[0.092,0.714]$. It is based on likelihood loss while re-estimating nuisance parameters, rather than assuming a symmetric quadratic shape around the estimate.

Dispersion Inference and Interval Choice

The same framework applies to $\alpha$. For example, if an unrestricted fit gives $\hat\alpha=0.35$ with standard error $0.14$, a direct 95% Wald interval is $0.35\pm1.96(0.14)=[0.076,0.624]$. Here both endpoints are positive, but that is not guaranteed in general. A Wald interval on $\log(\alpha)$, or a profile-likelihood interval formed by fixing $\alpha$ and re-optimizing $\beta$, respects the positive parameter space. A likelihood-ratio test of an interior value $\alpha_0>0$ compares the unrestricted likelihood with the likelihood maximized subject to $\alpha=\alpha_0$; a score test evaluates the dispersion score and its nuisance-adjusted information under that restriction.

Profile likelihood is often useful when the likelihood is noticeably asymmetric or the estimate is close to the edge of its allowable range. It does not automatically guarantee accurate coverage: the chi-squared cutoff is still an asymptotic approximation, and small samples or weak identification can make it unreliable. In particular, testing the Poisson limit involves a boundary issue rather than the ordinary interior setting assumed for the procedures here.

Common Mistakes

Using a Wald standard error as if it were an exact result. Wald inference is a local, large-sample approximation. Check whether its interval is plausible and compare with a profile interval when curvature or parameter constraints are consequential.
Evaluating a score test at the unrestricted fit. The score test is defined at the restricted fit, with nuisance parameters estimated under the null. Evaluating it elsewhere changes the test.
Ignoring nuisance parameters in the score variance. Use efficient information, or the corresponding nuisance-adjusted variance, rather than treating the tested score as isolated.
Confusing the likelihood-ratio statistic with the log-likelihood difference. The statistic is twice the unrestricted-minus-restricted maximized log-likelihood difference.
Applying an interior chi-squared reference distribution at a boundary. The standard result requires an interior null and regularity conditions; boundary tests need their own calibration.

Key takeaway: Wald, score, and likelihood-ratio inference are complementary views of the fitted negative-binomial likelihood. Wald uses unrestricted curvature, score uses the restricted fit, and likelihood ratio compares maximized fits. Profile likelihood gives an interval by repeating that comparison across candidate parameter values.

Check Your Understanding

Use the definitions and calculations above to answer these questions.

For a one-coefficient Wald test, what estimate and uncertainty are used, and what does a 95% interval excluding zero imply?
Why must the nuisance-adjusted information be used in a score test?
A full model has log-likelihood $-80$, and its restricted model has log-likelihood $-82$. What is the likelihood-ratio statistic?
How is a profile-likelihood interval for a regression coefficient constructed?
Why can a direct Wald interval for the NB2 dispersion parameter be unsuitable?