Negative Binomial Estimation With Fixed Dispersion

Introduction

Count data often vary more than a Poisson model allows. A negative binomial model accommodates this extra variation with a size, or shape, parameter. When that parameter is specified in advance, estimating the mean is direct: the likelihood score depends on the observations through their total count. The important work is to use a clear parameterization, form the likelihood for the observed data, and handle the zero-count boundary correctly.

This tutorial considers independent, complete observations from one negative binomial distribution with a common mean and a fixed size parameter. It does not cover censored or truncated counts, nor does it estimate the size parameter.

Definition: Let $X$ be a negative binomial count with known size parameter $r>0$ and unknown mean $\mu\geq 0$. For $\mu>0$, its probability mass function is $$ P(X=x)=\frac{\Gamma(x+r)}{\Gamma(r)x!} \left(\frac{r}{r+\mu}\right)^r \left(\frac{\mu}{r+\mu}\right)^x, \qquad x=0,1,2,\ldots . $$ Under this parameterization, $E[X]=\mu$ and $\operatorname{Var}(X)=\mu+\mu^2/r$. The value $r$ is held fixed during estimation. At $\mu=0$, the distribution is understood as the limiting distribution concentrated at zero.

Likelihood and the score equation

Suppose $X_1,\ldots,X_n$ are independent observations from this same model, and all observed counts are complete. Write $x_1,\ldots,x_n$ for the observed values and $T=\sum_{i=1}^{n}x_i$ for their total. Independence means the likelihood is the product of the individual probabilities. Because $r$ is known, terms that do not involve $\mu$ can be collected as constants.

L(\mu)= \prod_{i=1}^{n} \frac{\Gamma(x_i+r)}{\Gamma(r)x_i!} \left(\frac{r}{r+\mu}\right)^r \left(\frac{\mu}{r+\mu}\right)^{x_i} \propto \frac{\mu^T}{(r+\mu)^{nr+T}}, \qquad \mu>0.

Taking logarithms converts the product into a sum. The log-likelihood, including its terms that depend on $\mu$, is

\ell(\mu)=C+T\log\mu-(nr+T)\log(r+\mu),

where $C$ does not depend on $\mu$. Differentiating gives the score:

U(\mu)=\frac{d\ell}{d\mu} =\frac{T}{\mu}-\frac{nr+T}{r+\mu} =\frac{r(T-n\mu)}{\mu(r+\mu)}.

Theorem: For $n$ independent, complete observations from the negative binomial model above, with fixed $r>0$, the maximum likelihood estimate is $$ \widehat{\mu}=\frac{T}{n}=\frac{1}{n}\sum_{i=1}^{n}x_i. $$ If $T=0$, the maximum is at the boundary $\widehat{\mu}=0$. If $T>0$, it is an interior maximum.

Why the observed average maximizes the likelihood

For $\mu>0$, the denominator $\mu(r+\mu)$ in the score is positive, as is $r$. Therefore the sign of the score is exactly the sign of $T-n\mu$. Below $T/n$, the log-likelihood is increasing; above $T/n$, it is decreasing. This establishes a unique interior maximum when $T>0$, without needing to solve a more complicated equation.

The curvature also confirms the maximum. At $\mu=T/n>0$, the second derivative is negative:

\ell''(\widehat{\mu}) =-\frac{T}{\widehat{\mu}^{\,2}} +\frac{nr+T}{(r+\widehat{\mu})^2} =-\frac{nr}{\widehat{\mu}(r+\widehat{\mu})}<0.

When $T=0$, every observation is zero and the likelihood for $\mu>0$ is proportional to $(r/(r+\mu))^{nr}$, which decreases as $\mu$ increases. Its largest value is the limiting value at $\mu=0$. Thus the zero estimate is not obtained by substituting $T=0$ into an interior score equation that divides by $\mu$; it is obtained by checking the boundary of the parameter space.

Record the model.
Confirm that observations are independent complete counts with one common mean $\mu$, and that the size parameter $r$ is fixed.

Calculate the sufficient total.
Find $T=\sum x_i$ and the number of observations $n$. For a frequency table, multiply each count value by its frequency before summing.

Use the score and check the boundary.
If $T>0$, set the score to zero to obtain $\widehat{\mu}=T/n$. If $T=0$, maximize at $\widehat{\mu}=0$.

Interpret using the stated parameterization.
With this definition of $r$, the fitted variance is $\widehat{\mu}+\widehat{\mu}^{\,2}/r$; it is not generally equal to the fitted mean.

Worked calculations

Worked Example: Estimating the Mean From Individual Counts

A hypothetical insurer records the number of small equipment breakdown claims reported by each of five independent policyholders during a specified year. The complete counts are $0,1,2,3,4$. Assume each count follows the negative binomial model above with known $r=2$ and unknown common mean $\mu$. Estimate $\mu$, and verify the likelihood maximum. No interest-rate or mortality assumption enters this frequency model.

Method. Use the complete-data likelihood score with fixed $r$. First, $n=5$, and the observed total is $T=0+1+2+3+4=10$. Thus

\widehat{\mu}=\frac{T}{n}=\frac{10}{5}=2.

To check directly, the score for this sample is

U(\mu)=\frac{2(10-5\mu)}{\mu(2+\mu)}.

The score is positive for $0<\mu<2$, zero at $\mu=2$, and negative for $\mu>2$. The estimate is therefore the unique maximum. As an additional likelihood check, at $\mu=2$ each probability uses $r/(r+\mu)=1/2$ and $\mu/(r+\mu)=1/2$. The five probabilities, in observed order, are $1/4,1/4,3/16,1/8,5/64$. Their product is

L(2)=\frac14\cdot\frac14\cdot\frac{3}{16}\cdot\frac18\cdot\frac{5}{64} =\frac{15}{131072}\approx 0.0001144.

The fitted variance for one policyholder is $2+2^2/2=4$, which is greater than the fitted mean of 2. Common mistake avoided: the fixed size $r=2$ is not itself the estimated mean. The mean estimate comes from the score and is the sample average.

Worked Example: All Observed Counts Are Zero

In a hypothetical sample of three independent service contracts, the complete number of covered device failures is $0,0,0$. Assume the counts follow the stated negative binomial model with known $r=4$. Find the maximum likelihood estimate of the common mean. No interest-rate or mortality assumption is relevant.

Here $n=3$, $T=0$, and the likelihood as a function of $\mu$ is

L(\mu)=\left(\frac{4}{4+\mu}\right)^{12},\qquad \mu\geq 0.

For every $\mu>0$, $4/(4+\mu)<1$, so $L(\mu)<1$. At $\mu=0$, each count is zero with probability one and $L(0)=1$. Consequently, $\widehat{\mu}=0$. The score for positive $\mu$ is $U(\mu)=-12/(4+\mu)<0$, which also shows that the likelihood decreases as $\mu$ increases from the boundary.

Common mistake avoided: dividing the score equation by $\mu$ and treating $\mu=0$ as inadmissible would discard the actual maximum. The parameter space here includes the limiting zero-mean distribution, so the all-zero sample gives a boundary estimate.

Worked Example: Estimation From a Frequency Table

A hypothetical administrator groups eight independent, complete monthly counts of water-main incidents as follows: two months had 0 incidents, three had 1, two had 2, and one had 5. Assume a common negative binomial mean $\mu$ and known $r=3$. Estimate the mean and give the fitted variance. No interest-rate or mortality assumption enters this count model.

Method. A frequency table represents eight observations, not four. The number of observations is $n=2+3+2+1=8$, and the count total is

$$ T=2(0)+3(1)+2(2)+1(5)=0+3+4+5=12. $$

The likelihood score therefore gives

\widehat{\mu}=\frac{12}{8}=1.5.

Since $T>0$, this is an interior maximum; the score is positive below 1.5 and negative above 1.5. With $r=3$, the fitted variance is

\widehat{\operatorname{Var}}(X) =\widehat{\mu}+\frac{\widehat{\mu}^{\,2}}{r} =1.5+\frac{(1.5)^2}{3} =1.5+\frac{2.25}{3} =2.25.

Common mistake avoided: averaging the four distinct count values, or averaging only the positive counts, would not use the sample correctly. Each value must be weighted by how many observations it represents, including the zero-count months.

Parameterization and scope

The word “dispersion” is used with more than one convention. Here the fixed parameter is $r$, and the variance is $\mu+\mu^2/r$. Some texts instead use $\alpha=1/r$, giving variance $\mu+\alpha\mu^2$. If $\alpha$ is fixed, the same model and estimate result after setting $r=1/\alpha$. Always translate the stated variance formula before inserting a parameter into the probability mass function.

Another equivalent negative binomial parameterization describes the number of failures before $r$ successes, using success probability $p=r/(r+\mu)$. Under that convention, $p$ is not the mean: the mean is $r(1-p)/p$. If $r$ is fixed, estimating $\mu$ by the observed average is equivalent to estimating $p$ as $r/(r+\overline{x})$, but reporting $p$ as the mean would be incorrect.

The result depends on the stated data structure. If counts have different exposure periods and therefore different means, or if the observed counts are incomplete, the common-mean complete-data likelihood above is not the appropriate likelihood without modification. The estimate also does not say that the negative binomial model fits well; it only maximizes the likelihood within the assumed fixed-$r$ family.

Why the total is enough for estimating the mean

The log-likelihood expression makes the role of the data explicit: once $n$ and $T$ are known, all $\mu$-dependent terms are $T\log\mu-(nr+T)\log(r+\mu)$. The individual arrangement of the observed counts affects the likelihood through factors that do not involve $\mu$, so it cannot change the maximizing value. This is why a grouped frequency table can be handled by recovering its observation count and total.

The same conclusion can be checked against the model expectation: the sample average is a natural estimate because each observation has mean $\mu$. That observation alone is not the likelihood proof; the score sign and boundary check establish the maximum. This distinction is useful on exam problems, where a plausible estimator must still be justified from the specified likelihood.

Key takeaway: For independent complete counts with a common negative binomial mean and fixed size $r$, the maximum likelihood estimate is the sample average. The observed total determines the score; all-zero data give the boundary estimate zero. Verify the parameterization before interpreting the fixed dispersion or calculating a variance.

Check Your Understanding

For each problem, use the negative binomial parameterization defined above and assume independent, complete counts with a common mean.

With known $r=2$, the observed counts are $1,0,4,2,3$. Find the maximum likelihood estimate of $\mu$. (Answer: $\widehat{\mu}=2$)
With known $r=5$, four observed counts are $0,0,2,1$. Find the estimate of $\mu$. (Answer: $\widehat{\mu}=0.75$)
A frequency table has one observation equal to 0, four equal to 1, and two equal to 3. If $r=3$, find the estimate of $\mu$. (Answer: $\widehat{\mu}=10/7\approx1.4286$)
Six independent counts are all zero, and $r=1.5$ is fixed. State the maximum likelihood estimate and whether it is an interior or boundary estimate. (Answer: $\widehat{\mu}=0$, a boundary estimate)
For $n=10$ observations with total count $T=30$, the known size is $r=4$. Find the maximum likelihood estimate and the corresponding fitted variance. (Answer: $\widehat{\mu}=3$, fitted variance $=3+9/4=5.25$)

What You'll Learn