Boundary Groups in Beta-Binomial Fits

Boundary Counts Are Valid Observations

A grouped binary response can produce a count at either end of its possible range: a group may have no successes, or every trial may be a success. These outcomes can look exceptional in a data table, but neither is intrinsically a problem for a beta-binomial model. The important distinction is between a boundary observation, such as \(y_i=0\), and a boundary parameter value, such as a mean success probability of zero.

Let group \(i\) have \(n_i\geq 1\) trials and observed success count \(y_i\in\{0,\ldots,n_i\}\). In the mean-correlation parameterization, write the common mean as \(\mu\) and the intraclass correlation as \(\rho\), with \(0<\mu<1\) and \(0<\rho<1\). Define \(\alpha=\mu(1-\rho)/\rho\), \(\beta=(1-\mu)(1-\rho)/\rho\), and \(\kappa=\alpha+\beta=(1-\rho)/\rho\).

For independent groups, the likelihood is the product of their group probabilities. The formulas simplify at the two count boundaries because one rising factorial is \((a)_0=1\).

Calculating Endpoint Contributions

The rising-factorial forms are useful both for hand calculations and for evaluating the likelihood. For a zero group, the factors in the numerator start at \(\beta\); for an all-success group they start at \(\alpha\). The denominator starts at \(\kappa=\alpha+\beta\). These expressions retain the group size: a zero in a group of ten trials is not the same likelihood contribution as a zero in a group of two trials.

The same calculation can be made directly from the beta-function form of the beta-binomial probability: \(\binom{n_i}{y_i}B(y_i+\alpha,n_i-y_i+\beta)/B(\alpha,\beta)\). For \(y_i=0\), this becomes \(B(\alpha,n_i+\beta)/B(\alpha,\beta)\); for \(y_i=n_i\), it becomes \(B(n_i+\alpha,\beta)/B(\alpha,\beta)\). Using \(B(a,b)=\Gamma(a)\Gamma(b)/\Gamma(a+b)\) and \(\Gamma(a+r)/\Gamma(a)=(a)_r\) gives the endpoint formulas above.

Observed Boundaries Versus Parameter Boundaries

An observed endpoint count does not force the fitted mean to an endpoint. In the example, both an all-zero group and an all-success group have positive probability with \(\mu=0.4\). Conversely, a data set containing only zero counts can make the likelihood approach its maximum at the degenerate parameter limit \(\mu\downarrow0\). Those are different statements: one concerns the observed response, the other concerns where the likelihood is maximized.

The mean-correlation parameterization describes interior beta distributions for \(0<\mu<1\) and \(0<\rho<1\). At the edges, it is useful to speak precisely of limits. The next result gives the two dispersion limits for a fixed positive group size.

Proof. Set \(\kappa=(1-\rho)/\rho\), so that \(\alpha=\mu\kappa\) and \(\beta=(1-\mu)\kappa\). As \(\rho\downarrow0\), \(\kappa\to\infty\). For fixed \(r\), \((c\kappa)_r/\kappa^r=\prod_{j=0}^{r-1}(c+j/\kappa)\to c^r\) when \(c>0\). Consequently, for each \(y\in\{0,\ldots,n\}\), \[ \binom{n}{y}\frac{(\mu\kappa)_y((1-\mu)\kappa)_{n-y}}{(\kappa)_n} \longrightarrow \binom{n}{y}\mu^y(1-\mu)^{n-y}. \] This is the binomial probability at \(y\).

As \(\rho\uparrow1\), \(\kappa\downarrow0\). For \(n\geq1\), \[ \frac{(c\kappa)_n}{(\kappa)_n} =c\prod_{j=1}^{n-1}\frac{c\kappa+j}{\kappa+j} \longrightarrow c. \] Thus the zero probability, with \(c=1-\mu\), tends to \(1-\mu\), and the all-success probability, with \(c=\mu\), tends to \(\mu\). For an interior count \(1\leq y\leq n-1\), both numerator rising factorials contain a factor proportional to \(\kappa\), while the denominator contains only one such factor. The remaining factors have finite positive limits, so that probability tends to zero. When \(n=1\), there are no interior counts, and the endpoint limits still apply. \(\square\)

When the Likelihood Approaches a Degenerate Fit

For interior parameter values, a boundary count contributes a finite log-likelihood term. For example, a zero group contributes \(\log(\beta)_{n_i}-\log(\kappa)_{n_i}\), and an all-success group contributes \(\log(\alpha)_{n_i}-\log(\kappa)_{n_i}\). These terms should be included in the same way as contributions from interior counts. The fact that \(y_i\) is zero does not mean its likelihood contribution is zero or undefined.

A parameter boundary is different. If every observed group has zero successes, then for fixed \(0<\rho<1\), as \(\mu\downarrow0\), \(\beta=(1-\mu)\kappa\to\kappa\). For each group, \[ P(Y_i=0)=\frac{(\beta)_{n_i}}{(\kappa)_{n_i}}\longrightarrow1. \] Therefore the full likelihood for all-zero data tends to one. Since a likelihood is a product of probabilities and cannot exceed one, this is a supremum at the degenerate all-zero model. At the limiting value \(\mu=0\), the response is zero with probability one. The same reasoning, with \(\mu\uparrow1\), applies when every group has all successes.

If both zero and all-success groups are present, neither \(\mu=0\) nor \(\mu=1\) assigns positive probability to all the observed groups. However, when all groups are at count endpoints, the likelihood can still favor values of \(\rho\) near one: in that limit each group is almost always entirely zero or entirely successful. Whether the maximum is actually at that boundary depends on the complete data and the other parameters; endpoint counts alone are not a proof that it is.

Practical Checks for Boundary Groups

When fitting a beta-binomial model, calculate endpoint contributions with the group’s own trial count and use the same parameterization throughout. For numerical work, log rising factorials or log-gamma functions avoid forming very large products. The likelihood must also be interpreted in the model’s parameter space: if an optimizer returns a value close to a boundary, check whether it is approaching a genuine limiting model or merely reporting numerical difficulty.

A useful check is to compare the observed endpoint count with the probability at the fitted interior parameters, then separately examine the relevant parameter limits. For example, an observed zero group may have a modest probability under the fitted model and still be entirely compatible with it. The question of whether a parameter estimate is at a boundary requires examining the likelihood over the parameter space, not labeling an observation as degenerate.

Common Mistakes

• Calling a boundary count impossible. Under interior beta-binomial parameters, both \(P(Y_i=0)\) and \(P(Y_i=n_i)\) are positive for finite \(n_i\). Use the endpoint formulas rather than discarding the group.
• Equating a boundary observation with a boundary estimate. An observed zero is a data value; \(\mu=0\) is a parameter boundary. One does not follow automatically from the other.
• Dropping the group-size dependence. The endpoint probability is a product with \(n_i\) factors. Replacing it with \(1-\mu\) or \(\mu\) is justified only in the \(\rho\uparrow1\) limit, not for general interior \(\rho\).
• Using the wrong numerator at an endpoint. Zero successes use \((\beta)_{n_i}\); all successes use \((\alpha)_{n_i}\). Checking that their limiting probabilities are \(1-\mu\) and \(\mu\), respectively, is a useful diagnostic.