Boundary Fits in Saturated Grouped Models

Why Boundary Fits Arise

In grouped binomial data, each group contributes a number of successes and a number of trials. A saturated model assigns a separate success probability to every group, so it can match each group’s observed success proportion exactly. When that proportion is zero or one, the fitted probability is at the edge of the allowed probability range. This is a valid maximum-likelihood estimate on the probability scale, even though it cannot be represented by a finite logistic coefficient.

The distinction matters because a boundary estimate and an infinite logit are two descriptions of the same limiting fit, not evidence that the observed sample has made the underlying success probability known with certainty. The derivation below makes clear what is being maximized and in which parameter space.

The Saturated Grouped-Binomial Model

Suppose there are \(G\) groups. In group \(i\), let \(m_i\) be the number of trials and \(y_i\) the number of successes, where \(m_i\) is a positive integer and \(y_i\in\{0,1,\ldots,m_i\}\). Assume the group counts are independent and, conditional on their group probabilities, \(Y_i\sim\operatorname{Binomial}(m_i,p_i)\). A saturated grouped model gives each group its own parameter \(p_i\), with \(0\leq p_i\leq 1\).

Definition: The saturated grouped-binomial model has one unrestricted success probability \(p_i\in[0,1]\) for each observed group \(i\). Its likelihood is the product of the group-specific binomial likelihoods. “Saturated” here means that no two observed groups are required to share a probability.

The likelihood is

L(p_1,\ldots,p_G) = \prod_{i=1}^{G} \binom{m_i}{y_i}p_i^{y_i}(1-p_i)^{m_i-y_i}.

At an endpoint, a factor with exponent zero is interpreted as \(1\), including \(0^0\) in this likelihood expression. Thus, for example, the group likelihood at \(p_i=0,y_i=0\) is \(\binom{m_i}{0}=1\), as required by the binomial probability. Since each \(m_i>0\), every group contributes information about its own probability.

Deriving the Group-Specific Maximum

Because the likelihood is a product of terms that each depend on only one \(p_i\), maximizing it over the separate parameters reduces to maximizing each group’s likelihood. The binomial coefficient is constant with respect to \(p_i\), so it does not change the maximizing value.

Theorem: In the saturated grouped-binomial model, the maximum-likelihood estimate of the probability in group \(i\) is \(\hat p_i=y_i/m_i\). It is an interior estimate when \(0<y_i<m_i\), equals \(0\) when \(y_i=0\), and equals \(1\) when \(y_i=m_i\).

Proof. Fix a group and write \(m=m_i\), \(y=y_i\), and \(p=p_i\). Its likelihood is \(L_i(p)=\binom{m}{y}p^y(1-p)^{m-y}\). If \(0<y<m\), then for \(0<p<1\) the log-likelihood, up to the constant \(\log\binom{m}{y}\), is

\ell_i(p)=y\log p+(m-y)\log(1-p)+\text{constant}.

Its first and second derivatives are

\ell_i'(p)=\frac{y}{p}-\frac{m-y}{1-p} =\frac{y-mp}{p(1-p)}, \qquad \ell_i''(p)=-\frac{y}{p^2}-\frac{m-y}{(1-p)^2}<0.

The first derivative is zero at \(p=y/m\), and strict concavity makes this the unique maximum in the open interval. At either endpoint the likelihood is zero when \(0<y<m\), while it is positive at \(y/m\), so this is also the maximum over the closed interval \([0,1]\).

If \(y=0\), then \(L_i(p)=(1-p)^m\), which decreases from \(1\) at \(p=0\) to \(0\) at \(p=1\). Its unique maximum is at \(p=0=y/m\). If \(y=m\), then \(L_i(p)=p^m\), which increases from \(0\) to \(1\); its unique maximum is at \(p=1=y/m\). This proves the group-specific claim in every case. Since the full likelihood is a product of these separate group likelihoods, choosing each group’s maximizer maximizes the full likelihood. \(\square\)

Worked Estimates Across Groups

Worked Example: Interior and Boundary Estimates Together

Suppose three independent groups have success counts and trial totals \((y_1,m_1)=(3,8)\), \((y_2,m_2)=(0,6)\), and \((y_3,m_3)=(7,7)\). Find the saturated maximum-likelihood estimates.

For group 1, the observed proportion is \(3/8=0.375\), so \(\hat p_1=0.375\), an interior estimate. For group 2, \(\hat p_2=0/6=0\), a lower-boundary estimate. For group 3, \(\hat p_3=7/7=1\), an upper-boundary estimate.

Each estimate maximizes its own group likelihood. For example, group 1’s likelihood is proportional to \(p_1^3(1-p_1)^5\); its log-likelihood derivative vanishes at \(p_1=3/8\). Group 2’s likelihood is \((1-p_2)^6\), maximized at zero. Group 3’s is \(p_3^7\), maximized at one. The groups’ estimates are separate because this model imposes no equality constraint across groups.

A saturated estimate is determined by the observed proportion within its own group; trial totals affect the likelihood’s shape but not the location of that group’s maximum. The estimate reaches a boundary exactly when the observed count is at an endpoint of its allowable range.

Worked Example: Checking the Likelihood for an All-Success Group

A group has \(m=4\) trials and \(y=2\) successes. Its likelihood is \(L(p)=\binom{4}{2}p^2(1-p)^2=6p^2(1-p)^2\). The estimate is \(\hat p=2/4=1/2\). At this value, \(L(1/2)=6(1/4)(1/4)=3/8\). At either endpoint, \(L(0)=L(1)=0\). This confirms that the maximum is interior rather than at a boundary.

Now consider \(m=4,y=4\). The likelihood is \(L(p)=p^4\), so its maximum over \([0,1]\) is \(L(1)=1\), attained at \(\hat p=1\). No finite interior value of \(p\) attains that maximum. The difference follows directly from the observed counts, not from a different optimization rule.

Why the Logit Has No Finite Boundary Value

In a logistic representation, a probability is written as \(p=\exp(\eta)/(1+\exp(\eta))\), where \(\eta\) is a real-valued linear predictor. For every finite \(\eta\), this expression lies strictly between zero and one. Consequently, the probability-scale estimate \(0\) or \(1\) cannot correspond to a finite \(\eta\).

Result: For a single group with a logistic parameter \(\eta\in\mathbb R\), an all-failure or all-success group has a likelihood whose supremum is approached as \(\eta\) tends to negative or positive infinity, respectively. There is no finite maximizing \(\eta\).

Derivation. For a group with \(m>0\) trials and \(y\) successes, write \(p(\eta)=\exp(\eta)/(1+\exp(\eta))\). Apart from the constant binomial coefficient, its likelihood is

L(\eta)\ \propto\ p(\eta)^y\{1-p(\eta)\}^{m-y} =\frac{\exp(y\eta)}{\{1+\exp(\eta)\}^{m}}.

If \(y=0\), this is \(\{1+\exp(\eta)\}^{-m}\). It is strictly decreasing in \(\eta\), because its derivative is \(-m\exp(\eta)\{1+\exp(\eta)\}^{-m-1}<0\). Its supremum is \(1\), approached as \(\eta\to-\infty\), and no finite \(\eta\) attains it. If \(y=m\), the likelihood is \(\{\exp(\eta)/(1+\exp(\eta))\}^{m}=p(\eta)^m\). Since \(p(\eta)\) increases strictly toward \(1\) as \(\eta\to+\infty\), the likelihood increases toward \(1\) without attaining it at a finite \(\eta\). Thus the probability-scale maxima at zero and one correspond to limiting, not finite, logistic parameters. \(\square\)

Worked Example: Translating Boundary Probabilities to Logits

Consider a group with \(0\) successes in \(5\) trials. Its probability-scale estimate is \(\hat p=0\). For a finite logit \(\eta\), the likelihood is \(L(\eta)=\{1+\exp(\eta)\}^{-5}\). At \(\eta=0\), this is \(1/32\). At \(\eta=-\log(9)\), \(\exp(\eta)=1/9\), so \(L(\eta)=(10/9)^{-5}=(9/10)^5<1\). As \(\eta\to-\infty\), \(L(\eta)\to1\), its supremum.

For \(5\) successes in \(5\) trials, the estimate is \(\hat p=1\). At \(\eta=0\), the likelihood is \((1/2)^5=1/32\). As \(\eta\to+\infty\), \(p(\eta)^5\to1\), but remains less than \(1\) for every finite \(\eta\). The two endpoint fits therefore have no finite logit, although their fitted probabilities are well defined.

Why the Saturated Model Matters

The saturated model provides a reference point: it shows what each group’s likelihood can achieve when its probability is unconstrained by other groups. If a group has zero observed successes or all observed trials are successes, its maximum occurs at the corresponding endpoint. In a model that links groups through shared regression coefficients, the fitted probabilities need not equal every observed group proportion; whether finite coefficients exist is a separate issue.

A boundary estimate should not be read as proof that the true probability is exactly zero or exactly one. It describes the maximum of the likelihood for the observed data under the model. A small group can reach an endpoint after only a few trials, so the observed endpoint alone does not establish certainty about future outcomes or the underlying process.

Common Mistakes

Excluding zero and one from the probability parameter space. If the model allows \(p_i\in[0,1]\), an endpoint can be an attained maximum. Restricting to \(0<p_i<1\) changes the statement: the same likelihood may have only a supremum, not a maximum.
Reporting a finite logit for an endpoint estimate. The logit \(\log\{p/(1-p)\}\) is not finite at \(p=0\) or \(p=1\). The probability estimate is valid; its corresponding logistic parameter is a limit at negative or positive infinity.
Calling an endpoint estimate proof of certainty. The estimate maximizes the likelihood for the observed sample. It does not establish that the underlying probability is structurally fixed at zero or one.
Pooling groups without a model constraint. In the saturated model, each group has its own \(p_i\). Replacing all the group-specific proportions by a single overall proportion imposes a common-probability restriction and is not the saturated fit.

Key takeaway: In a saturated grouped-binomial model, \(\hat p_i=y_i/m_i\) for each group. Zero successes produce a probability estimate of zero; all successes produce an estimate of one. These are attained boundary maxima on the closed probability scale, but they have no finite logistic parameter.

Check Your Understanding

Use the group-specific likelihood and the distinction between probability and logit scales.

A group has \(4\) successes in \(10\) trials. What is its saturated maximum-likelihood estimate, and is it interior or on the boundary?
For \(0\) successes in \(6\) trials, write the likelihood as a function of \(p\) and identify its maximum over \([0,1]\).
Why does a probability estimate of one not correspond to a finite logistic linear predictor?
A saturated model has two groups with observed proportions \(0.2\) and \(0.8\). Why is assigning both groups the overall proportion not the saturated fit?
Explain why an estimate of zero from a group with no observed successes does not by itself prove that the underlying success probability is zero.