Binomial Counts and Trial-Count Log Partitions

Why the Trial Count Appears in the Canonical Form

A binomial response records the number of successes among a specified number of trials. Its probability depends on both the observed success count and the number of opportunities for success. In exponential-family form, these features enter in different places: the observed count multiplies the natural parameter, while the fixed trial count scales the log-partition function. The binomial coefficient is a parameter-free part of the mass function and belongs in the base measure.

Keeping these roles separate is useful when writing likelihoods for binomial data, especially when different observations have different numbers of trials. The key is not to treat the trial count as another observed response: it is fixed for the particular binomial distribution.

Binomial Mass Function in Canonical Form

Definition: For a fixed positive integer $n$ and $0<p<1$, a random variable $Y$ has a binomial distribution with trial count $n$ and success probability $p$, written $Y\sim\operatorname{Binomial}(n,p)$, if its support is $y=0,1,\ldots,n$ and $$ P(Y=y)=\binom{n}{y}p^y(1-p)^{n-y}. $$ For this tutorial, $n$ is fixed and known. The canonical natural parameter is $\eta=\log\{p/(1-p)\}$.

Solving the natural-parameter equation for $p$ gives

p=\frac{e^\eta}{1+e^\eta}, \qquad 1-p=\frac{1}{1+e^\eta}.

Substituting these expressions into the binomial mass function isolates the part depending on the observed count $y$, the part depending on $n$ and $\eta$, and the parameter-free coefficient.

\begin{aligned} P(Y=y) &=\binom{n}{y} \left(\frac{e^\eta}{1+e^\eta}\right)^y \left(\frac{1}{1+e^\eta}\right)^{n-y}\\ &=\binom{n}{y} \exp\left\{y\eta-n\log(1+e^\eta)\right\}. \end{aligned}

Canonical representation: For fixed $n$, the binomial mass function can be written $$ f(y;\eta)=h_n(y)\exp\{y\eta-A_n(\eta)\}, \qquad y\in\{0,\ldots,n\}, $$ where $$ h_n(y)=\binom{n}{y}, \qquad A_n(\eta)=n\log(1+e^\eta), \qquad \eta\in\mathbb{R}. $$ Thus the response statistic is $y$, the base measure contains the binomial coefficient, and the fixed trial count multiplies the log-partition function.

The natural-parameter space is all of $\mathbb{R}$: every finite real $\eta$ gives a probability $p$ strictly between zero and one, and the resulting mass function is normalized. The degenerate cases $p=0$ and $p=1$ are not represented by finite $\eta$; they correspond to limiting values $\eta\to-\infty$ and $\eta\to+\infty$, respectively.

Worked Example: One Success Among Three Trials

Let $Y\sim\operatorname{Binomial}(3,2/5)$. Write its canonical parameter and calculate $P(Y=1)$ in both the original and canonical forms.

The natural parameter is $$ \eta=\log\left(\frac{2/5}{3/5}\right)=\log(2/3). $$ The trial-count log-partition function at this parameter is $$ A_3(\eta)=3\log(1+e^\eta) =3\log(1+2/3)=3\log(5/3). $$ In the original form, $$ P(Y=1)=\binom{3}{1}\frac{2}{5}\left(\frac{3}{5}\right)^2 =3\cdot\frac{2}{5}\cdot\frac{9}{25} =\frac{54}{125}=0.432. $$ In canonical form, the same probability is $$ \binom{3}{1}\exp\{\eta-A_3(\eta)\} =3\left(\frac{2}{3}\right)\left(\frac{3}{5}\right)^3 =\frac{54}{125}=0.432. $$ Both calculations agree. The coefficient $3$ is the base-measure term, and the exponent includes one copy of $\eta$ because the observed count is one.

Why This Is a Valid Probability Model

The log-partition function is the term that makes the mass function sum to one over the support. For this binomial representation, that fact follows directly from the binomial theorem.

Theorem: For every fixed nonnegative integer $n$ and every $\eta\in\mathbb{R}$, $$ \sum_{y=0}^{n}\binom{n}{y} \exp\{y\eta-n\log(1+e^\eta)\}=1. $$ Consequently, $A_n(\eta)=n\log(1+e^\eta)$ is the log-partition function for the binomial count with fixed trial count $n$.

Proof. Since $\eta$ is real, $1+e^\eta>0$, so the logarithm is finite and the common factor $\exp\{-n\log(1+e^\eta)\}$ can be taken outside the sum. Therefore

\begin{aligned} \sum_{y=0}^{n}\binom{n}{y} \exp\{y\eta-n\log(1+e^\eta)\} &=\exp\{-n\log(1+e^\eta)\} \sum_{y=0}^{n}\binom{n}{y}(e^\eta)^y\\ &=\exp\{-n\log(1+e^\eta)\}(1+e^\eta)^n\\ &=(1+e^\eta)^{-n}(1+e^\eta)^n\\ &=1. \end{aligned}

The second equality is the binomial theorem applied to $1+e^\eta$. This verifies normalization for every finite $\eta$, not just for a particular probability value. $\square$

Worked Likelihoods

Worked Example: A Likelihood Contribution at Zero Natural Parameter

Suppose one observation is a binomial count $y=5$ from $n=10$ trials, and evaluate its likelihood contribution at $\eta=0$. First, $\eta=0$ implies $$ p=\frac{e^0}{1+e^0}=\frac{1}{2}, \qquad A_{10}(0)=10\log 2. $$

The likelihood contribution, considered as a function of $\eta$ for this observed $y$, is $$ L(\eta)=\binom{10}{5}\exp\{5\eta-10\log(1+e^\eta)\}. $$ Since $\binom{10}{5}=252$, substitution gives $$ L(0)=252\exp\{-10\log 2\} =\frac{252}{2^{10}} =\frac{252}{1024} =\frac{63}{256} \approx 0.2461. $$

At $\eta=0$, this likelihood contribution equals the probability of observing exactly five successes under $p=1/2$. The coefficient $252$ must remain in the likelihood: it is parameter-free, but it is part of the binomial mass.

Worked Example: A Count from Four Trials

Let $Y\sim\operatorname{Binomial}(4,1/4)$. Find the probability of two successes using the canonical representation, then check it directly.

The natural parameter and log-partition value are $$ \eta=\log\left(\frac{1/4}{3/4}\right)=\log(1/3)=-\log 3, \qquad A_4(\eta)=4\log(1+1/3)=4\log(4/3). $$ For $y=2$, the canonical calculation is $$ \begin{aligned} P(Y=2) &=\binom{4}{2}\exp\{2(-\log 3)-4\log(4/3)\}\\ &=6\left(\frac{1}{9}\right)\left(\frac{3}{4}\right)^4\\ &=\frac{6}{9}\cdot\frac{81}{256} =\frac{54}{256} =\frac{27}{128} \approx 0.2109. \end{aligned} $$ The direct calculation confirms the result: $$ \binom{4}{2}\left(\frac{1}{4}\right)^2 \left(\frac{3}{4}\right)^2 =6\cdot\frac{1}{16}\cdot\frac{9}{16} =\frac{54}{256} =\frac{27}{128}. $$

Different Trial Counts in a Likelihood

When several independent binomial observations share a success probability but have different known trial counts, each observation has its own binomial coefficient and its own trial-count multiplier in the log-partition term. For observations $y_i$ from trial counts $n_i$, the product likelihood is

L(\eta)=\prod_{i=1}^{m}\binom{n_i}{y_i} \exp\{y_i\eta-n_i\log(1+e^\eta)\}.

Equivalently, the log-likelihood is

\ell(\eta)= \sum_{i=1}^{m}\log\binom{n_i}{y_i} +\eta\sum_{i=1}^{m}y_i -\left(\sum_{i=1}^{m}n_i\right)\log(1+e^\eta).

This form makes the roles visible: the observed counts contribute to the coefficient of $\eta$, and the total number of trials scales the log-partition term. The binomial coefficients contribute to the likelihood, but they do not depend on $\eta$.

Worked Example: Two Independent Counts with Unequal Trial Counts

Suppose $Y_1\sim\operatorname{Binomial}(2,1/2)$ and $Y_2\sim\operatorname{Binomial}(3,1/2)$ are independent. Calculate the joint probability of $Y_1=1$ and $Y_2=2$, and express the joint likelihood contribution canonically.

Here $p=1/2$, so $\eta=0$. The two binomial coefficients are $\binom{2}{1}=2$ and $\binom{3}{2}=3$. Their canonical contributions multiply to $$ \begin{aligned} P(Y_1=1,Y_2=2) &=2\exp\{0-2\log 2\}\cdot 3\exp\{0-3\log 2\}\\ &=6\exp\{-5\log 2\} =\frac{6}{32} =\frac{3}{16}. \end{aligned} $$ The direct calculation gives the same answer: $$ \left[\binom{2}{1}\left(\frac12\right)^1 \left(\frac12\right)^1\right] \left[\binom{3}{2}\left(\frac12\right)^2 \left(\frac12\right)^1\right] =\frac{2}{4}\cdot\frac{3}{8} =\frac{3}{16}. $$

For a general common $\eta$, the joint likelihood contribution for these observed counts would be $$ 6\exp\{(1+2)\eta-(2+3)\log(1+e^\eta)\}. $$ The total observed count is $3$, while the total number of trials is $5$; these are distinct quantities and appear in different places.

Common Mistake and Practical Importance

A frequent error is to put the trial count into the response statistic or to use $\log(1+e^\eta)$ without multiplying it by $n$. The response statistic is the observed number of successes $y$, not the number of trials. The factor $n$ appears because each of the $n$ trial-level normalizing factors contributes a factor $1+e^\eta$.

Common pitfall: Writing $\binom{n}{y}\exp\{y\eta-\log(1+e^\eta)\}$ omits $n-1$ copies of the normalizing term and does not generally define a probability mass function. The correction is to use $A_n(\eta)=n\log(1+e^\eta)$. Indeed, summing the incorrect expression over $y$ gives $(1+e^\eta)^n/(1+e^\eta)$, which is not generally one.

The coefficient $\binom{n}{y}$ also has a distinct role. It counts how many trial-level success/failure patterns produce the same total count $y$. It is independent of $p$ and $\eta$, so it belongs in the base measure; nevertheless, it is part of the probability mass and should not be dropped when calculating a probability or a full likelihood.

Key takeaway: For a binomial count with fixed trial count $n$, the canonical parameter is the log-odds $\eta=\log\{p/(1-p)\}$, the base measure is $\binom{n}{y}$, and the log-partition function is $n\log(1+e^\eta)$. Keep the observed count and the trial count in their separate roles.

Check Your Understanding

Use the canonical representation to answer these questions.

For $Y\sim\operatorname{Binomial}(n,p)$, express $\eta$ in terms of $p$ and $p$ in terms of $\eta$.
Identify the base measure and log-partition function for a binomial count with fixed trial count $n$.
Why does the log-partition function contain a factor of $n$?
For $n=4$, $p=1/3$, and $y=1$, calculate the natural parameter and write the canonical mass at the observed count.
For independent binomial observations with trial counts $n_1,\ldots,n_m$ and a common natural parameter, what quantity multiplies $\log(1+e^\eta)$ in the log-likelihood?