Bernoulli and Binomial Sufficient Statistics

Why a Success Count Can Summarize the Data

Suppose several trials share one unknown success probability. The observed order of successes and failures may be important for recording the data, but for learning about that common probability, the likelihood may depend on the observations only through how many successes occurred. This is a concrete use of the likelihood factorization criterion: a statistic is sufficient when the likelihood separates into a parameter-dependent function of that statistic and a factor that does not depend on the parameter.

The fixed design matters. For individual Bernoulli trials, the number of trials is fixed. For grouped binomial observations, each group has a known number of trials, and those counts may differ across groups. In both settings below, every trial has the same success probability \(p\). The central question is whether the likelihood retains any information about \(p\) beyond the total number of successes.

Definition: Let \(X\) have a joint probability mass function \(f_\theta(x)\) on a sample space that does not depend on \(\theta\). A statistic \(T(X)\) is sufficient for \(\theta\) if the joint mass function can be written as \(f_\theta(x)=g_\theta(T(x))h(x)\), where \(g_\theta\) may depend on \(\theta\) and \(h\) does not. This is the likelihood factorization criterion. For a finite sample space, it also means that the conditional distribution of the data given \(T(X)\) does not depend on \(\theta\).

Independent Bernoulli Trials

Let \(X_1,\ldots,X_n\) be independent Bernoulli random variables with common success probability \(p\), where \(0<p<1\). Thus each \(X_j\) is either 0 or 1, with \(P(X_j=1)=p\) and \(P(X_j=0)=1-p\). The observed data vector is \(x=(x_1,\ldots,x_n)\), and define the success count \(T(x)=\sum_{j=1}^n x_j\).

Theorem: For independent Bernoulli observations with common \(p\), the total number of successes \(T=\sum_{j=1}^n X_j\) is sufficient for \(p\).

Proof. By independence, the joint probability of any particular data vector \(x\in\{0,1\}^n\) is the product of its Bernoulli probabilities:

f_p(x)=\prod_{j=1}^{n}p^{x_j}(1-p)^{1-x_j} =p^{\sum_j x_j}(1-p)^{n-\sum_j x_j} =p^{T(x)}(1-p)^{n-T(x)}.

This has the factorized form \(g_p(T(x))h(x)\), with \(g_p(t)=p^t(1-p)^{n-t}\) and \(h(x)=1\). The factorization criterion therefore gives sufficiency. \(\square\)

The statistic retains the number of successes, not their positions. The likelihood for \(p\) is identical for any two data vectors with the same \(T\). That does not mean the original data vectors are identical; it means their relative positions contribute no additional information about the common \(p\) through this likelihood.

Worked Example: Five Bernoulli Observations

Suppose the five observed outcomes are \(x=(1,0,1,1,0)\). There are \(T=3\) successes and \(n-T=2\) failures. The joint likelihood is

$$ L(p;x)=p^3(1-p)^2. $$

For example, at \(p=0.4\), this likelihood is

L(0.4;x)=0.4^3(0.6)^2 =0.064\times 0.36 =0.02304.

The different vector \(x'=(0,1,1,0,1)\) also has three successes and two failures, so its likelihood at every \(p\) is the same: \(p^3(1-p)^2\), and at \(p=0.4\) it is \(0.02304\). The equality follows from the likelihood factorization, not from the two observed sequences being the same.

What the Conditional Distribution Shows

Sufficiency can also be understood by asking what remains random about the data after the success count is known. For a particular vector \(x\) with \(T(x)=t\), the probability of that vector is \(p^t(1-p)^{n-t}\). There are \(\binom{n}{t}\) binary vectors with exactly \(t\) successes, so

P_p(X=x\mid T=t) =\frac{p^t(1-p)^{n-t}}{\binom{n}{t}p^t(1-p)^{n-t}} =\frac{1}{\binom{n}{t}}, \qquad T(x)=t.

Thus, conditional on \(T=t\), every placement of the \(t\) successes among the \(n\) trial positions is equally likely. This conditional distribution does not involve \(p\), as sufficiency requires. The denominator is nonzero for \(0<p<1\); the formula is applied only when \(t\) is a possible value of \(T\).

Worked Example: Conditional Probability of an Order

Take \(n=5\) and suppose it is known that exactly \(T=2\) trials were successes. There are

\binom{5}{2}=\frac{5!}{2!3!}=10

possible placements of the two successes. For the particular vector \((1,0,0,1,0)\), the conditional probability given \(T=2\) is therefore \(1/10\). Equivalently, its unconditional probability is \(p^2(1-p)^3\), and \(P_p(T=2)=10p^2(1-p)^3\), so their ratio is \(1/10\). The value does not change with \(p\).

Grouped Binomial Observations

Now suppose there are \(G\) groups. Group \(i\) contains a fixed, known number \(m_i\) of trials, and \(Y_i\) records the number of successes in that group. Assume the groups are independent and every trial in every group has the same success probability \(p\). Then \(Y_i\sim\operatorname{Bin}(m_i,p)\), while the trial counts \(m_i\) may differ. Write \(M=\sum_{i=1}^{G}m_i\) for the total number of trials and \(S=\sum_{i=1}^{G}Y_i\) for the total number of successes.

Theorem: If the group trial counts \(m_1,\ldots,m_G\) are fixed and known, and the independent group counts satisfy \(Y_i\sim\operatorname{Bin}(m_i,p)\) for a common \(p\), then \(S=\sum_{i=1}^{G}Y_i\) is sufficient for \(p\).

Proof. For a possible count vector \(y=(y_1,\ldots,y_G)\), the joint mass function is the product of the binomial mass functions:

\begin{aligned} f_p(y) &=\prod_{i=1}^{G}\binom{m_i}{y_i}p^{y_i}(1-p)^{m_i-y_i}\\ &=\left\{\prod_{i=1}^{G}\binom{m_i}{y_i}\right\} p^{\sum_i y_i}(1-p)^{\sum_i(m_i-y_i)}\\ &=\left\{\prod_{i=1}^{G}\binom{m_i}{y_i}\right\} p^{S(y)}(1-p)^{M-S(y)}. \end{aligned}

The factor \(\prod_i\binom{m_i}{y_i}\) depends on the observed group counts and fixed trial counts, but not on \(p\). The remaining factor depends on the data only through \(S(y)\). This is the required factorization, so \(S\) is sufficient for \(p\). \(\square\)

Unequal group sizes change the parameter-free combinatorial factor, but they do not prevent this reduction. The total number of trials \(M\) is known and fixed, so once \(S\) is given, the number of failures is \(M-S\). In particular, the pooled success proportion \(S/M\) contains the same information as \(S\) about \(p\) when \(M\) is fixed and positive.

Worked Example: Three Groups With Unequal Trial Counts

Suppose the fixed group sizes are \(m_1=2\), \(m_2=4\), and \(m_3=3\), and the observed success counts are \(y_1=1\), \(y_2=3\), and \(y_3=0\). The total number of trials is \(M=2+4+3=9\), and the total number of successes is \(S=1+3+0=4\). The likelihood is

\begin{aligned} L(p;y) &=\binom{2}{1}\binom{4}{3}\binom{3}{0} p^{1+3+0}(1-p)^{(2-1)+(4-3)+(3-0)}\\ &=(2)(4)(1)p^4(1-p)^5\\ &=8p^4(1-p)^5. \end{aligned}

At \(p=0.3\), this is \(8(0.3)^4(0.7)^5=8(0.0081)(0.16807)=0.010890936\). Any other group-count vector with the same fixed sizes and four total successes has the same parameter-dependent factor \(p^4(1-p)^5\). Its likelihood can have a different numerical coefficient, but that coefficient does not depend on \(p\).

How Group Successes Are Allocated Given Their Total

For grouped data, sufficiency says more than that the likelihood has a convenient algebraic form. Conditional on the total \(S=s\), the distribution of the group allocation \((Y_1,\ldots,Y_G)\) does not depend on \(p\). For a feasible vector \(y\) with \(\sum_i y_i=s\), dividing its joint probability by \(P_p(S=s)\) gives

P_p(Y_1=y_1,\ldots,Y_G=y_G\mid S=s) =\frac{\prod_{i=1}^{G}\binom{m_i}{y_i}}{\binom{M}{s}}.

To see the denominator, regard all \(M\) trial positions as distinct and split them into groups of sizes \(m_1,\ldots,m_G\). Choosing \(s\) positions from all \(M\) positions can be done by first choosing \(y_i\) positions from group \(i\), for each \(i\), and summing over all feasible vectors whose counts total \(s\). Hence

\sum_{\substack{0\le y_i\le m_i\\\sum_i y_i=s}} \prod_{i=1}^{G}\binom{m_i}{y_i} =\binom{M}{s}.

This counting identity also verifies that the conditional probabilities sum to one. The allocation favors group-count vectors that can be realized in more ways, but the probability assigned to each vector does not involve \(p\).

Worked Example: Two Groups Given Two Total Successes

Let \(m_1=2\) and \(m_2=3\), and condition on \(S=2\). The possible values of \(Y_1\) are 0, 1, and 2; \(Y_2\) must equal \(2-Y_1\). Their combinatorial weights are

\begin{array}{c|c|c} Y_1 & Y_2 & \binom{2}{Y_1}\binom{3}{Y_2}\\ 0 & 2 & 1\cdot 3=3\\ 1 & 1 & 2\cdot 3=6\\ 2 & 0 & 1\cdot 1=1 \end{array}

The weights sum to \(3+6+1=10=\binom{5}{2}\). Therefore the conditional probabilities are \(3/10\), \(6/10\), and \(1/10\), respectively. For example, \(P(Y_1=1,Y_2=1\mid S=2)=6/10=0.6\). The same result follows by dividing the joint probability \(6p^2(1-p)^3\) by \(P(S=2)=\binom{5}{2}p^2(1-p)^3\); the factors involving \(p\) cancel.

Assumptions, Uses, and a Common Pitfall

The reduction to a total succeeds because the probability parameter is common across trials and the trial counts are fixed and known. If success probabilities differ across groups, the joint likelihood will generally retain group-specific success counts rather than depending on their sum alone. Likewise, changing the trial-count setup changes what is known in the likelihood; do not discard quantities that vary with the data or are part of the model.

A common mistake is to reason that unequal group sizes make the group-specific counts \(Y_1,\ldots,Y_G\) all necessary for inference about a common \(p\). That conclusion overlooks the factorization: the unequal sizes affect \(\prod_i\binom{m_i}{y_i}\), which is free of \(p\), while the parameter-dependent term is \(p^S(1-p)^{M-S}\). The correction is not to ignore the design, but to keep the known total \(M\) and recognize that the likelihood depends on the observed counts through \(S\). Conversely, it would be a mistake to claim that \(S\) is sufficient merely because the observations are binary; the common-\(p\) assumption is essential to this argument.

Key takeaway: For independent Bernoulli trials with a common success probability, the success count is sufficient. For independent grouped binomial observations with a common probability and fixed known group sizes, the sum of the group counts is sufficient as well. The group allocation conditional on that total contains no further information about the common probability.

Check Your Understanding

Use the likelihood factorization and the fixed-design assumptions in each answer.

For \(n\) independent Bernoulli observations, write the joint mass function and identify its factor that depends on the data through \(T\).
Why does the conditional distribution of the positions of successes given \(T=t\) not depend on \(p\)?
Three independent binomial groups have fixed sizes \(2,5,4\) and observed counts \(1,2,3\). Find \(M\), \(S\), and the parameter-dependent part of the likelihood.
For grouped binomial data, which part of the joint mass function depends on the group-count vector but not on \(p\)?
State the key probability assumption that makes the total success count sufficient in these examples, and explain what changes if group probabilities are not common.