UMVU Estimation in Binomial Models

Why the Success Count Gives UMVU Estimators

Suppose a fixed number of trials each produces either a success or a failure, with the same unknown success probability. The observed number of successes contains the information in the sample about that probability. In this setting, it can do more than provide a convenient summary: it can be used to construct the unique minimum-variance unbiased estimator of many quantities, including powers of the success probability and probabilities of events in a future binomial count.

The key steps are to find an unbiased starting estimator and then condition it on the observed total number of successes. The resulting estimator is a function of a complete sufficient statistic, so the Lehmann–Scheffé theorem identifies it as UMVU: uniformly minimum-variance among all unbiased estimators of the target.

The count \(T\) is sufficient for \(p\): the joint probability of the observed sequence is \(p^T(1-p)^{n-T}\), which depends on the sample through \(T\). It is also complete for \(p\in(0,1)\). To see the completeness property, suppose a function \(g\) satisfies \(E_p[g(T)]=0\) for every \(p\in(0,1)\). Then

Divide by the positive quantity \((1-p)^n\), and set \(u=p/(1-p)\). As \(p\) ranges over \((0,1)\), \(u\) ranges over all positive values. The resulting polynomial \(\sum_{t=0}^{n}g(t)\binom{n}{t}u^t\) is zero for every \(u>0\), so all its coefficients are zero. Since \(\binom{n}{t}>0\), \(g(t)=0\) for every \(t\). This is completeness.

Estimating Powers of the Success Probability

For an integer \(r\) with \(1\le r\le n\), the target \(p^r\) is the probability that a specified set of \(r\) trials are all successes. The indicator of that event is unbiased for \(p^r\), but it depends on which particular trials succeeded. Conditioning on \(T\) averages over all such sets and gives an estimator depending only on the observed count.

where \((a)_r=a(a-1)\cdots(a-r+1)\) is the falling factorial. If \(T<r\), then \(\binom{T}{r}=0\), so the estimator is zero. For \(r=0\), the target is the known constant \(p^0=1\), estimated by 1.

Here is the conditioning argument. For any fixed set of \(r\) trial positions, let \(I\) equal 1 if all those trials succeed and 0 otherwise. Independence gives \(E_p[I]=p^r\). Conditional on observing \(T=t\), every arrangement of \(t\) successes among the \(n\) positions is equally likely. There are \(\binom{n}{r}\) possible sets of \(r\) positions, and \(\binom{t}{r}\) of them consist entirely of successes. Therefore

Thus the displayed estimator is unbiased by the law of iterated expectation. It is a function of the complete sufficient statistic \(T\), so the Lehmann–Scheffé theorem makes it the unique UMVU estimator of \(p^r\).

Estimating Probabilities of Binomial Events

A related target is the probability that a new block of \(m\) independent trials has a specified success count or satisfies a specified event. Let \(Y\sim\operatorname{Bin}(m,p)\), where \(m\le n\), and let \(A\) be any subset of \(\{0,1,\ldots,m\}\). The target is \(P_p(Y\in A)\). For example, \(A=\{2\}\) asks for the probability of exactly two successes, while \(A=\{1,3\}\) asks for the probability of one or three successes.

Imagine using the first \(m\) observations from the sample as the block. The indicator that their success count belongs to \(A\) is unbiased for \(P_p(Y\in A)\). Conditional on the total \(T=t\), the number of successes in those \(m\) positions has a hypergeometric distribution: \(k\) successes can be selected into the block in \(\binom{t}{k}\binom{n-t}{m-k}\) ways, out of \(\binom{n}{m}\) possible blocks. Conditioning therefore gives the estimator below.

Use the convention that \(\binom{a}{b}=0\) when \(b<0\) or \(b>a\), for nonnegative integer \(a\). This handles counts that cannot occur for the observed \(T\), without requiring separate adjustments. The estimator is the conditional expectation of the unbiased block-event indicator, so it remains unbiased; being a function of the complete sufficient count, it is UMVU.

Conditions, Interpretation, and Exam Traps

The event-probability construction requires \(m\le n\): the observed sample must contain a block of the target size whose event indicator can serve as an unbiased starting estimator. The power estimator likewise requires \(r\le n\). These are not merely computational restrictions. Any estimator based on \(n\) Bernoulli trials has an expectation that is a polynomial in \(p\) of degree at most \(n\). Consequently, a target such as \(p^r\) with \(r>n\) cannot have an unbiased estimator based on this sample. For an event involving \(m>n\) future trials, unbiased estimation is not guaranteed by this construction and may be impossible; check the target against the degree limit rather than applying the formula outside its conditions.

For event probabilities, the formula can estimate any subset \(A\) of possible counts, including “at least one,” “no more than two,” or a union of specified counts. The count \(T\) is not itself the future count; it is the observed sufficient statistic used to average the block indicator over all possible blocks. This distinction prevents a frequent error: confusing the observed sample count with the random count \(Y\) whose probability is being estimated.

Another trap is to substitute \(T/n\) for \(p\) in a target formula and assume the result is UMVU. For example, for the probability of exactly two successes in three trials, substitution gives \(\binom{3}{2}(T/n)^2(1-T/n)\), which is generally not the conditional expectation of a block-event indicator. The UMVU method instead conditions an unbiased indicator on \(T\), preserving unbiasedness and producing a function of the complete sufficient statistic.