Binomial Proportion Risk And Boundary Behavior

Risk of a Binomial Sample Proportion

When a binomial probability is close to zero or one, the sample proportion can have a small squared error on the original probability scale. But a small absolute error does not necessarily mean precise estimation relative to the size of the probability itself. The exact risk formula makes both aspects visible and shows how they change with the true proportion and the sample size.

Suppose $X$ counts successes in $n$ independent Bernoulli trials, each with success probability $p$. The parameter space includes the boundaries $p=0$ and $p=1$. We estimate $p$ by the sample proportion, and use squared-error loss to assess its error.

Definition: Let $n$ be a positive integer, $p\in[0,1]$, and $X\sim\operatorname{Bin}(n,p)$. The sample proportion is $\hat p=X/n$. Under squared-error loss $L(p,\hat p)=(\hat p-p)^2$, its risk at $p$ is $$ R(p,\hat p)=E_p[(\hat p-p)^2], $$ where $E_p$ denotes expectation under the binomial distribution with success probability $p$.

The risk is a function of the fixed, unknown parameter $p$, not a quantity computed from a particular observed value of $X$. For this estimator, it has a particularly simple exact form.

Theorem: For every positive integer $n$ and every $p\in[0,1]$, the sample proportion is unbiased and its squared-error risk is $$ E_p[\hat p]=p,\qquad R(p,\hat p)=\operatorname{Var}_p(\hat p)=\frac{p(1-p)}{n}. $$

Proof. Represent the binomial count as $X=\sum_{i=1}^n Y_i$, where $Y_i$ is the indicator of success on trial $i$. The variables $Y_1,\ldots,Y_n$ are independent Bernoulli variables with $E_p[Y_i]=p$ and $\operatorname{Var}_p(Y_i)=p(1-p)$. Therefore

E_p[\hat p] =\frac{1}{n}\sum_{i=1}^n E_p[Y_i] =p, \qquad \operatorname{Var}_p(\hat p) =\frac{1}{n^2}\sum_{i=1}^n\operatorname{Var}_p(Y_i) =\frac{p(1-p)}{n}.

Since $E_p[\hat p]-p=0$, the bias-variance identity gives $E_p[(\hat p-p)^2]=\operatorname{Var}_p(\hat p)+\{E_p[\hat p]-p\}^2=\operatorname{Var}_p(\hat p)$. This proves both claims, including at $p=0$ and $p=1$, where the Bernoulli variables are constant. $\square$

Worked Example: Exact Risk for a Small Sample

Let $n=8$ and $p=1/4$. Then $E_p[X]=np=2$, and $\operatorname{Var}_p(X)=np(1-p)=8(1/4)(3/4)=3/2$. Dividing by $n$ gives $E_p[\hat p]=2/8=1/4$, so the estimator is unbiased. Its risk is

R(1/4,\hat p) =\operatorname{Var}_p(X/8) =\frac{\operatorname{Var}_p(X)}{8^2} =\frac{3/2}{64} =\frac{3}{128} =0.0234375.

The root mean squared error is $\sqrt{3/128}\approx0.1531$. This is an error scale, not a guaranteed distance from the estimate to the true value in any one sample.

Where Risk Is Largest

The risk curve is a scaled parabola in $p$. Its symmetry and maximum follow directly from writing $p(1-p)$ as a completed square.

Result: For fixed $n$, the risk is symmetric about $p=1/2$, strictly increases on $0\leq p\leq1/2$, strictly decreases on $1/2\leq p\leq1$, and is maximized at $p=1/2$, where its value is $1/(4n)$.

Proof. First, $p(1-p)=(1-p)p$, so $R(p,\hat p)=R(1-p,\hat p)$. Differentiating on $(0,1)$ gives

\frac{d}{dp}R(p,\hat p)=\frac{1-2p}{n}.

This derivative is positive for $p<1/2$ and negative for $p>1/2$. Thus the risk increases up to $1/2$ and decreases thereafter. At the maximum,

R(1/2,\hat p)=\frac{(1/2)(1/2)}{n}=\frac{1}{4n}.

The endpoint values are both zero, as also follows from the risk formula. $\square$

The maximum risk decreases in proportion to $1/n$: doubling the sample size halves the risk at every fixed $p$. The root mean squared error, which is the square root of risk, decreases in proportion to $1/\sqrt n$.

Worked Example: Comparing Risk Across the Parameter Space

For $n=20$, compare the risks at $p=0.05$, $p=0.50$, and $p=0.95$.

At $p=0.05$,

R(0.05,\hat p)=\frac{0.05(0.95)}{20} =\frac{0.0475}{20} =0.002375.

At $p=0.50$,

R(0.50,\hat p)=\frac{0.50(0.50)}{20} =\frac{0.25}{20} =0.0125.

At $p=0.95$, symmetry gives the same risk as at $p=0.05$:

R(0.95,\hat p)=\frac{0.95(0.05)}{20}=0.002375.

The middle value is the maximum, $1/(4\cdot20)=0.0125$. The two boundary-near values are equal because replacing $p$ by $1-p$ does not change the risk.

Boundary Behavior and Relative Accuracy

At $p=0$, every trial is a failure with probability one, so $X=0$ and $\hat p=0$ exactly. At $p=1$, every trial is a success, so $X=n$ and $\hat p=1$ exactly. Thus the absolute squared-error risk is zero at both boundaries. For fixed $n$, the risk approaches zero linearly as $p$ approaches zero or one:

R(p,\hat p)=\frac{p}{n}-\frac{p^2}{n} \quad\text{as }p\to0, \qquad R(p,\hat p)=\frac{1-p}{n}-\frac{(1-p)^2}{n} \quad\text{as }p\to1.

This boundary behavior is about absolute squared error. When $p>0$, a scale-relative measure is the risk divided by $p^2$:

\frac{R(p,\hat p)}{p^2}=\frac{1-p}{np}.

For fixed $n$, this ratio grows without bound as $p$ decreases toward zero. It is not defined at $p=0$, where both the target and the risk are zero. Consequently, risk that is small on the original probability scale can still represent substantial error relative to a very small $p$.

Worked Example: Rare Events Near the Lower Boundary

Take $n=100$. If $p=0.01$, the exact risk and relative risk are

R(0.01,\hat p)=\frac{0.01(0.99)}{100}=0.000099, \qquad \frac{R(0.01,\hat p)}{(0.01)^2} =\frac{0.000099}{0.0001}=0.99.

The probability of observing no successes is $P_p(X=0)=(1-p)^{100}=0.99^{100}\approx0.3660$. Now take the smaller probability $p=0.001$, still with $n=100$:

R(0.001,\hat p)=\frac{0.001(0.999)}{100}=0.00000999, \qquad \frac{R(0.001,\hat p)}{(0.001)^2} =\frac{0.00000999}{0.000001}=9.99.

Here $P_p(X=0)=0.999^{100}\approx0.9048$. The absolute risk is smaller than in the first case, but the relative risk is larger, and a zero estimate is more likely. Both statements are consistent: the absolute error scale shrinks with $p$, while the error is large compared with $p$ itself.

The same distinction appears when the true probability approaches a boundary as the sample size grows. For a fixed constant $c>0$, set $p_n=c/n$ for integers $n>c$. Then

Result: If $p_n=c/n$ with fixed $c>0$, then $$ R(p_n,\hat p)=\frac{c}{n^2}\left(1-\frac{c}{n}\right), \qquad P_{p_n}(X=0)=\left(1-\frac{c}{n}\right)^n\xrightarrow{}e^{-c}. $$

Derivation. Substituting $p_n=c/n$ into the exact risk formula gives

R(p_n,\hat p) =\frac{(c/n)(1-c/n)}{n} =\frac{c}{n^2}\left(1-\frac{c}{n}\right).

The event $X=0$ means that all $n$ trials are failures, so independence gives $P_{p_n}(X=0)=(1-p_n)^n=(1-c/n)^n$. For the limit, $n\log(1-c/n)\to-c$, using $\log(1-u)/u\to-1$ as $u\to0$. Exponentiating yields $(1-c/n)^n\to e^{-c}$. Thus the absolute risk tends to zero at order $n^{-2}$, even though the probability of seeing no successes tends to the positive value $e^{-c}$. $\square$

Why the Distinction Matters

Risk summarizes average squared error over repeated samples at a specified parameter value. It does not say that every estimate will be close to $p$, nor does a small risk by itself describe performance relative to the magnitude of a small parameter. The binomial proportion makes these distinctions especially clear: at a boundary the observation is deterministic, while for probabilities of order $1/n$, zero counts can remain common even as absolute risk becomes very small.

A common mistake is to see the factor $p(1-p)$ become small near zero and conclude that estimation is therefore uniformly precise in every relevant sense. The correction is to specify the scale: $R(p,\hat p)$ is absolute squared-error risk, whereas $R(p,\hat p)/p^2$, when $p>0$, measures squared error relative to the squared parameter. These answer different questions. Another mistake is to substitute the observed proportion into the risk formula and call the result the estimator’s risk. The risk is calculated under the sampling distribution at the true $p$; the observed value alone does not reveal that true risk.

Check Your Understanding

Use the exact risk formula and the boundary discussion to answer these questions.

For $n=40$, calculate the risk at $p=0.20$ and the maximum risk over $p\in[0,1]$.
Why does the sample proportion have zero risk at $p=0$ and $p=1$?
For fixed $n$, what happens to $R(p,\hat p)/p^2$ as $p$ approaches zero from above?
If $p_n=3/n$, find the exact risk and the limiting probability of observing zero successes.
Explain why a small absolute risk does not necessarily imply good relative accuracy for a rare event.

Binomial Proportion Risk And Boundary Behavior

What You'll Learn