Maximum Likelihood Estimation for Bernoulli Trials

Why Estimate a Bernoulli Success Probability?

Many actuarial and operational questions record a yes-or-no outcome for each trial: whether a hypothetical claim passes an eligibility rule, whether a component meets a quality standard, or whether a customer responds to an offer. When the trials share a common success probability, the Bernoulli model gives a direct way to estimate that probability from the observed outcomes.

The central result is simple to state: under independent trials with a common success probability, the maximum likelihood estimate (MLE) is the number of observed successes divided by the number of trials. The likelihood argument matters, however. It shows which assumptions support that estimate, explains how to handle samples with no successes or no failures, and helps prevent errors when differentiating.

Definition: A Bernoulli random variable $X_i$ takes the value $1$ for a success and $0$ for a failure. Its success probability is $p$, where $0\leq p\leq 1$, so $P(X_i=1)=p$ and $P(X_i=0)=1-p$. An independent, identically distributed Bernoulli sample consists of $n$ such variables that are mutually independent and all have the same success probability $p$.

This tutorial concerns the individual binary observations $X_1,\ldots,X_n$. The model requires the trial outcomes to be recorded as zero or one, the trials to be independent, and the success probability to be the same on each trial. If probabilities vary between trials or outcomes are dependent, the likelihood derived here is not the appropriate common-$p$ likelihood.

Constructing the Likelihood

Let the observed values be $x_1,\ldots,x_n$, where each $x_i$ is either zero or one. For one observation, the probability of the observed value can be written as $p^{x_i}(1-p)^{1-x_i}$: this equals $p$ when $x_i=1$ and $1-p$ when $x_i=0$. Independence makes the probability of the full observed sequence the product of these individual probabilities.

L(p;x_1,\ldots,x_n) =\prod_{i=1}^{n}p^{x_i}(1-p)^{1-x_i} =p^{\sum_{i=1}^{n}x_i}(1-p)^{n-\sum_{i=1}^{n}x_i}.

Write $s=\sum_{i=1}^{n}x_i$ for the observed number of successes. Then $n-s$ is the number of failures, and the likelihood is $L(p)=p^s(1-p)^{n-s}$. The likelihood is a function of the unknown parameter $p$, with the observed data held fixed. It is not a probability distribution over possible values of $p$.

Theorem: For $n$ independent, identically distributed Bernoulli observations with common success probability $p\in[0,1]$, the maximum likelihood estimate is $$ \widehat{p}_{\mathrm{MLE}}=\frac{s}{n} =\frac{1}{n}\sum_{i=1}^{n}x_i. $$ This includes the boundary cases $s=0$, for which $\widehat{p}_{\mathrm{MLE}}=0$, and $s=n$, for which $\widehat{p}_{\mathrm{MLE}}=1$.

Why the Estimate Is the Observed Proportion

For a sample containing at least one success and at least one failure, $0<s<n$, the likelihood is positive for $0<p<1$. Taking its logarithm preserves the location of its maximum because the logarithm is strictly increasing. The log-likelihood is

\ell(p)=\log L(p)=s\log p+(n-s)\log(1-p).

Differentiate with respect to $p$ and set the score equal to zero. The solution is the only interior stationary point. Its second derivative is negative throughout $0<p<1$, so it is a maximum; the likelihood is zero at both endpoints when $0<s<n$.

\begin{aligned} \ell'(p) &=\frac{s}{p}-\frac{n-s}{1-p}=0,\\ s(1-p)&=(n-s)p,\\ s-sp&=np-sp,\\ \widehat{p}_{\mathrm{MLE}}&=\frac{s}{n}. \end{aligned}

\ell''(p)=-\frac{s}{p^2}-\frac{n-s}{(1-p)^2}<0 \qquad (0<p<1,\ 0<s<n).

This calculation justifies the interior maximum. The endpoint samples can be checked directly instead: if $s=0$, then $L(p)=(1-p)^n$, which is largest at $p=0$; if $s=n$, then $L(p)=p^n$, which is largest at $p=1$. Thus the proportion formula also gives the maximum in both boundary cases.

Record the data.
Confirm that each observation is coded as $0$ for failure or $1$ for success, and identify the sample size $n$.

Count successes.
Calculate $s=\sum_{i=1}^{n}x_i$. The number of failures is $n-s$.

Apply the model.
Under the independent, common-$p$ Bernoulli model, write $L(p)=p^s(1-p)^{n-s}$.

Find and check the maximum.
For a mixed sample, maximize the log-likelihood; for an all-failure or all-success sample, check the appropriate endpoint. Report $s/n$.

Worked Applications

Worked Example: Estimating a Pass Probability From Individual Tests

A hypothetical laboratory performs 12 independent checks of the same type of sensor. Each check is recorded as $1$ if the sensor passes and $0$ otherwise. The outcomes, in order, are $1,0,1,1,0,1,0,1,1,0,1,1$. Assume each check has the same pass probability $p$. Find the MLE and the likelihood at that estimate.

Method: count the individual successes, then maximize their Bernoulli likelihood. The model is $X_i\sim\operatorname{Bernoulli}(p)$, independently for $i=1,\ldots,12$, with $0\leq p\leq1$. No interest rate or mortality assumption is involved in this hypothetical testing scenario.

There are eight ones and four zeros, so $n=12$, $s=8$, and $n-s=4$. The likelihood from the observed sequence is

$$ L(p)=p^8(1-p)^4. $$

Since the sample includes both outcomes, the interior maximum can be found from the score equation:

\frac{8}{p}-\frac{4}{1-p}=0 \quad\Longrightarrow\quad 8(1-p)=4p \quad\Longrightarrow\quad 8=12p \quad\Longrightarrow\quad \widehat{p}_{\mathrm{MLE}}=\frac{8}{12}=\frac{2}{3}.

Substituting the estimate into the likelihood gives

L\!\left(\frac{2}{3}\right) =\left(\frac{2}{3}\right)^8\left(\frac{1}{3}\right)^4 =\frac{256}{531441} \approx 0.0004817.

The likelihood value is the probability of this particular ordered sequence under the fitted model. It is not the estimated pass probability: that estimate is $2/3$. The common mistake avoided here is treating the likelihood value itself as the parameter estimate.

Worked Example: Separating Successes From Failures

In a hypothetical pilot study, 15 independently selected participants each receive the same reminder. A response is coded $1$ and no response is coded $0$. The record contains nine responses and six nonresponses. Estimate the response probability and identify the error in using the nonresponse fraction instead.

Method: use the likelihood score, matching each count to its outcome. Assume $X_i\sim\operatorname{Bernoulli}(p)$ independently, with the same $p$ for all 15 participants. The response is the success, so $s=9$ and $n-s=6$. No interest rate or mortality assumption applies.

The likelihood and log-likelihood are

L(p)=p^9(1-p)^6, \qquad \ell(p)=9\log p+6\log(1-p).

Differentiating and setting the score to zero gives

\frac{9}{p}-\frac{6}{1-p}=0 \quad\Longrightarrow\quad 9(1-p)=6p \quad\Longrightarrow\quad 9=15p \quad\Longrightarrow\quad \widehat{p}_{\mathrm{MLE}}=\frac{9}{15}=0.600.

The second derivative is $-9/p^2-6/(1-p)^2$, which is negative for $0<p<1$, confirming a maximum. The incorrect calculation $6/15=0.400$ estimates the nonresponse probability, not the response probability. The two estimates add to one, as they should for these complementary outcomes.

Worked Example: Every Observed Trial Is a Success

A hypothetical technician tests eight independently selected replacement filters. Every filter meets the specified standard. Under a common Bernoulli pass-probability model, find the MLE and explain why solving an interior score equation is not appropriate.

Method: maximize the likelihood on the full parameter space, including its boundary. Let $X_i=1$ indicate a filter passes, and assume $X_i\sim\operatorname{Bernoulli}(p)$ independently with $0\leq p\leq1$. Here $n=8$, $s=8$, and the number of failures is zero. No interest rate or mortality assumption is relevant.

The likelihood is

L(p)=p^8(1-p)^0=p^8,\qquad 0\leq p\leq1.

For $0\leq p\leq1$, $p^8$ increases as $p$ increases, so its largest value occurs at $p=1$. In particular, $L(1)=1$, and no other allowed value of $p$ gives a larger likelihood. Therefore,

\widehat{p}_{\mathrm{MLE}}=\frac{s}{n}=\frac{8}{8}=1.

The interior log-likelihood derivative is $8/p$, which has no zero for $0<p<1$. That does not mean there is no MLE: it means the maximum is at the boundary. The mistake avoided is requiring a zero derivative in the interior even when the parameter space includes endpoints. Similarly, if every observation were a failure, the likelihood would be $(1-p)^8$ and its maximum would occur at $p=0$.

Common Mistakes and When the Result Applies

The proportion-of-successes result is easy to remember, but it is important to interpret both the data coding and the model correctly. The likelihood counts each individual outcome through its contribution; it does not require a particular order of successes and failures. Reordering the same observations leaves the likelihood unchanged because the exponents $s$ and $n-s$ do not change.

Using failures in the numerator. If success is coded as $1$, the estimate of $p=P(X_i=1)$ is $s/n$, not $(n-s)/n$. The latter estimates the failure probability.
Ignoring endpoints. All zeros and all ones produce boundary MLEs, $0$ and $1$, respectively. A stationary point is not required to exist in the interior.
Assuming independence without justification. Multiplying the individual probabilities uses independence. If outcomes are dependent, this product need not be their joint probability.
Assuming one probability when trials differ. A common-$p$ model is inappropriate if different observations have different success probabilities. The observed proportion then does not, by this argument alone, maximize the appropriate likelihood.
Confusing a likelihood with a probability for the parameter. The likelihood evaluates the observed data for candidate values of $p$; it is not a probability distribution for $p$.

For an interior estimate, the score equation balances the observed successes against the total number of trials. This explains why the MLE is an empirical proportion rather than, for example, the ratio of successes to failures. The estimate describes the value of $p$ that makes the observed binary sample most likely under the stated model. The derivation establishes the MLE; it does not by itself establish that the estimate is unbiased or quantify its sampling uncertainty.

Key takeaway: For $n$ independent Bernoulli observations with a common success probability, count the ones and divide by $n$. The MLE is $\widehat{p}_{\mathrm{MLE}}=s/n$, including the boundary values zero and one when all observations are failures or successes. This treatment is aligned with the SOA VEE Mathematical Statistics syllabus.

Check Your Understanding

For each problem, assume the stated individual outcomes satisfy the Bernoulli model unless the problem asks you to assess that assumption.

A hypothetical greenhouse records whether each of 18 independently treated seedlings survives a week; 13 survive. Write the likelihood for the common survival probability $p$ and find its MLE. (Answer: $L(p)=p^{13}(1-p)^5$; $\widehat p_{\mathrm{MLE}}=13/18\approx0.7222$.)
A hypothetical device has a binary indicator for whether it completes a diagnostic cycle. In 20 independent cycles, the indicator is zero 7 times. If one is defined as success, find the success-probability MLE and the failure-probability MLE. (Answer: success MLE $=13/20=0.6500$; failure MLE $=7/20=0.3500$.)
A hypothetical survey records 11 independent yes-or-no outcomes, with four yes responses. Let success mean a no response instead. State $s$, write the likelihood, and find the MLE for this definition of success. (Answer: $s=7$, $L(p)=p^7(1-p)^4$, and $\widehat p_{\mathrm{MLE}}=7/11\approx0.6364$.)
In a hypothetical batch of 10 independent inspection trials, every item fails. Write the likelihood for the pass probability $p$, identify where it is maximized on $[0,1]$, and give the MLE. (Answer: $L(p)=(1-p)^{10}$, maximized at $p=0$; $\widehat p_{\mathrm{MLE}}=0$.)
A hypothetical monitoring study reports 6 successes and 4 failures among 10 independent binary trials. A candidate claims the MLE is $0.4$ because there are four failures. Give the correct likelihood, MLE, and the candidate’s error. (Answer: $L(p)=p^6(1-p)^4$; $\widehat p_{\mathrm{MLE}}=6/10=0.6000$; $0.4$ is the failure-probability estimate, not the success-probability estimate.)

What You'll Learn