Maximum Likelihood Estimation for Geometric Trials

Geometric Waiting Times and the Count Convention

A geometric observation records how long it takes for a success to occur in repeated trials. The count convention matters: in this tutorial, the successful trial is included. Thus a recorded waiting time of 1 means success on the first trial, while a waiting time of 4 means three failures followed by a success. This is the convention used to derive the maximum likelihood estimator (MLE) below.

Definition: Suppose trials are independent, each trial has the same success probability $p$, and $0<p\leq 1$. The geometric waiting time $X$ is the number of trials up to and including the first success. Its support is $x=1,2,3,\ldots$, and for $0<p<1$, $$ P(X=x)=p(1-p)^{x-1}. $$ If $p=1$, then $P(X=1)=1$. For a sample $X_1,\ldots,X_n$, the usual likelihood model also assumes the waiting times are independent and generated using the same $p$.

The factor $p$ in the probability is for the successful trial. The exponent $x-1$ counts the failures before it. If instead a variable records the number of failures before the first success, it takes values $0,1,2,\ldots$; that is a different way to record the same waiting process. Here the observed values start at 1.

Building the Likelihood

Let $x_1,\ldots,x_n$ be observed positive integer waiting times. Independence means their joint probability is the product of their individual probabilities. Define $T=\sum_{j=1}^n x_j$, the total number of trials counted across the observations. For $0<p<1$, the likelihood is

L(p)=\prod_{j=1}^n p(1-p)^{x_j-1} =p^n(1-p)^{\sum_{j=1}^n(x_j-1)} =p^n(1-p)^{T-n}.

The observed data are fixed when maximizing the likelihood, so $n$ and $T$ are fixed quantities. The parameter that can vary is $p$. Taking logarithms gives the log-likelihood

\ell(p)=\log L(p)=n\log p+(T-n)\log(1-p),\qquad 0<p<1.

The log is useful because it turns a product into a sum and preserves the location of the maximum: since the logarithm is strictly increasing, maximizing $L(p)$ is equivalent to maximizing $\ell(p)$ wherever the likelihood is positive.

Deriving and Checking the MLE

First consider $T>n$, meaning at least one observed waiting time exceeds 1. Differentiate the log-likelihood with respect to $p$, set the score equal to zero, and solve:

\frac{d\ell}{dp}=\frac{n}{p}-\frac{T-n}{1-p}=0 \quad\Longrightarrow\quad n(1-p)=p(T-n) \quad\Longrightarrow\quad n=pT \quad\Longrightarrow\quad \hat p=\frac{n}{T}.

This calculation gives a candidate, but a score equation alone does not establish that it is a maximum. For $T>n$, the candidate satisfies $0<n/T<1$. The second derivative is

\frac{d^2\ell}{dp^2}=-\frac{n}{p^2}-\frac{T-n}{(1-p)^2}<0,\qquad 0<p<1.

Thus the log-likelihood is strictly concave on the interior, and its stationary point is the unique interior maximum. The likelihood tends to zero at either endpoint when $T>n$, so no boundary value gives a larger likelihood.

Theorem: For independent geometric waiting times counted up to and including the successful trial, the MLE of the common success probability is $$ \hat p=\frac{n}{\sum_{j=1}^n x_j}. $$ If all observations equal 1, then $\sum x_j=n$, and the MLE is the boundary value $\hat p=1$. If at least one observation exceeds 1, the displayed estimate lies strictly between 0 and 1 and is the unique maximum.

The all-ones case needs its own check. If every observation is 1, the likelihood is $L(p)=p^n$, including at $p=1$. This increases as $p$ increases on $0<p\leq1$, so it is maximized at $p=1$. The interior score calculation does not produce an interior solution in this case. The formula $n/T$ still gives the correct answer, but the justification is a boundary argument rather than an interior score solution.

Worked Examples

Worked Example: Estimating Success Probability From Individual Waiting Times

In a hypothetical equipment check, four independent units are tested repeatedly under the same conditions. The recorded numbers of tests up to and including the first successful reading are $2,3,1,4$. Estimate the probability $p$ of success on any one test by maximum likelihood, and verify the maximum.

Method. Use the geometric model with counts including the successful trial. There are $n=4$ observations, and their total is $$ T=2+3+1+4=10. $$

For these data, the likelihood is $$ L(p)=p^4(1-p)^{(2-1)+(3-1)+(1-1)+(4-1)} =p^4(1-p)^6. $$

The log-likelihood and its score are $$ \ell(p)=4\log p+6\log(1-p),\qquad \frac{d\ell}{dp}=\frac{4}{p}-\frac{6}{1-p}. $$ Setting the score to zero gives $4(1-p)=6p$, so $4=10p$ and $\hat p=4/10=0.4$.

The second derivative is $-4/p^2-6/(1-p)^2$, which is negative for $0<p<1$. Therefore this stationary point is the unique maximum. The estimate is $\hat p=0.4$.

Check and mistake avoided. The data total 10 trials across four waiting periods, so the estimate is four successes divided by ten counted trials. Treating the recorded values as numbers of failures would give $4/(4+10)=2/7$, which uses the wrong count convention.

Worked Example: Using a Frequency Table

A hypothetical laboratory records the number of attempts, including the successful attempt, needed to complete a calibration. The frequency table shows that 1 attempt occurred twice, 3 attempts occurred three times, and 6 attempts occurred once. Find the MLE of the common per-attempt success probability.

Method. Frequency counts represent repeated observations, so the sample size is $n=2+3+1=6$. The total number of counted attempts is $$ T=(2)(1)+(3)(3)+(1)(6)=2+9+6=17. $$

The likelihood, with each distinct value raised to its frequency, is $$ L(p)=\{p\}^{2}\{p(1-p)^2\}^{3}\{p(1-p)^5\}^{1} =p^6(1-p)^{6+5}=p^6(1-p)^{11}. $$

The log-likelihood is $6\log p+11\log(1-p)$. Its score equation is $$ \frac{6}{p}-\frac{11}{1-p}=0 \quad\Longrightarrow\quad 6(1-p)=11p \quad\Longrightarrow\quad p=\frac{6}{17}. $$

At this estimate, the second derivative is $-6/p^2-11/(1-p)^2<0$. Since at least one waiting time exceeds 1, the likelihood also approaches zero at the endpoints. Thus the unique MLE is $\hat p=6/17$, or 0.3529 to four decimal places.

Check and mistake avoided. The sample size is six observations, not the total of 17 attempts. The numerator counts observed successes, one per waiting period; the denominator is the sum of the waiting times.

Worked Example: Recognizing a Boundary Estimate

A hypothetical set of three independent trials of a repeated screening procedure records waiting times $1,1,1$, with each waiting time including the successful screening. Find the MLE and explain why differentiating the interior log-likelihood does not give the answer.

Method. Here $n=3$, $T=1+1+1=3$, and the likelihood is $$ L(p)=p^3(1-p)^0=p^3,\qquad 0<p\leq1. $$

For every $p$ in this parameter range, $p^3$ increases as $p$ increases. Its largest value is $L(1)=1$, so the MLE is $\hat p=1$. This agrees with $n/T=3/3=1$.

The interior log-likelihood is $3\log p$, whose derivative is $3/p>0$; it has no interior stationary point. The maximum is at the allowed endpoint, not at a solution to a score equation.

Check and mistake avoided. A candidate obtained by setting a score to zero is not required to exist for a maximum. Because all three successes occurred on the first attempt, the likelihood favors the largest allowed success probability.

Why the Estimate Has This Form

The score equation provides a direct justification for the estimator: the data contribute $n$ successful trials, one at the end of each waiting period, and $T-n$ failures before those successes. The likelihood has one factor of $p$ for each success and one factor of $1-p$ for each failure. Maximizing balances these two contributions, yielding $\hat p=n/T$. This is a likelihood argument; it does not require replacing $p$ by a sample average of the observed waiting times.

Check the data convention.
Confirm that each $x_j$ counts trials through the success, so $x_j-1$ is the number of preceding failures.

Find the sample size and total.
Set $n$ equal to the number of waiting periods and $T=\sum x_j$. For frequency data, multiply each value by its frequency before summing.

Form the likelihood.
Use $L(p)=p^n(1-p)^{T-n}$, or the equivalent product of the individual geometric probabilities.

Maximize and check the case.
For $T>n$, solve the score equation and verify concavity. For $T=n$, all waiting times are 1 and the maximum is at $p=1$.

Common Exam Trap: Shifting the Count

The most common error is to use the formula for a failure count when the observed values include the successful trial. For example, with observations $2,3,1,4$, the sum $T=10$ includes four successful trials and six failures. The correct likelihood is $p^4(1-p)^6$, giving $\hat p=4/10=0.4$.

A wrong calculation might treat the four observed values as failure counts and use $p^4(1-p)^{10}$, producing $4/(4+10)=2/7$. That model would describe four observed counts of failures before success, not four total trial counts through success. If the data genuinely were failure counts $K_j$, their likelihood would be $p^n(1-p)^{\sum K_j}$, and the corresponding estimate would be $n/(n+\sum K_j)$. For waiting times $X_j=K_j+1$, this becomes $n/\sum X_j$, exactly the formula derived here.

Key takeaway: For independent geometric waiting times that include the successful trial, add the recorded counts and divide the number of waiting periods by that total. Check whether every count is 1: if so, the MLE is the boundary value $p=1$.

This estimation problem is aligned with the SOA VEE Mathematical Statistics syllabus. Its central skill is translating the verbal observation convention into the correct probability model before maximizing. The same principle applies broadly in likelihood work: a shift in what the data count changes the likelihood, even when the underlying repeated-trial process is unchanged.

Check Your Understanding

For each problem, use the convention that the recorded waiting time includes the successful trial. State the MLE and check whether it is an interior or boundary estimate.

A technician records waiting times $3,1,2,4$ for four independent calibration runs with a common success probability. Find the likelihood in terms of $p$, then find the MLE. (Answer: $L(p)=p^4(1-p)^6$, $\hat p=0.4000$, interior.)
A hypothetical screening system produces six waiting times, all equal to 1. Write the likelihood and find its maximum over $0<p\leq1$. (Answer: $L(p)=p^6$, $\hat p=1$, boundary.)
A frequency table reports waiting time 1 twice, waiting time 3 three times, and waiting time 6 once. Find the sample size, total waiting time, and MLE. (Answer: $n=6$, $T=17$, $\hat p=6/17\approx0.3529$, interior.)
In five independent trials of a process, the recorded numbers of failures before success are $0,1,4,2,3$. Convert to the equivalent total-trial convention and find the MLE of $p$. (Answer: total waiting times are $1,2,5,3,4$, so $T=15$ and $\hat p=5/15=1/3$, interior.)
For waiting times $2,2,2,1,3$, a candidate uses $5/(5+10)$ as the MLE. Identify the count-convention error and give the correct estimate. (Answer: the observations already include each successful trial, so $T=10$ and the correct MLE is $5/10=0.5000$, not $1/3$.)

What You'll Learn