Binomial Frequency Maximum Likelihood

Introduction

A claim count is often observed over a known number of opportunities: for example, how many of a portfolio's covered vehicles have at least one claim during a specified period. If each opportunity has the same claim probability and the outcomes are independent, the count has a binomial model. Estimating that probability by maximum likelihood is simple when the model is stated correctly, but pooling groups with different sizes is a frequent source of error.

This tutorial derives the estimate from the likelihood and checks it using the score equation. The setting is deliberately specific: complete counts, known trial totals, and one common claim probability. If observations are censored, truncated, or generated by a different frequency model, this likelihood does not apply without modification.

Definition: Let \(X_j\) be the number of successes in group \(j\), out of a known \(n_j\) independent trials. Under the common-probability binomial model, \(X_j\sim\operatorname{Binomial}(n_j,p)\), where \(p\in[0,1]\) is the probability of success on each trial. The observed count \(x_j\) is complete when the number of successes and the total number of trials are known, with no outcomes omitted or censored.

In a hypothetical claim-frequency application, a trial might be a covered vehicle over a fixed policy period, and success might mean that the vehicle has at least one claim during that period. The binomial model then assumes each vehicle-period contributes at most one success. The probability \(p\) must be common across the trials being pooled. The model does not automatically fit a count of multiple claims per vehicle.

Likelihood for Complete Binomial Counts

Suppose there are \(m\) independent groups. Group \(j\) has \(n_j\) known trials and a fully observed count \(x_j\). Conditional on the model assumptions, the probability of observing \(x_j\) is \(\binom{n_j}{x_j}p^{x_j}(1-p)^{n_j-x_j}\). Independence across groups makes the likelihood the product of these probabilities.

L(p)=\prod_{j=1}^{m}\binom{n_j}{x_j}p^{x_j}(1-p)^{n_j-x_j} =\left(\prod_{j=1}^{m}\binom{n_j}{x_j}\right)p^{X}(1-p)^{N-X}, \qquad X=\sum_{j=1}^{m}x_j,\quad N=\sum_{j=1}^{m}n_j.

Here \(X\) is the total number of successes and \(N\) is the total number of trials. The binomial coefficients do not depend on \(p\), so they affect the likelihood's numerical value but not which value of \(p\) maximizes it. The part that depends on \(p\) is \(p^X(1-p)^{N-X}\). Thus the relevant information for estimating the common probability is the pooled success count and pooled trial count—not an unweighted average of group-specific rates.

Theorem: If \(0<X<N\), the maximum likelihood estimate under the common-probability binomial model is \(\hat p=X/N\). If \(X=0\), the maximum occurs at the boundary \(\hat p=0\); if \(X=N\), it occurs at the boundary \(\hat p=1\).

Derivation and Score-Equation Check

For an interior value \(0<p<1\), take the logarithm of the likelihood. The terms involving binomial coefficients are constant with respect to \(p\). Differentiating the remaining terms gives the score, the derivative of the log-likelihood.

\ell(p)=\text{constant}+X\log p+(N-X)\log(1-p)

U(p)=\frac{d\ell(p)}{dp} =\frac{X}{p}-\frac{N-X}{1-p}.

An interior maximum must have score zero. Solving the score equation shows why the estimate is the pooled proportion:

\begin{aligned} \frac{X}{p}-\frac{N-X}{1-p}&=0,\\ X(1-p)&=(N-X)p,\\ X-Xp&=Np-Xp,\\ X&=Np,\\ \hat p&=\frac{X}{N}. \end{aligned}

The score equation identifies a stationary point, so a maximum must still be checked. For \(0<X<N\), the second derivative is

\ell''(p)=-\frac{X}{p^2}-\frac{N-X}{(1-p)^2}<0 \qquad (0<p<1).

The log-likelihood is therefore strictly concave on the interior, and its stationary point is the unique interior maximum. At the boundaries, direct comparison is needed because the interior score formula is not defined at \(p=0\) or \(p=1\). If \(X=0\), the likelihood's \(p\)-dependent part is \((1-p)^N\), maximized at \(p=0\). If \(X=N\), it is \(p^N\), maximized at \(p=1\). These are valid maximum likelihood estimates even though there is no interior score root.

Check the data and model.
Confirm each group has a known total \(n_j\), a complete count \(x_j\), and that the independent trials share one probability \(p\).

Pool counts and totals.
Compute \(X=\sum_j x_j\) and \(N=\sum_j n_j\). Do not average the group rates unless their trial totals happen to be equal.

Estimate and check.
Use \(\hat p=X/N\). For \(0<X<N\), substitute into the score equation and use the negative second derivative to verify a maximum. For all failures or all successes, check the appropriate boundary directly.

Worked Examples

Worked Example: Pooling Groups With Unequal Trial Totals

A hypothetical insurer divides covered vehicles into three rating groups for a study period. The complete results are: Group A, 3 vehicles with a claim out of 20; Group B, 5 out of 30; and Group C, 4 out of 10. Assuming independent vehicle outcomes and one common claim probability, estimate that probability by maximum likelihood and verify the estimate with the score equation.

Method and assumptions. Each vehicle-period is one Bernoulli trial, the three groups share the same claim probability \(p\), and all counts and trial totals are observed. The grouped likelihood is binomial, so first pool successes and trials.

The total number of successes is \(X=3+5+4=12\). The total number of trials is \(N=20+30+10=60\), and the total number of failures is \(N-X=60-12=48\). Therefore,

\hat p=\frac{X}{N}=\frac{12}{60}=0.20.

Score check. At \(p=0.20\), the score is \(U(0.20)=12/0.20-48/0.80=60-60=0\). The second derivative is \(-12/(0.20)^2-48/(0.80)^2=-300-75=-375<0\), so the stationary point is a maximum. The estimate is a claim probability of \(0.20\), or 20%.

Common mistake avoided. An unweighted average of the group proportions would be \((3/20+5/30+4/10)/3=(0.15+0.1667+0.40)/3\approx0.2389\). That gives each group equal weight despite their different trial totals. Pooling the counts correctly weights each trial equally and gives \(0.20\).

Worked Example: Writing and Solving the Grouped Score Equation

A hypothetical warranty administrator records complete claim indicators over three independent sets of covered items. The results are 4 claims among 12 items, 9 among 30 items, and 7 among 28 items. Under a common binomial claim probability, find the maximum likelihood estimate and show the score-equation calculation.

Method and assumptions. Treat each item as one trial with the same probability \(p\) of at least one claim during the stated observation period. The trial totals are fixed and known, and all outcomes are observed.

Pool the successes: \(X=4+9+7=20\). Pool the trials: \(N=12+30+28=70\). There are \(N-X=70-20=50\) failures. The estimate is

\hat p=\frac{20}{70}=\frac{2}{7}\approx0.2857.

The score based on the pooled counts is \(U(p)=20/p-50/(1-p)\). Substituting \(p=2/7\) gives

U\left(\frac{2}{7}\right) =\frac{20}{2/7}-\frac{50}{5/7} =70-70=0.

The second derivative at this point is \(-20/(2/7)^2-50/(5/7)^2=-245-98=-343<0\). Thus the score root is the unique interior maximum. Rounded to four decimal places, the estimate is \(0.2857\), or \(28.57\%\).

Common mistake avoided. The observed rates are \(4/12\), \(9/30\), and \(7/28\), but the likelihood does not assign equal influence to those three proportions. It counts 20 successes among 70 trials; using the group count as the weighting unit rather than the number of trials would not maximize the stated likelihood.

Worked Example: A Boundary Estimate With No Claims

A hypothetical insurer observes no claims among three complete groups of vehicle-periods: 0 claims out of 14, 0 out of 19, and 0 out of 27. Under the common-probability binomial model, find the maximum likelihood estimate. Explain why solving the interior score equation is not the right method here.

Method and assumptions. The 60 vehicle-period outcomes are independent, each has the same claim probability \(p\), and all outcomes are observed. The pooled success count is \(X=0+0+0=0\); the pooled trial total is \(N=14+19+27=60\).

The \(p\)-dependent part of the likelihood is

L(p)\ \propto\ p^0(1-p)^{60}=(1-p)^{60}.

For every \(p\in[0,1]\), \(1-p\leq1\), so \((1-p)^{60}\leq1\). Equality holds at \(p=0\). Hence the maximum likelihood estimate is the boundary value \(\hat p=0\).

For \(0<p<1\), the score is \(U(p)=0/p-60/(1-p)=-60/(1-p)<0\). It has no interior zero; the log-likelihood decreases throughout the interior toward the right. Setting the score equal to zero and reporting an interior estimate would therefore be a mistake. In the analogous all-success case \(X=N\), the likelihood is maximized at the other boundary, \(p=1\).

Check. At \(p=0\), the model assigns probability 1 to zero successes in 60 trials, and no larger likelihood is possible. This confirms the boundary result.

Assumptions, Interpretation, and Exam Traps

The pooled estimate is meaningful only relative to its model. It is not a general rule for every observed claim count. In particular, the binomial likelihood above requires known trial totals, independent outcomes, a common success probability, and a complete success count for each group. If the data do not satisfy these conditions, the resulting estimate may not describe the intended process.

Do not pool rates without their denominators. The estimator is total successes divided by total trials. Averaging group proportions equally is generally different.
Do not confuse the claim count with the trial total. A count such as 8 claims among 40 covered units contributes 8 successes and 40 trials, not 8 trials.
Check what counts as a success. The binomial interpretation requires a binary outcome for each trial. A model for the number of multiple claims per policy is not this one.
Do not force an interior score solution at a boundary. With no successes or no failures, the likelihood is maximized at 0 or 1, respectively.
Do not ignore heterogeneity. If trial probabilities differ materially, a single common-\(p\) binomial model may be inappropriate; the pooled estimate then answers a model-based, not necessarily homogeneous-risk, question.

The calculation is aligned with the SOA Exam ASTAM (Advanced Short-Term Actuarial Mathematics) syllabus: identify the likelihood from the stated complete-data model, preserve the known trial totals, differentiate the log-likelihood, and check whether the estimate is interior or on a boundary. The central exam discipline is to state the model before using the compact pooled formula.

Key takeaway: For independent complete binomial counts with known totals and one common probability, the maximum likelihood estimate is total successes divided by total trials. The score equation verifies this estimate when it is interior; all failures and all successes produce boundary estimates at 0 and 1.

Check Your Understanding

For each problem, assume the stated groups are independent and share one binomial success probability unless the question says otherwise.

A hypothetical roadside-assistance program records 2 claims among 10 covered units and 6 claims among 30 covered units. Find the maximum likelihood estimate of the common claim probability. (Answer: \(8/40=0.2000\))
A device-insurance study observes 3 claims among 12 insured devices and 9 among 28. Calculate the pooled maximum likelihood estimate. (Answer: \(12/40=0.3000\))
Three complete groups record 0 claims among 18 units, 5 among 22, and 7 among 20. Find the estimate and state whether it is an interior or boundary estimate. (Answer: \(12/60=0.2000\), interior)
A complete study records zero claims in groups of 14, 19, and 27 vehicle-periods. Give the maximum likelihood estimate and explain its location in the parameter space. (Answer: \(\hat p=0\), a boundary estimate)
A hypothetical portfolio has 2 claims among 8 units, 12 among 32, and 9 among 60. Compute the common-probability estimate and state why an unweighted mean of the three group rates is not the likelihood estimate. (Answer: \(23/100=0.2300\); the group totals differ, so the likelihood pools successes and trials rather than weighting each group rate equally)

What You'll Learn