Binomial Frequency Estimation with Exposure

Why Known Trial Counts Matter

A binomial frequency model describes the number of successes among a known number of trials. In an insurance application, a trial might be one eligible policy during a specified period, and a success might be that the policy has at least one claim during that period. The model is useful only when the trial definition is clear: the same probability must apply to each trial, and the trial outcomes must be independent.

When data are reported in groups, the number of trials can differ from one group to another. That does not by itself prevent estimation of a common probability. If the trials are otherwise comparable, the correct estimate uses the total number of successes divided by the total number of trials. An unweighted average of group percentages generally gives the wrong estimate because it gives a small group as much influence as a large group.

Definition: For group $j$, let $X_j$ be the number of successes among $m_j$ known trials. A common-probability binomial model assumes $X_j\sim\operatorname{Binomial}(m_j,p)$, where each trial has the same success probability $p$, and all trials are independent. Across groups, the same $p$ applies. The total number of trials is $M=\sum_j m_j$, and the total number of successes is $X=\sum_j X_j$.

In this setting, a “trial” must have a consistent meaning. If a trial is a one-year policy, then a half-year policy is not automatically an equivalent trial: its chance of having a claim may be different simply because it is observed for less time. Similarly, the binomial model counts whether a trial has a success, not how many events occur within it. If multiple claims per policy-period are possible and the count itself is being modeled, a Bernoulli outcome may not describe the data of interest.

Estimating the Common Probability

The method-of-moments estimate and the maximum-likelihood estimate coincide for this model. The estimate is the pooled success proportion, with every trial contributing equally. The likelihood derivation also shows why the total trial count, rather than the number of groups, determines the weighting.

\widehat p=\frac{\sum_j X_j}{\sum_j m_j}=\frac{X}{M}.

To see the moment argument, each trial outcome has expected value $p$. The mean success proportion across all $M$ trials therefore has expectation $p$. For the likelihood argument, the grouped counts are independent binomial variables with a common $p$. Their likelihood, as a function of $p$, is

L(p)=\prod_j {m_j\choose X_j}p^{X_j}(1-p)^{m_j-X_j} \ \propto\ p^X(1-p)^{M-X}.

For $0<X<M$, the log-likelihood, omitting terms that do not depend on $p$, is $\ell(p)=X\log p+(M-X)\log(1-p)$. Differentiating and setting the score to zero gives

\frac{d\ell}{dp}=\frac{X}{p}-\frac{M-X}{1-p}=0 \quad\Longrightarrow\quad X(1-p)=p(M-X) \quad\Longrightarrow\quad \widehat p=\frac{X}{M}.

The second derivative is $-X/p^2-(M-X)/(1-p)^2$, which is negative for $0<p<1$; the stationary point is a maximum. If $X=0$, the likelihood is maximized at the boundary $\widehat p=0$. If $X=M$, it is maximized at $\widehat p=1$. These boundary estimates are possible, but a standard-error approximation based on an interior estimate needs extra care in those cases.

Approximate standard error: Under the independent common-$p$ binomial model, the total $X$ has variance $Mp(1-p)$. Substituting $\widehat p$ for $p$ gives the estimated standard error $$ \widehat{\operatorname{SE}}(\widehat p)=\sqrt{\frac{\widehat p(1-\widehat p)}{M}}. $$ This is a large-sample approximation, not an exact statement about the sampling distribution of $\widehat p$. It can be unreliable when the estimated probability is near 0 or 1 or the total trial count is small.

Worked Examples

Worked Example: Pooling Groups of Different Sizes

A hypothetical insurer records whether each eligible policy had at least one claim during a defined policy year. In one portfolio segment, 16 of 800 policies had a claim. In a second segment with the same coverage and policy-year definition, 42 of 1,200 policies had a claim. Assuming independent trials with a common claim probability, estimate that probability. Also compare the pooled estimate with the unweighted average of the two segment rates.

Method. Treat each policy-year as one Bernoulli trial. The groups have different known trial counts, so pool successes and trials rather than averaging the two rates equally.

The segment rates are $16/800=0.0200$ and $42/1{,}200=0.0350$. The pooled counts are $X=16+42=58$ successes and $M=800+1{,}200=2{,}000$ trials. Thus,

\widehat p=\frac{58}{2{,}000}=0.0290.

The unweighted average of the segment rates is $(0.0200+0.0350)/2=0.0275$. It differs from the pooled estimate because it gives the 800-policy group the same weight as the 1,200-policy group. The estimated standard error under the common-$p$ model is

\widehat{\operatorname{SE}}(\widehat p) =\sqrt{\frac{0.0290(1-0.0290)}{2{,}000}} =\sqrt{\frac{0.028159}{2{,}000}} =\sqrt{0.0000140795} \approx 0.0038.

Check. The pooled estimate lies between the two group rates, closer to 0.0350 because the second group has more trials. The common mistake avoided here is averaging group rates without weighting by their trial counts.

Worked Example: Likelihood Estimate and Estimated Uncertainty

A hypothetical claims study divides a homogeneous book of business into three administrative batches. The batches contain 500, 1,000, and 1,500 one-year policy trials, with 9, 26, and 45 policies, respectively, having at least one claim. Estimate the common claim probability by maximum likelihood and calculate its approximate standard error.

Method. Use the grouped binomial likelihood. Each batch has a known number of trials, and the model assumption is that every policy-year has the same claim probability $p$, independently of the others.

The total number of successes is $X=9+26+45=80$. The total number of trials is $M=500+1{,}000+1{,}500=3{,}000$. The likelihood is proportional to $p^{80}(1-p)^{2{,}920}$, because $3{,}000-80=2{,}920$ trials had no claim. Maximizing it gives

\widehat p=\frac{80}{3{,}000}=0.026666\ldots\approx 0.0267.

Using the unrounded estimate in the standard-error calculation,

\widehat{\operatorname{SE}}(\widehat p) =\sqrt{\frac{(80/3{,}000)(1-80/3{,}000)}{3{,}000}} =\sqrt{\frac{(0.026666\ldots)(0.973333\ldots)}{3{,}000}} =\sqrt{0.00000865185\ldots} \approx 0.0029.

Check. The estimated probability is positive and below 1, as required for an interior maximum. The standard error is smaller than the estimate, which is plausible with 3,000 trials. The common mistake avoided is using the number of batches, three, as the denominator in either the pooled estimate or its standard error.

Worked Example: Different Observation Durations

A hypothetical study records at least one claim for 1,000 policies observed for six months and 1,000 policies observed for a full year. There are 15 policies with a claim in the six-month group and 40 in the year-long group. A proposed analysis pools all 2,000 policies as identical binomial trials. Calculate the pooled estimate and assess whether a common claim probability is justified by the trial definitions.

Method. First calculate the rates by observation duration. Then check whether a “trial” represents the same exposure period in both groups. A pooled arithmetic estimate can always be calculated, but it has the common-probability interpretation only if the trials are comparable.

The six-month rate is $15/1{,}000=0.0150$, and the one-year rate is $40/1{,}000=0.0400$. If all policies are mechanically pooled, the result is

\widehat p_{\text{pooled}}=\frac{15+40}{1{,}000+1{,}000} =\frac{55}{2{,}000}=0.0275.

Under a common probability of 0.0275, the expected number of claim-positive policies in each group of 1,000 would be $1{,}000(0.0275)=27.5$. The observed counts are 15 and 40. More importantly, the groups do not have the same trial definition: one records six months of exposure and the other records twelve. A policy observed for longer generally has more opportunity to experience a claim, so the common-$p$ assumption is not supported by the setup alone. The pooled value 0.0275 is a weighted average of the two observed rates, not automatically a probability for either duration.

Check. The arithmetic is correct, but its interpretation depends on the model. The common mistake avoided is treating unequal observation durations as equivalent trials merely because each row represents one policy. A suitable model must account for the exposure period, or the analysis must define comparable trial units before pooling.

When Common-Probability Pooling Is Appropriate

Different group sizes are compatible with a common binomial probability. For example, pooling 500 and 2,000 policy-years is appropriate if each policy-year is defined in the same way, the claim indicator has the same probability for every trial, and outcomes are independent. The sizes affect how much information each group contributes, not the assumed probability itself.

Different amounts of exposure require a more careful decision. If exposure is represented by a known integer number of comparable, independent trials, those trials can be counted in $M$. If instead a record covers a different duration, coverage, or risk level, one record may not represent one equivalent Bernoulli trial. A common probability is inappropriate when those differences change the probability of the recorded success. In particular, a one-year claim indicator and a six-month claim indicator are not interchangeable simply because both are binary outcomes.

A useful first diagnostic is to calculate $X_j/m_j$ separately for groups defined by exposure duration or risk characteristics. Noticeable differences do not by themselves prove the model false; random variation can produce different sample rates. But large or systematic differences, especially when they align with a known change in trial conditions, are a reason to question the shared-$p$ assumption before reporting a pooled estimate.

Define the trial.
State exactly what one success and one trial mean, including the observation period and coverage.

Confirm the common-probability assumption.
Ask whether the same probability applies to every trial and whether trials can reasonably be treated as independent.

Pool only comparable trials.
For a valid common-$p$ model, use total successes divided by total trials, and use the total trial count in the standard-error estimate.

Interpret the result in context.
If exposure changes the success probability, do not describe a mechanical pooled ratio as a shared claim probability without a model that addresses the difference.

Common Exam Traps

Averaging group rates instead of pooling counts. The pooled estimate is $\sum_j X_j/\sum_j m_j$, not the ordinary average of the $X_j/m_j$ values unless all groups have equal trial counts.
Confusing a policy record with a trial. A policy observed for six months and one observed for a year may have different claim probabilities. The binary outcome format does not make the trials equivalent.
Ignoring the event definition. A binomial model counts successes among trials. If the modeled observation is the number of claims per policy rather than whether a policy had any claim, the binomial setup may not match the data.
Using the wrong denominator for uncertainty. The estimated standard error uses the total number of comparable trials $M$, not the number of groups or the number of claims.
Treating a fitted value as proof of the assumptions. A likelihood estimate can be calculated even when groups are not comparable. The model assumptions still need to be checked and the estimate interpreted accordingly.

Key takeaway: For independent, comparable Bernoulli trials with a common probability, the binomial estimate is total successes divided by total trials. Known differences in the number of trials are handled by pooling; differences that change the probability per trial must be addressed before interpreting that pooled estimate.

Check Your Understanding

For each problem, identify the trial definition before choosing a calculation.

A hypothetical program records 12 claim-positive outcomes among 600 comparable policy-years in one region and 21 among 900 in another. Find the pooled probability estimate and the unweighted average of the two regional rates. (Answer: pooled estimate $=33/1{,}500=0.0220$; unweighted average $=(0.0200+0.0233\ldots)/2\approx0.0217$.)
A study has 2,400 comparable independent trials and 72 successes. Find the maximum-likelihood estimate and its approximate standard error. (Answer: $\widehat p=0.0300$; $\widehat{\operatorname{SE}}=\sqrt{0.03(0.97)/2{,}400}\approx0.0035$.)
Two comparable groups have 250 trials with 5 successes and 750 trials with 30 successes. Calculate the pooled estimate, then state why the unweighted average of group rates would be inappropriate. (Answer: $\widehat p=35/1{,}000=0.0350$; the unweighted average is $0.0300$, which gives the smaller group equal weight despite its fewer trials.)
A hypothetical portfolio has 800 policies observed for three months, with 8 claim-positive policies, and 800 policies observed for one year, with 28 claim-positive policies. Calculate the mechanical pooled rate and explain why it is not automatically a common per-policy probability. (Answer: $36/1{,}600=0.0225$; the observation durations differ, so the groups are not equivalent trials without an exposure-aware model.)
In 1,200 comparable independent trials, every trial is claim-free. State the maximum-likelihood estimate and explain one limitation of the usual plug-in standard-error formula. (Answer: $\widehat p=0$, a boundary maximum; the plug-in standard error is zero and does not adequately express uncertainty from a finite sample.)

What You'll Learn