Rare Binary Events Remain Bernoulli Outcomes

Rarity Changes Frequency, Not the Individual Outcome

A success can be uncommon without becoming a different kind of individual response. If each eligible trial records whether an event occurred, the recorded outcome is still either 1 (event) or 0 (no event). A small success probability means that 1 is expected infrequently; it does not change the possible values of the response.

This distinction matters in a generalized linear mixed model because the response family describes the distribution of an outcome on a specified observational unit. The unit here is one trial, and the response is binary. How often successes appear in a data set is a separate feature of the data. It can affect how much information is available for estimation, but it does not by itself change the individual-trial distribution.

Definition: An individual binary trial has response $Y_{ij}\in\{0,1\}$, where $i$ indexes trials and $j$ indexes clusters. With success probability $p_{ij}$, its conditional response distribution is Bernoulli: $$ Y_{ij}\mid b_j \sim \operatorname{Bernoulli}(p_{ij}), \qquad P(Y_{ij}=1\mid b_j)=p_{ij}. $$ The value 1 denotes the defined success event, and 0 denotes its complement. A rare event has a small success probability; rarity is not a third possible response value or a replacement distribution.

For a Bernoulli response, the probability mass function is $p_{ij}^{y_{ij}}(1-p_{ij})^{1-y_{ij}}$ for $y_{ij}\in\{0,1\}$. Its conditional mean is $p_{ij}$, and its conditional variance is $p_{ij}(1-p_{ij})$. These formulas remain in force when $p_{ij}$ is small. For example, if $p_{ij}=0.005$, the response is still either 0 or 1, and its variance is $0.005(0.995)=0.004975$.

Bernoulli Responses in a Clustered Model

To make the role of clusters explicit, consider a logistic random-intercept model. Let $x_{ij}$ be a vector of observed covariates and let $b_j$ be a cluster-specific random intercept. Conditional on that random effect and the covariates, the success probability is

\operatorname{logit}(p_{ij})=\log\!\left(\frac{p_{ij}}{1-p_{ij}}\right) =\mathbf{x}_{ij}^{\mathsf T}\boldsymbol{\beta}+b_j.

Here $\boldsymbol{\beta}$ contains fixed-effect coefficients. A standard random-intercept specification assumes $b_j\sim N(0,\sigma_b^2)$, with random effects independent across clusters, and assumes responses are conditionally independent given the random effects and covariates. The Bernoulli distribution is conditional on the cluster effect. A cluster effect can shift the success probability upward or downward, while each individual response remains binary.

A rare event need not have exactly the same probability for every trial. Covariates and cluster effects may produce different $p_{ij}$ values. The event may be rare overall even if some trials have higher probabilities than others; conversely, an overall low rate alone does not establish that every covariate pattern or cluster has a low conditional probability.

Worked Example: A Small Probability on Each Trial

Suppose, hypothetically, that each of 100 eligible trials has the same success probability $p=0.0025$, and outcomes are independent. For one trial, $Y_i\sim\operatorname{Bernoulli}(0.0025)$: its possible values are 0 and 1.

The expected number of successes across the 100 trials is $100(0.0025)=0.25$. The probability of observing no successes is

P(\text{no successes})=(1-0.0025)^{100} =0.9975^{100}\approx 0.7783.

Therefore, the probability of at least one success is $1-0.7783=0.2217$. It is quite plausible to see no successes in this small set of trials. That outcome does not show that the individual response ceased to be Bernoulli: each trial still had outcomes 0 or 1, with success probability 0.0025.

Why Integrating Over Clusters Does Not Change Binary Support

A GLMM distinguishes conditional and marginal descriptions. Conditional probabilities fix the random effect at a specified value. A marginal probability averages over the random-effect distribution. For a single binary response, this averaging changes the success probability but does not change the support of the response.

Result: If $Y\mid B\sim\operatorname{Bernoulli}(p(B))$, then marginally $Y\sim\operatorname{Bernoulli}(\bar p)$, where $\bar p=E[p(B)]$. This statement concerns the distribution of one response; it does not imply that multiple responses sharing $B$ are marginally independent.

The justification follows directly from the law of total probability. Because $Y$ can only be 0 or 1,

P(Y=1)=E\{P(Y=1\mid B)\}=E[p(B)]=\bar p, \qquad P(Y=0)=E\{1-p(B)\}=1-\bar p.

These are exactly the two probabilities of a Bernoulli distribution with success probability $\bar p$. But if two responses share the same random effect, they can be dependent after that effect is integrated out. Under conditional independence,

\operatorname{Cov}(Y_{ij},Y_{kj}) =E[p_{ij}(B)p_{kj}(B)] -E[p_{ij}(B)]E[p_{kj}(B)].

When both trials have the same conditional probability function $p(B)$, this reduces to $\operatorname{Var}\{p(B)\}$, which is nonnegative. Thus cluster heterogeneity can induce positive marginal association even though the outcomes are conditionally independent. Rarity does not remove this distinction between conditional and marginal behavior.

Worked Example: Rare Outcomes With Cluster Heterogeneity

Consider a hypothetical setting with two types of clusters. In one type, each trial has conditional success probability $0.02$; in the other, it has probability $0.08$. Suppose each type has probability $1/2$ in the population. Both conditional probabilities describe Bernoulli trials, and both are small in this example.

The marginal success probability for a randomly selected trial is

\bar p=\tfrac12(0.02)+\tfrac12(0.08)=0.05.

For 50 conditionally independent trials in a cluster of the first type, the expected number of successes is $50(0.02)=1$, and the probability of no successes is $0.98^{50}\approx0.3642$. For 50 trials in a cluster of the second type, those quantities are $50(0.08)=4$ and $0.92^{50}\approx0.0155$. The response for every individual remains 0 or 1. The different cluster probabilities affect how often successes occur and how outcomes within a cluster relate marginally; they do not turn an individual response into a different distribution.

Individual Trials and Grouped Records

When trials have a common probability and satisfy the relevant independence conditions, their success total can be represented with a binomial likelihood. That representation summarizes a group of Bernoulli outcomes; it does not redefine the response recorded on each trial. The earlier material on Bernoulli and binomial data representations distinguishes these individual and grouped forms. In a GLMM, grouping also requires care: a shared random effect can make responses conditionally independent but marginally dependent, so a grouped likelihood must reflect the model's conditioning and assumptions.

The key question is always what one row represents. If one row is one eligible trial and records whether the event happened, a rare event remains a binary response. If one record instead reports a number of events over a unit or time window, that is a different response definition and requires its own support and likelihood to be considered. The event being rare is not enough to decide that the recorded response is a count.

Worked Example: Aggregating Identically Distributed Trials

Suppose, hypothetically, that 400 independent eligible trials share the same success probability $p=0.01$. Let $K$ be the total number of successes. Each individual $Y_i$ is Bernoulli with probability 0.01, and the total has the binomial distribution $K\sim\operatorname{Binomial}(400,0.01)$.

The expected total and variance are

E(K)=400(0.01)=4, \qquad \operatorname{Var}(K)=400(0.01)(0.99)=3.96.

The chance of seeing no successes is

P(K=0)=(1-0.01)^{400}=0.99^{400}\approx0.0180.

The grouped total is useful for this calculation because the trials have a common probability and are independent under the stated hypothetical assumptions. The individual observations are nevertheless Bernoulli. In a clustered model, a common cluster effect can make the probabilities vary by cluster and create marginal dependence, so those assumptions must be checked before using this simple grouped expression.

What Rarity Can Affect

Although rarity does not change the response family, it can affect the information in a finite data set. If very few successes are observed, there may be little information for estimating how success probability changes across covariates or clusters. Some covariate patterns may contain no successes, or the sample may contain no successes at all. These are data and estimation difficulties, not evidence that the individual outcome has acquired a new support.

Before fitting or interpreting a model, check the construction of the response and the trial definition. Confirm that 1 denotes the intended event, 0 denotes its complement, and every recorded row corresponds to an eligible trial. Then examine the number of successes overall and across relevant covariate patterns and clusters. A very sparse pattern can make estimated relationships unstable, while an incorrectly defined denominator or event can make the likelihood represent the wrong process altogether.

The main alternatives are not interchangeable remedies for a low success frequency. Complementing the binary response changes which outcome is called success but preserves its information when the coding is consistent. A grouped representation summarizes eligible trials under its assumptions. A count response, in contrast, records a number of events on its observational unit; it is not justified merely because the event is rare. The earlier material on complementary binary outcomes covers the first distinction, while the data structure must determine the response representation.

Common Mistakes and Practical Checks

“The event is rare, so the response cannot be Bernoulli.” This confuses the frequency of 1s with the response support. For one trial, the outcome is still 0 or 1; a small $p_{ij}$ simply makes 1 less probable.
“No successes were observed, so the probability must be zero.” A finite sample can contain no successes even when the true probability is positive. For $n$ independent trials with common probability $p$, that probability is $(1-p)^n$, which can be substantial when $p$ is small or $n$ is limited.
“The marginal success probability describes every cluster.” The marginal probability averages over random effects. Conditional probabilities can differ across clusters; report which probability scale is being described.
“Conditional independence means marginal independence.” Responses sharing a random effect can be conditionally independent yet marginally associated. The random effect is part of the conditioning statement.
“A rare event should automatically be recorded as a count.” Choose the response according to the observational unit and recorded variable. Rarity alone does not change a trial-level 0/1 outcome into a count response.

Key takeaway: For an individual eligible trial, a binary event indicator remains Bernoulli whether successes are common or rare. In a GLMM, random effects can change conditional probabilities and induce marginal dependence, but they do not change the binary support of an individual response.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

A trial has success probability 0.003. What are the possible response values, and what does the small probability mean?
Under the logistic random-intercept model, what does the random intercept change, and what does it leave unchanged about an individual response?
For 200 independent trials with common success probability 0.01, calculate the expected number of successes and the probability of no successes.
Explain why a response can be marginally Bernoulli while two responses in the same cluster are marginally dependent.
A data set records the number of events during each day rather than a 0/1 indicator for each eligible trial. Does event rarity alone establish that the recorded response is Bernoulli? Explain.

Solutions. (1) The possible values are 0 and 1. The value 1 is uncommon, with probability 0.003, but the individual trial still has a Bernoulli response. (2) The random intercept shifts the trial's conditional log odds and therefore changes its conditional success probability; it does not change the response support $\{0,1\}$. (3) The expected number is $200(0.01)=2$. The probability of no successes is $0.99^{200}\approx0.1340$. (4) For one response, averaging its conditional success probability gives a marginal Bernoulli probability. For two responses sharing a random effect, their conditional probabilities co-vary across possible random-effect values, which can induce marginal dependence. (5) No. The recorded unit is a day and the response is an event total, not an individual trial indicator. Its distribution must be chosen from that response definition and its support; rarity alone does not make it Bernoulli.