Variance And Prediction Intervals For Event Counts

Why a Projected Count Needs an Uncertainty Measure

A projected event count at a planned cutoff is not usually a known number. Even if enrollment and follow-up are specified, each participant may or may not experience the event by that date. A projection therefore has two useful parts: a summary of the expected count and a description of the count’s variability. The latter matters for planning: a date when the expected count equals the target does not guarantee that the target will have been reached.

The focus here is a fixed set of participants and a fixed cutoff. For participant $i$, let $Y_i=1$ if the participant has an event by the cutoff and $Y_i=0$ otherwise. The projected count is $D=\sum_{i=1}^{n}Y_i$. The probability $p_i=P(Y_i=1)$ is the participant’s event probability by that cutoff, based on the planned enrollment time, follow-up, and event assumptions. The earlier material on averaging event probabilities across enrollment times gives the basis for obtaining such probabilities when participants have different entry times.

Definition: A prediction interval for an event count is a range of possible counts, calculated from a predictive distribution for $D$, that contains the future count with a stated probability under the model. Its coverage describes repeated-trial behavior under the assumed event probabilities and other model conditions; it is not a confidence interval for an unknown mean.

Mean and Variance Under the Event-Count Model

The basic assumptions for the calculations below are that the cohort and cutoff are fixed, each $Y_i$ is a Bernoulli indicator with the specified probability $p_i$, and the indicators are independent across participants. Independence means that, under the model, knowing one participant’s event status does not change another participant’s event probability. These assumptions may be reasonable for a simple projection, but shared risks, site effects, or other dependence can make them inadequate.

Since $E(Y_i)=p_i$ and $\operatorname{Var}(Y_i)=p_i(1-p_i)$, linearity of expectation gives the mean count. Independence makes the covariances zero, so the variance is the sum of the individual variances.

E(D)=\sum_{i=1}^{n}p_i,\qquad \operatorname{Var}(D)=\sum_{i=1}^{n}p_i(1-p_i),\qquad SD(D)=\sqrt{\operatorname{Var}(D)}.

If all participants have the same event probability $p$, then $D\sim\operatorname{Binomial}(n,p)$, $E(D)=np$, and $\operatorname{Var}(D)=np(1-p)$. When the probabilities differ, the count has a Poisson-binomial distribution rather than an ordinary binomial distribution. Its mean and variance still follow from the sums above, but the full distribution is not determined by the mean and variance alone.

Justification: Write $D=\sum_iY_i$. Then $E(D)=\sum_iE(Y_i)=\sum_ip_i$. Also, $$ \operatorname{Var}(D)=\sum_i\operatorname{Var}(Y_i)+2\sum_{i<j}\operatorname{Cov}(Y_i,Y_j). $$ Under independence, $\operatorname{Cov}(Y_i,Y_j)=0$ for $i\ne j$, and each Bernoulli variance is $p_i(1-p_i)$. This yields the stated variance formula.

Worked Examples: From a Count Distribution to a Planning Decision

Worked Example: A Binomial Projection and Its Prediction Interval

Suppose, hypothetically, that 20 participants each have probability $p=0.30$ of experiencing the event by a fixed cutoff. Assume their event indicators are independent and have the same probability. Find the projected mean, variance, standard deviation, and an equal-tail 95% prediction interval.

Under these assumptions, $D\sim\operatorname{Binomial}(20,0.30)$. Therefore,

E(D)=np=20(0.30)=6.00\text{ events}, \qquad \operatorname{Var}(D)=np(1-p)=20(0.30)(0.70)=4.20\text{ events}^2.

Thus $SD(D)=\sqrt{4.20}\approx2.049$ events. This standard deviation describes the spread of projected counts around the mean under the model; it is not an uncertainty measure for an individual participant’s probability.

For a discrete equal-tail interval, use the lower and upper quantiles of the binomial distribution. The lower endpoint is the smallest integer $k$ with $P(D\leq k)\geq0.025$, and the upper endpoint is the smallest integer $k$ with $P(D\leq k)\geq0.975$. Here, $P(D\leq1)\approx0.00764$, $P(D\leq2)\approx0.0355$, $P(D\leq9)\approx0.9520$, and $P(D\leq10)\approx0.9829$. The interval is consequently 2 to 10 events. Its actual coverage is

P(2\leq D\leq10) =P(D\leq10)-P(D\leq1) \approx0.982855-0.007637 =0.975218.

So this interval covers approximately 97.52% of counts under the assumed model, not exactly 95%. Discreteness often makes an equal-tail prediction interval more conservative than its nominal level.

Worked Example: Different Probabilities and the Count Distribution

Consider three hypothetical participants whose event probabilities by the cutoff are, in order, $0.20$, $0.40$, and $0.80$. Assume their event indicators are independent. Calculate the mean and variance, then find a prediction range with at least 90% probability.

The expected count is $0.20+0.40+0.80=1.40$ events. The variance and standard deviation are

\operatorname{Var}(D)=0.20(0.80)+0.40(0.60)+0.80(0.20)=0.56\text{ events}^2, \qquad SD(D)=\sqrt{0.56}\approx0.7483\text{ events}.

To obtain the exact count probabilities, multiply the relevant event and non-event probabilities. The probability of zero events is $0.80(0.60)(0.20)=0.096$. Exactly one event can occur in three mutually exclusive ways, giving $0.20(0.60)(0.20)+0.80(0.40)(0.20)+0.80(0.60)(0.80)=0.472$. Exactly two events have probability $0.20(0.40)(0.20)+0.20(0.60)(0.80)+0.80(0.40)(0.80)=0.368$, and three events have probability $0.20(0.40)(0.80)=0.064$. These probabilities sum to 1.

The range 1 to 3 events has probability $0.472+0.368+0.064=0.904$, so it is a prediction range with 90.4% coverage. It is not equal-tailed: all of the excluded probability, 9.6%, is at zero. This illustrates that discrete prediction ranges can be selected in different ways. State the interval rule as well as its resulting coverage, especially when selecting the narrowest range rather than equal-tail quantiles.

Worked Example: Using the Distribution to Assess a Target Date

Suppose, hypothetically, that 12 participants are expected to have independent probabilities of $0.25$ of an event by a proposed cutoff. What is the chance of reaching a target of at least four events by that cutoff, and what is an equal-tail 95% prediction interval?

The model is $D\sim\operatorname{Binomial}(12,0.25)$. Its mean is $12(0.25)=3.00$ events, its variance is $12(0.25)(0.75)=2.25\text{ events}^2$, and its standard deviation is $1.50$ events. To assess the target, calculate the lower-tail probability and subtract it from 1:

P(D\geq4)=1-P(D\leq3) \approx1-0.648781=0.351219.

Under these assumptions, the probability of having at least four events by the proposed cutoff is approximately 35.12%. The expected count of three events alone would not show this chance of meeting the target.

For the equal-tail interval, $P(D\leq0)\approx0.0317$, which exceeds 0.025, while $P(D\leq5)\approx0.9456$ and $P(D\leq6)\approx0.9858$. The smallest upper endpoint reaching 0.975 is therefore 6, giving an interval of 0 to 6 events. Its coverage is approximately 98.58%. This interval and the target-attainment probability answer different questions: the interval describes a range of plausible counts, while $P(D\geq4)$ directly addresses whether the target is met.

Exact Probabilities, Simulation, and Approximations

When all participants share $p$, binomial probabilities provide the count distribution directly. With differing $p_i$, the exact Poisson-binomial distribution can be calculated by combining the Bernoulli distributions, as in the three-participant example. For a larger cohort, simulation provides a practical alternative: in each replicate, generate an independent Bernoulli outcome for every participant using that participant’s $p_i$, sum the outcomes, and repeat. Quantiles of the simulated totals approximate prediction-interval endpoints, and the fraction of totals meeting a target approximates its probability of attainment.

Simulation must represent the intended source of uncertainty. If the $p_i$ values are treated as fixed, simulated variation describes only the future event indicators conditional on those probabilities. It does not account for uncertainty in estimated event rates or other model parameters. If a planning analysis needs to reflect parameter uncertainty, that uncertainty must also be represented in the predictive calculation. In general, the law of total variance separates these components:

\operatorname{Var}(D) =E\{\operatorname{Var}(D\mid\boldsymbol{p})\} +\operatorname{Var}\{E(D\mid\boldsymbol{p})\},

where $\boldsymbol{p}$ denotes the probabilities treated as uncertain. Omitting the second term gives a conditional variance and may understate predictive uncertainty when the probabilities themselves are uncertain.

Approach	Useful when	Main caution
Exact binomial calculation	Participants have a common event probability and independent outcomes	Does not fit differing probabilities or dependence without modification
Exact Poisson-binomial calculation	Probabilities differ and the cohort is manageable	Requires the individual probabilities and independence assumption
Simulation of event indicators	The count distribution is cumbersome to calculate exactly	Approximate quantiles have simulation error; the simulated model must include the uncertainty of interest
Normal approximation	The count distribution is sufficiently spread and not strongly skewed	Can give poor or impossible endpoints for small, skewed, or boundary-heavy counts

A normal approximation might use $E(D)\pm1.96SD(D)$, with endpoints rounded to integer counts. This shortcut is not automatically a 95% prediction interval: discreteness, skewness, and the count’s bounds can make its coverage inaccurate. Exact probabilities or a sufficiently large simulation are preferable when practical, particularly when the planning decision depends on a tail probability such as $P(D\geq d^*)$.

Diagnostics, Limitations, and a Common Pitfall

Before relying on a projected interval, check that the probabilities correspond to the actual proposed cutoff and planned participant follow-up; that loss to follow-up and administrative censoring have been handled consistently; and that the independence and common- or differing-probability structure match the calculation. Compare the projected mean and spread with plausible event patterns. A variance of zero when some probabilities are strictly between 0 and 1, for example, signals a calculation or input error under the independent Bernoulli model.

The formulas condition on the specified cohort and cutoff. They do not, by themselves, account for uncertainty in how many participants will enroll, changes in event rates, dependence between participants, or delays in event adjudication. They also do not find the calendar date at which a target is reached. They quantify counts at a specified cutoff and can inform date planning by showing the probability of meeting a target there.

Common mistake: treating the expected count as a guaranteed count. If the expected count is 40, it does not follow that the trial will have 40 events at the cutoff. The expected value is a long-run average; the predictive distribution describes variation around it.
Common mistake: using a binomial variance with unequal probabilities. Replacing distinct $p_i$ values by their average can produce the wrong variance and count distribution. Under independence, use $\sum_i p_i(1-p_i)$ and an appropriate Poisson-binomial calculation or simulation.
Common mistake: calling a prediction interval a confidence interval. A prediction interval concerns a future realized count under a model. A confidence interval concerns an unknown parameter. Their interpretations and sources of uncertainty differ.
Common mistake: reporting nominal coverage without checking actual coverage. Discrete endpoints need not yield exactly the stated probability. Calculate or estimate the probability of the reported range under the model.

Key takeaway: For independent event indicators with fixed probabilities, the projected count has mean $\sum_i p_i$ and variance $\sum_i p_i(1-p_i)$. Use the count distribution—not just its mean—to describe predictive uncertainty and assess the chance of meeting an event target by a specified cutoff.

Check Your Understanding

Answer each question using the stated event-count model. Solutions follow.

For 10 independent participants with common event probability 0.20 by a fixed cutoff, find the mean and standard deviation of the projected count.
Two independent participants have event probabilities 0.10 and 0.50. Find the probability of exactly one event.
Explain why a projected mean of 8 events does not imply that at least 8 events will occur.
A 95% prediction interval has actual model-based coverage of 97%. Is that necessarily an error? Explain.
Name one source of predictive uncertainty not included when the participant probabilities are treated as fixed.

Solutions: (1) $E(D)=10(0.20)=2.00$ events and $\operatorname{Var}(D)=10(0.20)(0.80)=1.60\text{ events}^2$, so $SD(D)=\sqrt{1.60}\approx1.265$ events. (2) Exactly one event occurs if the first participant has the event and the second does not, or vice versa: $0.10(0.50)+0.90(0.50)=0.50$. (3) The mean is an average across repetitions of the modeled process, not a minimum or guaranteed realized count; counts vary according to their predictive distribution. (4) No. Discrete endpoints can make actual coverage exceed nominal coverage; report the calculated coverage. (5) Examples include uncertainty in estimated event rates, a random enrollment count, or dependence between participants.