Conditional Variance Benchmarks for Binomial and Poisson GLMs

Why a Variance Benchmark Is Needed

A fitted mean alone does not describe how much individual responses are expected to vary around that mean. In a generalized linear model (GLM), the response family supplies a reference relationship between the conditional mean and conditional variance. For Bernoulli, grouped-binomial, and Poisson responses, these relationships have specific forms, so the fitted means give a direct benchmark for the response variation under the stated model.

The word conditional matters. The variance is calculated at fixed covariates and, where applicable, fixed numbers of trials or exposure. It is not generally the variance obtained by pooling responses across different covariate values. Here, the focus is the variance assumption for each response and how to evaluate it using fitted means.

Definition: For response \(Y_i\), let \(\mu_i=E[Y_i\mid x_i]\), where \(x_i\) denotes the covariates. A conditional variance function \(V(\mu_i)\) specifies \(\operatorname{Var}(Y_i\mid x_i)\) as a function of the conditional mean, possibly also using known design quantities such as a trial count or exposure. The fitted reference variance is the value obtained by substituting the fitted mean \(\hat{\mu}_i\) and those known quantities into that variance function.

For the results below, take observations to be conditionally independent given their covariates and known design quantities. Conditional independence across observations is the usual GLM sampling assumption; the variance formulas for a single response also require the corresponding distributional assumptions stated in each case.

Bernoulli Responses

A Bernoulli response records one binary outcome: \(Y_i=1\) for a success and \(Y_i=0\) for a failure. At fixed \(x_i\), write its success probability as \(p_i\). Its conditional mean is \(p_i\), so the fitted mean is interpreted as a fitted success probability.

Definition: A Bernoulli response satisfies \(P(Y_i=1\mid x_i)=p_i\) and \(P(Y_i=0\mid x_i)=1-p_i\). Its conditional mean is \(\mu_i=p_i\), and its conditional variance is \(p_i(1-p_i)=\mu_i(1-\mu_i)\). The fitted reference variance is \(\hat{\mu}_i(1-\hat{\mu}_i)\).

The formula follows directly from the two possible response values. Since \(Y_i^2=Y_i\), \(E[Y_i^2\mid x_i]=p_i\). Therefore, \(\operatorname{Var}(Y_i\mid x_i)=E[Y_i^2\mid x_i]-E[Y_i\mid x_i]^2=p_i-p_i^2=p_i(1-p_i)\). The variance is zero at probabilities zero and one and is largest at \(p_i=1/2\), where it equals \(1/4\).

Worked Example: Bernoulli Fitted Probabilities

Suppose three patients have fitted probabilities of a binary event of \(0.20\), \(0.50\), and \(0.80\). For each patient, calculate the fitted reference variance of the binary response.

For the first patient, \(\hat{\mu}_1=0.20\), so the variance is \(0.20(1-0.20)=0.20(0.80)=0.16\).

For the second patient, \(\hat{\mu}_2=0.50\), so the variance is \(0.50(1-0.50)=0.25\).

For the third patient, \(\hat{\mu}_3=0.80\), so the variance is \(0.80(1-0.80)=0.16\). The first and third fitted probabilities are equally far from \(0.50\), and their reference variances are equal. These are variances of the individual zero-or-one responses, not variances of fitted probabilities across patients.

Grouped-Binomial Counts and Proportions

A grouped-binomial response counts successes among a known number \(m_i\) of trials. Its variance depends on both the success probability and the number of trials. The count and the observed proportion have different variances because one is \(m_i\) times the other.

Definition: If \(Y_i\mid x_i\sim\operatorname{Bin}(m_i,p_i)\), where \(m_i\) is a known positive integer and the trials are conditionally independent with the same success probability \(p_i\), then \(E[Y_i\mid x_i]=\mu_i=m_i p_i\) and \(\operatorname{Var}(Y_i\mid x_i)=m_i p_i(1-p_i)\). For the observed proportion \(R_i=Y_i/m_i\), the conditional mean is \(p_i\) and the conditional variance is \(p_i(1-p_i)/m_i\).

The derivation uses the independent-trial assumption. Write \(Y_i=\sum_{j=1}^{m_i}B_{ij}\), where each \(B_{ij}\) is a Bernoulli indicator with conditional success probability \(p_i\). Linearity of expectation gives \(E[Y_i\mid x_i]=m_i p_i\). Conditional independence makes the covariances between distinct indicators zero, so the variance of their sum is the sum of their variances.

\begin{aligned} \operatorname{Var}(Y_i\mid x_i) &=\sum_{j=1}^{m_i}\operatorname{Var}(B_{ij}\mid x_i)\\ &=m_i p_i(1-p_i). \end{aligned}

Since \(R_i=Y_i/m_i\), scaling a random variable by \(1/m_i\) scales its variance by \(1/m_i^2\). Consequently, \(\operatorname{Var}(R_i\mid x_i)=m_i p_i(1-p_i)/m_i^2=p_i(1-p_i)/m_i\). This proves both grouped-binomial variance formulas under the stated assumptions. \(\square\)

In terms of the expected count \(\mu_i=m_i p_i\), the count variance can also be written as \(\mu_i(1-\mu_i/m_i)\). If a model is fitted to the count, then \(\hat{p}_i=\hat{\mu}_i/m_i\), and its reference variance is \(\hat{\mu}_i(1-\hat{\mu}_i/m_i)\). If it is fitted to the proportion, the fitted mean is \(\hat{p}_i\), and its reference variance is \(\hat{p}_i(1-\hat{p}_i)/m_i\).

Worked Example: A Grouped-Binomial Count

A clinic records the number of patients who respond to a treatment among \(m=20\) patients in each group. For one group, suppose the fitted response probability is \(\hat{p}=0.40\). Calculate the fitted mean and reference variance for both the count and the proportion.

For the count \(Y\), the fitted mean is \(\hat{\mu}=m\hat{p}=20(0.40)=8\). Its fitted reference variance is \(m\hat{p}(1-\hat{p})=20(0.40)(0.60)=4.8\).

For the proportion \(R=Y/20\), the fitted mean is \(0.40\). Its reference variance is \(\hat{p}(1-\hat{p})/m=(0.40)(0.60)/20=0.24/20=0.012\).

These values agree with the scaling relationship: \(4.8/20^2=4.8/400=0.012\). The count and proportion describe the same group, but the numerical variance depends on which response scale is modeled.

Poisson Responses

For a Poisson response, the conditional mean itself is the conditional variance. When the response is a count observed over an exposure interval, the mean is the expected count over that exposure, not necessarily the event rate per unit exposure.

Definition: If \(Y_i\mid x_i\sim\operatorname{Poisson}(\mu_i)\), with \(\mu_i>0\), then \(E[Y_i\mid x_i]=\mu_i\) and \(\operatorname{Var}(Y_i\mid x_i)=\mu_i\). The fitted reference variance is therefore \(\hat{\mu}_i\).

To verify the variance from the Poisson probability mass function, use the factorial moments. For \(Y\sim\operatorname{Poisson}(\mu)\), the probability of \(Y=y\) is \(e^{-\mu}\mu^y/y!\), for \(y=0,1,2,\ldots\). Summing gives

\begin{aligned} E[Y] &=\sum_{y=1}^{\infty}y e^{-\mu}\frac{\mu^y}{y!} =\mu e^{-\mu}\sum_{k=0}^{\infty}\frac{\mu^k}{k!} =\mu,\\ E[Y(Y-1)] &=\sum_{y=2}^{\infty}y(y-1)e^{-\mu}\frac{\mu^y}{y!} =\mu^2 e^{-\mu}\sum_{k=0}^{\infty}\frac{\mu^k}{k!} =\mu^2. \end{aligned}

The sums are the exponential series, \(e^\mu\), and are finite for every \(\mu>0\). Because \(Y^2=Y(Y-1)+Y\), it follows that \(E[Y^2]=\mu^2+\mu\). Hence \(\operatorname{Var}(Y)=E[Y^2]-E[Y]^2=(\mu^2+\mu)-\mu^2=\mu\). Substituting the fitted mean gives the fitted reference variance. \(\square\)

Worked Example: Poisson Counts With Unequal Exposure

Suppose event counts are modeled with a common fitted rate of \(2.4\) events per unit of exposure. One observation has exposure \(t_1=3.5\) units and another has exposure \(t_2=1.0\) unit. Under a Poisson rate model, the fitted mean count is exposure times rate.

For the first observation, \(\hat{\mu}_1=3.5(2.4)=8.4\). Its fitted reference variance is \(8.4\), and its reference standard deviation is \(\sqrt{8.4}\), approximately \(2.90\).

For the second observation, \(\hat{\mu}_2=1.0(2.4)=2.4\). Its fitted reference variance is \(2.4\), and its reference standard deviation is \(\sqrt{2.4}\), approximately \(1.55\).

The exposure changes the expected count and therefore the variance benchmark. The rate \(2.4\) is not itself the variance of either count: the Poisson benchmark is the fitted expected count for the observation’s exposure.

Using Fitted Means on the Correct Scale

A GLM represents the conditional mean as a function of covariates, commonly through a link function. For these variance calculations, it is the fitted response mean \(\hat{\mu}_i\) that matters, rather than the linear predictor before it has been transformed by the inverse link. The appropriate conversion depends on the response:

Response	Conditional mean	Conditional variance	Fitted reference variance
Bernoulli indicator	\(\mu_i=p_i\)	\(\mu_i(1-\mu_i)\)	\(\hat{\mu}_i(1-\hat{\mu}_i)\)
Grouped-binomial count	\(\mu_i=m_i p_i\)	\(m_i p_i(1-p_i)\)	\(\hat{\mu}_i(1-\hat{\mu}_i/m_i)\)
Grouped-binomial proportion	\(\mu_i=p_i\)	\(p_i(1-p_i)/m_i\)	\(\hat{\mu}_i(1-\hat{\mu}_i)/m_i\)
Poisson count	\(\mu_i\)	\(\mu_i\)	\(\hat{\mu}_i\)

These are conditional statements at fixed covariates and design quantities. For instance, even if every Poisson response has conditional variance equal to its conditional mean, the unconditional variance across observations with differing means need not equal the unconditional mean. The law of total variance gives

\operatorname{Var}(Y_i) =E[\operatorname{Var}(Y_i\mid x_i)] +\operatorname{Var}(E[Y_i\mid x_i]).

Thus variation in the conditional means across covariate values contributes to marginal variation as well. This is why the reference variance for a particular observation must use that observation’s fitted mean, rather than a pooled mean that ignores its covariates.

Common Mistakes and Practical Meaning

Using the wrong response scale. The variance \(m p(1-p)\) applies to the grouped count, while \(p(1-p)/m\) applies to the proportion. Dividing or multiplying without accounting for the squared scale factor gives the wrong variance.
Using the rate as a Poisson count variance. With exposure \(t_i\) and rate \(\lambda_i\), the fitted mean count is \(t_i\lambda_i\). The Poisson reference variance is \(t_i\lambda_i\), not \(\lambda_i\), unless the exposure is one unit.
Treating a grouped count as one Bernoulli response. A count out of \(m_i\) trials has variance \(m_i p_i(1-p_i)\), not \(p_i(1-p_i)\). The latter is the variance for one Bernoulli response, or the numerator used when deriving the proportion variance.
Confusing conditional and marginal variation. The reference variance is evaluated at fixed covariates and design quantities. Variation in fitted means across observations is a separate contribution to overall variability.

These benchmarks matter because they make the variance assumption concrete. For every fitted observation, they give a response-scale-specific reference value that can be calculated from the fitted mean and, where relevant, the trial count or exposure. Interpreting that value correctly is a necessary first step before using residual-based tools to assess how well a fitted GLM describes the response variation.

Check Your Understanding

For each question, identify the response scale and the quantities needed before calculating a reference variance.

A Bernoulli response has fitted probability \(0.30\). What is its fitted reference variance?
A grouped-binomial count has \(m=12\) trials and fitted probability \(0.25\). Calculate the fitted count mean and count variance.
For the grouped-binomial setting in Question 2, what is the fitted variance of the observed proportion?
A Poisson model has fitted rate \(1.8\) per day and an observation has exposure \(4\) days. What are its fitted count mean and reference variance?
Why can the marginal variance across observations with differing covariates exceed the average conditional variance, even when the conditional variance assumptions hold?