Why Specify Variance Separately?
A marginal mean model describes the expected response at given covariate values. It does not, by itself, describe how variable individual responses are around that mean. Nor does it describe whether two responses from the same person, hospital, or other cluster tend to move together. These are distinct features of the joint data: the marginal mean, the marginal variance of each response, and the covariance between responses.
For a response \(Y_{ij}\) from cluster \(i\) at measurement \(j\), let \(X_i\) denote the observed covariates relevant to that cluster. Write its marginal mean and variance, conditional on those covariates, as
A marginal variance specification relates the second quantity to the first. It is a statement about the distribution of one response at a time, conditional on the covariates. It is not a statement about the association of two responses.
The convention for \(V\) matters: any constant dispersion factor is placed in \(\phi\), not absorbed into \(V\). For a Bernoulli response, the variance is fixed by the mean, so \(\phi=1\). For other response families, \(\phi\) may be fixed by a distributional model or treated as an additional scale parameter. A variance function is a moment specification; by itself, it need not identify the full response distribution.
Binary Responses: The Mean Determines the Variance
If \(Y\) is binary, it takes values 0 and 1. Let \(p=P(Y=1\mid X)\), so that \(\mu=E[Y\mid X]=p\). The variance is therefore determined by the mean alone.
To verify the result, use \(Y^2=Y\), which holds for both possible values of a binary response:
This variance is largest at \(\mu=1/2\), where it equals \(1/4\), and approaches zero as \(\mu\) approaches either 0 or 1. That behavior is a direct consequence of binary outcomes: responses become less variable when nearly all their probability is concentrated on one value.
Worked Example: Binary Variance at a Predicted Risk
Suppose a marginal binary model gives a predicted probability of \(0.30\) for an outcome at specified covariate values. Find the marginal variance of the response.
The mean is \(\mu=0.30\). For a Bernoulli response,
As a check, the response equals 1 with probability \(0.30\) and 0 with probability \(0.70\), so its variance is \(E[Y^2]-E[Y]^2=0.30-(0.30)^2=0.21\). This is the variance of one response at those covariate values, not a covariance between two measurements.
Count Responses: Equal Means Need Not Mean Equal Variances
The Poisson distribution has mean \(\mu\) and variance \(\mu\), giving \(V(\mu)=\mu\) and \(\phi=1\). Count data may instead be more variable than a Poisson model allows. One common alternative is the negative-binomial variance specification \(\operatorname{Var}(Y\mid X)=\mu+\kappa\mu^2\), where \(\kappa>0\). This form is often expressed using \(\phi=1\) and \(V(\mu)=\mu+\kappa\mu^2\), or using a separate scale convention; the stated formula makes the parameterization explicit.
Here is a direct derivation. For a Poisson random variable, \(P(Y=y)=e^{-\mu}\mu^y/y!\), for \(y=0,1,\ldots\). Its second factorial moment is
where \(r=y-2\) and the exponential series gives \(\sum_{r=0}^{\infty}\mu^r/r!=e^\mu\). Since \(Y^2=Y(Y-1)+Y\) and \(E[Y]=\mu\), it follows that \(E[Y^2]=\mu^2+\mu\). Therefore
Worked Example: Comparing Poisson and Negative-Binomial Variances
Two count models both have marginal mean \(\mu=2.4\). The first uses a Poisson variance; the second uses the negative-binomial form with \(\kappa=0.5\). Calculate and compare the variances.
For the Poisson model,
For the negative-binomial variance specification,
The means agree, but the variances do not: \(5.28\) is larger than \(2.4\). A mean model alone cannot tell us which variance relation is appropriate. The choice is a separate modeling assumption about the marginal spread of counts.
Continuous Responses: Several Common Variance Functions
There is no single variance function for continuous outcomes. The appropriate relation depends on the response model and the assumed way variability changes with the mean. Common examples include a Gaussian response with constant variance, a Gamma response with variance proportional to the square of its mean, and an inverse-Gaussian response with variance proportional to the cube of its mean.
| Response model | Variance specification | Typical support |
|---|---|---|
| Gaussian | \(\operatorname{Var}(Y\mid X)=\phi\), so \(V(\mu)=1\) | Real line |
| Gamma | \(\operatorname{Var}(Y\mid X)=\phi\mu^2\), so \(V(\mu)=\mu^2\) | Positive values |
| Inverse Gaussian | \(\operatorname{Var}(Y\mid X)=\phi\mu^3\), so \(V(\mu)=\mu^3\) | Positive values |
These are distribution-family variance relations under their usual parameterizations. In a marginal model, they can also serve as moment assumptions without asserting that the full conditional response distribution exactly belongs to that family. The relation should be plausible for the response scale and mean range being studied.
Worked Example: Variance of a Positive Continuous Outcome
At given covariate values, suppose a positive continuous response has mean \(\mu=10\) and is modeled with a Gamma variance function with \(\phi=0.20\). Calculate its variance and standard deviation.
The Gamma variance is \(\phi\mu^2\), so
The standard deviation is \(\sqrt{20}\approx4.47\). For comparison, if a Gaussian model had the same mean but constant variance \(\phi=0.20\), its variance would be \(0.20\), not \(20\). The values of \(\phi\) have different implications when paired with different variance functions; they should not be compared without stating the full variance specification.
Marginal Variance Is Not Within-Cluster Covariance
Even if every response has a specified marginal mean and variance, the joint distribution of responses in a cluster is not determined. For distinct measurements \(j\) and \(k\), their conditional covariance is
This quantity concerns how two responses vary together. The marginal variance concerns the spread of each response separately. The following result makes the distinction concrete for binary outcomes.
To prove this, let \(Y_1,Y_2\) each have Bernoulli mean \(p\), and write \(q=P(Y_1=1,Y_2=1)\). Their joint probabilities must be
All four probabilities are nonnegative precisely when \(\max(0,2p-1)\le q\le p\). The covariance is
Thus, when \(0<p<1\), different allowable choices of \(q\) give different covariances while leaving both marginal means and variances unchanged. For example, at \(p=1/2\), \(q=1/4\) gives covariance \(0\), whereas \(q=1/2\) gives covariance \(1/4\). Both joint distributions have the same Bernoulli marginals, so their individual variances are \(1/4\). This proves that marginal variance assumptions alone do not determine within-cluster dependence. \(\square\)
Worked Example: Same Binary Margins, Different Dependence
Let two repeated binary responses each have success probability \(1/2\). Compare an independent pair with a perfectly concordant pair, where the two responses always agree.
Under independence, each of the four outcome pairs has probability \(1/4\). In particular, \(q=P(Y_1=1,Y_2=1)=1/4\). Each marginal variance is \((1/2)(1/2)=1/4\), and the covariance is
For a perfectly concordant pair, \(P(1,1)=1/2\) and \(P(0,0)=1/2\), with the two discordant outcomes having probability zero. Now \(q=1/2\), while each response still has mean \(1/2\) and variance \(1/4\). The covariance is
The marginal mean and variance are identical in the two cases, but the within-pair covariance differs. A variance function cannot distinguish these dependence patterns.
Common Pitfall and Practical Interpretation
A common mistake is to reason, “The model has the correct mean, so its variance and dependence must also be correct.” That conclusion does not follow. Different response distributions can share a mean but have different variances, as the Poisson and negative-binomial calculation showed. And even fixed marginal means and variances do not uniquely determine the covariance between responses in a cluster, as the binary construction showed.
Keep the three questions separate: What is the expected response at the covariates? How variable is one response at those covariates? How do responses from the same cluster co-vary? A mean specification answers the first question. A variance function with its dispersion convention answers the second. A further joint-dependence specification is needed to answer the third. This separation is especially important when interpreting a marginal model: a variance choice may describe the marginal spread without making a claim that all responses are independent.
- Binary: the Bernoulli mean fixes the variance at \(\mu(1-\mu)\).
- Counts: Poisson variance equals the mean, while alternatives such as the negative-binomial form allow more spread.
- Continuous responses: Gaussian, Gamma, and inverse-Gaussian models use different mean-variance relations.
- Clusters: the variance of each response and the covariance between responses are distinct quantities.
Check Your Understanding
Use the mean-variance relations and distinctions developed above.
- A Bernoulli response has marginal mean \(0.65\). What is its marginal variance?
- At mean \(3\), calculate the Poisson variance and the negative-binomial variance when \(\kappa=0.4\). Why do they differ?
- A Gamma response has mean \(8\) and \(\phi=0.10\). Find its variance under the stated variance function.
- Two repeated binary responses each have mean \(1/2\) and variance \(1/4\). Does that information determine their covariance? Explain.
- In \(\operatorname{Var}(Y_{ij}\mid X_i)=\phi V(\mu_{ij})\), identify the roles of \(\phi\) and \(V\), and state what this equation does not specify.