Why Specify Moments Without a Full Distribution?
A probability model for a response can be specified in more than one way. A full distribution describes all possible response values and their probabilities. A mean-variance model specifies only the conditional mean and variance, leaving the rest of the distribution open. This can be useful when a familiar distribution captures the mean pattern but its variance is too restrictive, or when a probability law is not needed for the immediate modeling task.
The distinction is important: specifying a mean and variance does not, by itself, specify a likelihood. In particular, a mean-variance model does not tell us the probabilities of individual outcomes, nor does it imply conditional independence among responses. Those assumptions must be stated separately.
The conditioning is on the full covariate vector throughout. This convention matters when covariates are random: conditioning on all covariates can change a response's conditional distribution relative to conditioning on only its own covariate. If the design is fixed, one can read the conditional statements as statements at that fixed design.
A common additional assumption is that \(Y_1,\ldots,Y_n\) are conditionally independent given \(X\). It is not part of the definition of the moment model. Nor do the individual moment statements imply independence. For a model that makes no independence assumption, conditional covariances may remain nonzero and must be accounted for in variances of sums.
What Conditional Independence Adds
The distinction between marginal moments and the variance of an aggregate follows directly from the conditional covariance identity. For any responses with finite conditional second moments, $$ \operatorname{Var}\!\left(\sum_{i=1}^n Y_i\mid X\right) =\sum_{i=1}^n\operatorname{Var}(Y_i\mid X) +2\sum_{i<j}\operatorname{Cov}(Y_i,Y_j\mid X). $$ Thus conditional independence given \(X\) makes the covariance terms zero. Under the mean-variance assumptions above, it yields the following result.
Proof. Conditional independence implies \(\operatorname{Cov}(Y_i,Y_j\mid X)=0\) for each distinct pair \(i,j\). Expanding the conditional variance of the sum gives $$ \begin{aligned} \operatorname{Var}\!\left(\sum_{i=1}^nY_i\mid X\right) &=\sum_{i=1}^n\operatorname{Var}(Y_i\mid X) +2\sum_{i<j}\operatorname{Cov}(Y_i,Y_j\mid X)\\ &=\sum_{i=1}^n\operatorname{Var}(Y_i\mid X)\\ &=\sum_{i=1}^n\phi V(\mu_i) =\phi\sum_{i=1}^nV(\mu_i). \end{aligned} $$ Every conditional moment in this calculation is taken given the same full vector \(X\). \(\square\)
Without conditional independence, the final equality generally fails. Even if the individual conditional variances are specified, the conditional covariances of distinct responses are extra quantities. This is especially relevant when multiple observations arise from the same person, household, clinic, or other cluster.
Worked Example: Poisson-Like Counts With Extra Variation
Worked Example: Poisson-Like Counts With Extra Variation
Suppose counts \(Y_i\) have conditional means \(\mu_i=\exp(x_i^\mathsf{T}\beta)\) and variance function \(V(\mu)=\mu\), with \(\phi=1.5\). Assume the moment conditions and conditional independence hold given the full covariate vector \(X\). For a particular observation, let \(\mu_i=4\). Find its conditional variance, and describe what the assumptions say—and do not say—about its distribution.
The specified variance is $$ \operatorname{Var}(Y_i\mid X)=\phi V(\mu_i)=1.5(4)=6. $$ The mean is \(E[Y_i\mid X]=4\). These two moments do not require \(Y_i\mid X\) to have a Poisson distribution: a Poisson variable with mean 4 would have variance 4, not 6.
To check that these moments are compatible with a count distribution, consider a conditional law assigning probability \(1/3\) each to the values 1, 4, and 7. Its mean is $$ \frac{1+4+7}{3}=4, $$ and its variance is $$ \frac{(1-4)^2+(4-4)^2+(7-4)^2}{3} =\frac{9+0+9}{3}=6. $$ This is one possible law satisfying the specified moments, not a law implied by them. Other distributions can have the same mean and variance.
For several conditionally independent observations, the variance of their total is \(1.5\sum_i\mu_i\). That aggregate calculation uses conditional independence and the individual variance assumptions; the mean-variance specification alone would not justify it.
Worked Example: Grouped Binary Responses
Worked Example: Grouped Binary Responses
At each of several sites, a group contains \(m=20\) binary responses, and \(Y_j\) is the number of successes in group \(j\). Suppose the mean success probability is \(p_j=0.3\), so that \(E[Y_j\mid X]=m p_j\), and use the variance function \(V(\mu)=\mu(1-\mu/m)\). With \(\phi=1.4\), calculate the mean and variance for one group.
The mean is $$ \mu_j=m p_j=20(0.3)=6. $$ Therefore, $$ V(\mu_j)=6\left(1-\frac{6}{20}\right) =6(0.7)=4.2, $$ and the specified variance is $$ \operatorname{Var}(Y_j\mid X)=1.4(4.2)=5.88. $$
If the 20 responses were conditionally independent Bernoulli variables with success probability \(0.3\), the count would have binomial variance \(20(0.3)(0.7)=4.2\), corresponding to \(\phi=1\). The larger variance here describes extra variation at the group-count level. It does not, on its own, specify why that variation occurs or give a full probability law.
For a concrete check that the stated moments are possible for a count between 0 and 20, assign probabilities \(49/150\), \(52/150\), and \(49/150\) to counts 3, 6, and 9, respectively. The mean is 6 by symmetry, and the variance is $$ \frac{49}{150}(3-6)^2+\frac{52}{150}(6-6)^2+ \frac{49}{150}(9-6)^2 =\frac{882}{150}=5.88. $$ This unusual law illustrates the moment specification; it is not asserted to arise from independent Bernoulli trials.
If the groups are conditionally independent given \(X\), the variance of their total count is the sum of their group variances. If responses within or across groups are dependent, the corresponding covariance terms must also be included.
Worked Example: Why the Conditioning Set Matters
Worked Example: Why the Conditioning Set Matters
Let \(X_1\) be constant, let \(X_2\) be Bernoulli with success probability \(1/2\), and set \(Y_1=X_2\), \(Y_2=0\). Given the full vector \(X=(X_1,X_2)\), both responses are fixed, so they are conditionally independent (degenerate random variables are independent). In particular, $$ \operatorname{Var}(Y_1\mid X)=0,\qquad \operatorname{Var}(Y_2\mid X)=0. $$
But conditioning only on \(X_1\), which is constant, leaves \(X_2\) random. Consequently, $$ E[Y_1\mid X_1]=\frac12,\qquad \operatorname{Var}(Y_1\mid X_1) =\frac12\left(1-\frac12\right)=\frac14. $$ Thus \(\operatorname{Var}(Y_1\mid X)\) is not the same as \(\operatorname{Var}(Y_1\mid X_1)\). Conditional independence given the full vector does not license replacing moments conditional on that vector with moments conditional on an individual covariate. All the variance assumptions in a mean-variance model and any independence claims used with them must refer to a consistent conditioning set.
A Mean and Variance Do Not Identify a Distribution
The freedom left by a moment model is not just a technicality. For example, a Poisson distribution with mean 2 has variance 2. A different count distribution assigns probabilities \(1/4,1/2,1/4\) to 0, 2, and 4. Its mean is $$ \frac14(0)+\frac12(2)+\frac14(4)=2, $$ and its variance is $$ \frac14(0-2)^2+\frac12(2-2)^2+\frac14(4-2)^2=2. $$ These distributions agree on the first two moments but differ in their possible outcomes and probabilities. Therefore, specifying \(E[Y_i\mid X]\) and \(\operatorname{Var}(Y_i\mid X)\) does not determine tail probabilities or a likelihood \(L(\theta)\). Likelihood-based conclusions require a full probability model, not just these moments.
Common Mistakes
- Treating a variance function as a probability law. A Poisson-shaped variance function does not make a response Poisson. The specified mean and variance do not determine the remaining probabilities.
- Assuming moment conditions imply independence. Marginal conditional means and variances say nothing by themselves about \(\operatorname{Cov}(Y_i,Y_j\mid X)\).
- Changing the conditioning set mid-calculation. Conditional moments given the full vector \(X\) need not equal moments given only \(X_i\). Keep the conditioning set consistent unless equality is separately established.
- Interpreting overdispersion as a specific mechanism. A value \(\phi>1\) describes a variance relative to the chosen function; it does not identify dependence, latent heterogeneity, or any other cause.
Check Your Understanding
Use the moment assumptions and the conditioning set explicitly in each answer.
- What does a conditional mean-variance model specify, and what essential feature of a full distribution does it leave unspecified?
- If responses are conditionally independent given \(X\), and \(\operatorname{Var}(Y_i\mid X)=\phi V(\mu_i)\), write the conditional variance of their sum.
- For a group of 10 binary responses with mean success probability \(0.4\), calculate the binomial variance and the variance under the same mean-variance function with \(\phi=1.25\).
- Why does conditional independence given the full covariate vector not imply that conditional moments given only one observation's covariate are unchanged?
- What additional terms are needed in the variance of a sum if responses are not conditionally independent?