Building The Stochastic Component From Moments

From a Mean to a Stochastic Component

A GLM has a stochastic component: a probability model for the response, conditional on the predictors or other quantities treated as fixed. Giving only a mean does not complete that specification. One must also choose a response family, state which values the response can take, and give the family’s variance relationship and dispersion convention. These choices must agree with one another.

For response \(Y_i\), write its conditional mean as \(\mu_i=E[Y_i\mid x_i]\), where \(x_i\) denotes the predictors for observation \(i\). The mean model determines how \(\mu_i\) varies with the predictors. The stochastic component describes the distribution around that mean. In this tutorial, the focus is how a specified family and mean determine the support and variance; the mean model itself is not the subject.

Definition: A stochastic-component specification states the response family for \(Y_i\), its support, the permissible values of \(\mu_i\), and its variance in terms of \(\mu_i\) and any dispersion parameter. In the common variance-function notation, \(\operatorname{Var}(Y_i\mid x_i)=\phi V(\mu_i)\), where \(\phi\) is the dispersion parameter and \(V\) is the variance function. Some families have a fixed dispersion convention, rather than a freely varying \(\phi\).

The support is not a cosmetic detail. A mean of \(-2\) is possible for a normal response, but not for a Poisson count or a gamma response. Similarly, a binomial count with \(m\) trials cannot have a mean above \(m\). A valid specification must keep every proposed \(\mu_i\) within the domain of its selected family.

Common Families at a Glance

The table summarizes standard response-family conventions. For the binomial row, \(m\) is the number of trials and \(Y_i\) is the count of successes. The table gives conditional moments and support; the distribution name identifies more than those moments alone.

Family and response	Support	Mean domain	Variance and dispersion convention
Bernoulli response \(Y_i\)	\(\{0,1\}\)	\(0\leq\mu_i\leq1\)	\(\mu_i(1-\mu_i)\); fixed \(\phi=1\)
Binomial count \(Y_i\), \(m\) trials	\(\{0,1,\ldots,m\}\)	\(0\leq\mu_i\leq m\)	\(\mu_i(1-\mu_i/m)\); fixed \(\phi=1\)
Poisson count \(Y_i\)	\(\{0,1,2,\ldots\}\)	\(\mu_i\geq0\)	\(\mu_i\); fixed \(\phi=1\)
Normal response \(Y_i\)	\(\mathbb{R}\)	\(\mu_i\in\mathbb{R}\)	\(\phi\), with \(V(\mu_i)=1\)
Gamma response \(Y_i\)	\((0,\infty)\)	\(\mu_i>0\)	\(\phi\mu_i^2\)
Inverse-Gaussian response \(Y_i\)	\((0,\infty)\)	\(\mu_i>0\)	\(\phi\mu_i^3\)

In the usual gamma convention, \(\phi>0\) is the reciprocal of the shape parameter. For the inverse-Gaussian convention, \(\phi>0\) is the reciprocal of its shape parameter. For normal responses, \(\phi\) is the variance. In Bernoulli, binomial, and Poisson GLMs, the dispersion is conventionally fixed at one. Thus the same symbol \(\phi\) does not mean that every family has a freely estimated dispersion parameter.

Boundary means require care. A Bernoulli mean of zero or one, for example, corresponds to a degenerate response that is always zero or always one. A Poisson mean of zero likewise gives a response that is always zero. The positive-mean gamma and inverse-Gaussian families do not include zero as a nondegenerate response mean.

Two Useful Construction Results

First, changing the units of a response changes its mean and variance in predictable ways. This matters when a response is recorded as a count in one analysis and as a rate or proportion in another.

Result: If \(Z=cY\) for a fixed nonzero constant \(c\), then the support of \(Z\) is the support of \(Y\) multiplied by \(c\), \(E[Z]=cE[Y]\), and \(\operatorname{Var}(Z)=c^2\operatorname{Var}(Y)\). If \(\operatorname{Var}(Y)=\phi V(\mu)\), then, writing \(\mu_Z=E[Z]\), the variance function for \(Z\) is \(V_Z(\mu_Z)=c^2V(\mu_Z/c)\), with the same \(\phi\).

Proof. Since \(Z=cY\), every possible value \(y\) of \(Y\) is transformed to \(cy\), giving the stated support. Linearity of expectation gives \(E[Z]=E[cY]=cE[Y]\). Also,

\operatorname{Var}(Z) =E[(cY-cE[Y])^2] =c^2E[(Y-E[Y])^2] =c^2\operatorname{Var}(Y).

Because \(\mu_Z=c\mu\), we have \(\mu=\mu_Z/c\). Substitution into the original variance relationship gives \(\operatorname{Var}(Z)=\phi c^2V(\mu_Z/c)\), which establishes the stated transformed variance function. \(\square\)

A second construction result clarifies how binomial counts and proportions differ. Let \(Y\) be a binomial count with \(m\) trials and success probability \(p\). Define the observed proportion \(R=Y/m\). The count and proportion describe the same underlying successes, but their supports and variances are on different scales.

Result: For \(Y\sim\operatorname{Binomial}(m,p)\), \(E[Y]=mp\) and \(\operatorname{Var}(Y)=mp(1-p)\). For \(R=Y/m\), the support is \(\{0,1/m,\ldots,1\}\), \(E[R]=p\), and \(\operatorname{Var}(R)=p(1-p)/m\).

Proof. Represent \(Y\) as \(\sum_{j=1}^{m}B_j\), where the \(B_j\) are independent Bernoulli random variables with success probability \(p\). Then \(E[B_j]=p\) and \(\operatorname{Var}(B_j)=p(1-p)\). Linearity gives \(E[Y]=\sum_jp=mp\). Independence makes the covariances between distinct \(B_j\) equal zero, so \(\operatorname{Var}(Y)=\sum_jp(1-p)=mp(1-p)\). Dividing each possible count by \(m\) gives the stated support for \(R\). Applying the scaling result with \(c=1/m\) gives \(E[R]=p\) and \(\operatorname{Var}(R)=m^{-2}mp(1-p)=p(1-p)/m\). \(\square\)

Worked Constructions

Worked Example: A Binomial Count With Eight Trials

Suppose each response is the number of successes among \(m=8\) trials, and the proposed success probability is \(p=0.25\). Specify the mean, support, variance, and dispersion convention for the count.

The response is a binomial count, so its support is \(\{0,1,\ldots,8\}\). Its mean is \(\mu=mp=8(0.25)=2\), which is in the required interval from 0 to 8. The variance on the count scale is

\operatorname{Var}(Y)=mp(1-p) =8(0.25)(0.75) =1.5.

Equivalently, using the mean-based variance function, \(\operatorname{Var}(Y)=\mu(1-\mu/m)=2(1-2/8)=2(0.75)=1.5\). The standard binomial convention fixes \(\phi=1\).

If the recorded response were instead the proportion \(R=Y/8\), its support would be \(\{0,1/8,\ldots,1\}\), its mean would be \(0.25\), and its variance would be \(1.5/64=0.0234375\). Using the proportion formula gives the check \(0.25(0.75)/8=0.0234375\).

Worked Example: A Gamma Response With Specified Mean and Dispersion

Let a positive response have a gamma distribution with mean \(\mu=12\) and dispersion \(\phi=0.25\), using the convention that \(\phi\) is the reciprocal of the shape. Specify the distribution parameters and verify the variance.

The shape is \(a=1/\phi=4\). For a gamma distribution parameterized by shape \(a\) and scale \(b\), the mean is \(ab\). Therefore the required scale is \(b=\mu/a=12/4=3\). Its density is

f(y)=\frac{y^{a-1}e^{-y/b}}{\Gamma(a)b^a},\qquad y>0.

Thus the support is \((0,\infty)\), and the specified distribution is gamma with shape \(4\) and scale \(3\). Its mean is \(ab=4(3)=12\). Its variance is \(ab^2=4(3^2)=36\), which agrees with the gamma variance function:

\phi\mu^2=0.25(12^2)=0.25(144)=36.

The support and positive mean are consistent with the gamma family, and the dispersion convention produces the same variance as the shape-scale parameterization.

Worked Example: A Normal Response With Constant Variance

Suppose a continuous response is modeled as normal with mean \(\mu=-2\) and dispersion \(\phi=9\). State its distribution, support, and variance.

Under the course convention, \(N(\mu,\sigma^2)\) uses its second argument for the variance. Therefore \(Y\sim N(-2,9)\), the support is all real numbers, and \(\operatorname{Var}(Y)=9\). The normal variance function is \(V(\mu)=1\), so \(\phi V(\mu)=9(1)=9\). A negative mean is permissible because the support includes negative values; it would not be permissible for a gamma response.

Worked Example: A Poisson Count With Mean Four

Suppose a response is a count with a Poisson distribution and proposed mean \(\mu=4\). Its support is the nonnegative integers, and the mean lies in the Poisson domain. The Poisson mean-variance relationship gives \(\operatorname{Var}(Y)=\mu=4\). The dispersion convention is fixed at \(\phi=1\); the variance is not specified by adding a separate free dispersion factor in the standard Poisson GLM.

Why the Family Choice Matters

A construction is not complete until the family is named. For example, a mean of 12 and variance of 36 are compatible with the gamma specification above, but those two moments alone do not establish that the response is gamma. The response family also determines the possible values and the full probability distribution. Mean and variance are checks on a proposed family, not a substitute for choosing one.

A practical construction can be made in a fixed order:

Identify what one response represents.
Decide whether it is a binary outcome, a count, a continuous measurement, a positive duration, or another type of response.

Choose a family and state its support.
Check that the response’s possible values match the family, and that every proposed mean is in its domain.

Write the mean-variance relationship.
Use the response scale actually being modeled. For a binomial response, distinguish a success count from a success proportion.

Apply the dispersion convention.
State whether dispersion is fixed at one or is a parameter, and check the resulting variance numerically when possible.

The most common error is to use a familiar variance formula without checking what the response measures. For a binomial count, writing \(p(1-p)\) omits the number of trials \(m\); that is the variance of a single Bernoulli response, not the count. The correction is to specify the response and use \(mp(1-p)\) for the count or \(p(1-p)/m\) for the sample proportion. Another error is to call \(\phi\) a free parameter in every family. In the standard binomial and Poisson conventions it is fixed at one, whereas the normal, gamma, and inverse-Gaussian variance relationships use a dispersion parameter.

Key takeaway: A coherent stochastic component names the response family, gives its support and allowed mean domain, states the variance on the correct response scale, and follows that family’s dispersion convention. Check all four pieces together before treating the specification as complete.

Check Your Understanding

For each question, state the family or relationship requested and show the reasoning.

A binomial response counts successes in 5 trials and has success probability \(0.4\). Give its support, mean, and variance.
A binomial count has mean \(3\) with \(m=6\). What is its variance, and what would the variance be for the corresponding proportion?
For a gamma response with mean \(10\) and dispersion \(0.2\), give its support and variance under the reciprocal-shape convention.
Why is a proposed mean of \(-1\) incompatible with a Poisson or gamma response but permitted for a normal response?
A response is specified as Poisson with mean \(7\). State its variance and dispersion convention.