Conditional Distributions Given Predictors

Why Condition on Predictors?

A generalized linear model (GLM) describes how a response behaves for observations with specified predictor values. Its starting point is a conditional distribution: for each observation, what distribution is assumed for the response once its predictors are given? This is different from describing the response distribution across a population that mixes many predictor profiles.

For example, a hypothetical risk study might record whether each policy has a claim and include a risk-class indicator among its predictors. A GLM can specify a Bernoulli distribution for the claim indicator conditional on risk class. The probability of a claim may differ across classes. The model’s distributional assumption is that the conditional response is Bernoulli; the mean assumption specifies how the conditional claim probability behaves across predictor values.

The predictors can be treated as observed values on which the response model is conditioned. A probability model for the predictors themselves is not needed to specify this conditional distribution. If predictors are random and a population-wide, or marginal, response distribution is required, then their distribution also matters.

Two Distinct Parts of a GLM Specification

A useful way to organize a GLM specification is to ask two separate questions. First, what distribution is assumed for the response at a fixed predictor profile? Second, how does the conditional mean change as the predictor profile changes? The first question concerns the random component; the second concerns the mean structure.

• Distributional assumption: identify a suitable response family, its support, and any required dispersion or other parameters. This also determines how conditional variation relates to the family’s parameters.
• Conditional-mean assumption: state the proposed pattern of \(\mu_i=E[Y_i\mid\mathbf{x}_i]\) across predictor values. A mean model must be compatible with the possible values of the response and the selected distribution.

These parts are related but are not interchangeable. A Bernoulli response has conditional mean equal to its conditional success probability and variance equal to that probability times one minus that probability. Those facts come from the Bernoulli distribution. A separate model is still needed to describe how the probability changes with predictors. Likewise, a Poisson response has conditional mean and variance equal to its conditional rate parameter, but the assumption of a Poisson conditional distribution does not by itself say how that parameter changes across profiles.

Many GLMs use a member of the exponential-dispersion family. In a common parameterization, the conditional density or probability mass function has the form below, where \(\theta_i\) is a parameter associated with observation \(i\), \(\phi\) is a dispersion parameter, and the functions \(a\), \(b\), and \(c\) define the family.

For a regular member of this family, the conditional mean and variance are \(E[Y_i\mid\mathbf{x}_i]=b'(\theta_i)\) and \(\operatorname{Var}(Y_i\mid\mathbf{x}_i)=a(\phi)b''(\theta_i)\). These formulas describe the distribution’s parameterization. The mean model then specifies how the resulting mean depends on the predictors. The family form does not, on its own, impose a particular pattern across predictor profiles.

Conditional and Marginal Distributions

Suppose predictor profiles differ in the response means. The marginal distribution combines the conditional distributions over those profiles. In general, that mixture does not have the same distributional family as any one of its conditional components. A mixture of Bernoulli distributions is Bernoulli after averaging its success probability, but a mixture of Poisson distributions with different rates is not generally Poisson. Mixtures of normal distributions with different means are not generally normal.

The distinction can be quantified using the law of total expectation and the law of total variance. If \(X\) denotes a predictor profile and the relevant variances exist, then

The first term in the variance equation is the average within-profile variance. The second is the variance of the conditional means across profiles. Thus, marginal variation can exceed the average conditional variation when the mean differs across profiles.

What the Specification Does—and Does Not—Assume

A conditional GLM specification is a claim about responses among observations with given predictor values. It does not automatically assert that every response has an identical distribution: parameters such as the conditional mean may vary with those values. Nor does it automatically assert that the marginal response distribution belongs to the same family as the conditional response distributions.

When observations are modeled as conditionally independent given their predictor values, the joint conditional distribution factors into the product of their individual conditional distributions. This is a separate assumption from choosing a family for each response. Dependence that remains after conditioning would require an appropriate model for that dependence; merely specifying each observation’s marginal conditional distribution does not establish conditional independence.

These distinctions matter in GLM setup. A candidate should identify the response and its support, state the conditional family and any dispersion assumption, and describe the conditional mean as a function of predictors. Each statement should be interpreted conditionally. In particular, an estimated or specified conditional mean concerns the response scale; it is not itself a probability distribution for the predictors or a statement about the marginal response.