Marginal And Conditional Response Models

Why Distinguish Conditional and Marginal Models?

A response distribution can describe outcomes for people with a specified covariate value, or outcomes in a population that includes several covariate values. These are different probability questions. For example, the chance of a health outcome among people with a particular risk factor need not equal the chance across everyone, because the overall population also includes people without that risk factor.

In a generalized linear model, it is common to describe the response conditional on covariates. A marginal response distribution instead averages over the covariate distribution in a specified population. Understanding this distinction prevents a conditional probability from being reported as if it were an overall probability, and makes clear why an overall probability can change when the population's covariate composition changes.

Definition: Let \(Y\) be a response and \(X\) a covariate. The conditional response distribution at \(X=x\) is the distribution of \(Y\) among units with covariate value \(x\). For discrete \(Y\), write its conditional probability mass function as \(f_{Y\mid X}(y\mid x)=P(Y=y\mid X=x)\). The marginal response distribution is the distribution of \(Y\) without conditioning on \(X\), with mass function \(f_Y(y)=P(Y=y)\). The marginal distribution averages over the distribution of \(X\).

The word marginal means that the covariate is not specified in the response probability. It does not mean that the covariate has no relationship with the response. If response probabilities differ across covariate values, the marginal probability combines those different probabilities using the frequencies of the covariate values.

From Conditional Probabilities to Marginal Probabilities

Suppose \(X\) has a finite or countable set of possible values. The events \(X=x\) partition the population, so the law of total probability gives a direct formula for the marginal distribution of a discrete response.

Theorem: Suppose \(X\) is discrete, \(Y\) is discrete, and \(P(X=x)>0\) for the values included in the sum. Then

P(Y=y)=\sum_x P(Y=y\mid X=x)P(X=x).

In particular, the marginal response probability is a weighted average of the conditional response probabilities, with weights given by the covariate probabilities.

Proof. For each value \(x\), the event \(\{Y=y,X=x\}\) has probability \(P(Y=y\mid X=x)P(X=x)\), by the definition of conditional probability. The events \(\{Y=y,X=x\}\), as \(x\) varies, are disjoint and their union is \(\{Y=y\}\). Add their probabilities:

P(Y=y) =\sum_x P(Y=y,X=x) =\sum_x P(Y=y\mid X=x)P(X=x).

The sum is valid for a countable set of covariate values because it adds probabilities of disjoint events. \(\square\)

For a binary response, let \(Y=1\) indicate success. Write \(p(x)=P(Y=1\mid X=x)\). The theorem becomes

P(Y=1)=\sum_x p(x)P(X=x).

Thus the marginal success probability is not usually any one of the conditional probabilities. For a continuous covariate with density \(f_X(x)\), the corresponding formula, when the conditional probability is defined and the integral exists, is

P(Y=y)=\int P(Y=y\mid X=x)f_X(x)\,dx

for a discrete response value \(y\). The integral averages over the covariate density rather than summing over covariate categories.

Worked Example: A Marginal Binary Response Probability

Suppose \(X=1\) denotes exposure to a risk factor and \(X=0\) denotes no exposure. In the population, \(P(X=1)=0.40\), so \(P(X=0)=0.60\). The response \(Y\) is a binary health outcome, with

P(Y=1\mid X=0)=0.10,\qquad P(Y=1\mid X=1)=0.30.

Find the marginal probability \(P(Y=1)\). First apply the law of total probability:

P(Y=1) =P(Y=1\mid X=0)P(X=0)+P(Y=1\mid X=1)P(X=1).

Substitute the given values and calculate each contribution:

$$ P(Y=1)=(0.10)(0.60)+(0.30)(0.40) =0.06+0.12 =0.18. $$

The overall probability is 0.18, or 18%. It is neither conditional probability: it lies between 10% and 30% because both covariate groups have positive population proportions. If the same conditional probabilities applied in a population with \(P(X=1)=0.70\), the marginal probability would instead be \((0.10)(0.30)+(0.30)(0.70)=0.24\). The conditional response model stayed fixed; the population composition changed.

Marginalizing a Categorical Response

The same averaging applies when the response has more than two categories. In that case, calculate the marginal probability of each response category separately. The resulting probabilities must still add to one.

Worked Example: Combining Conditional Categorical Distributions

Let \(X=0\) and \(X=1\) identify two covariate groups, with \(P(X=0)=0.75\) and \(P(X=1)=0.25\). The response \(Y\) has categories A, B, and C. Its conditional distributions are:

Covariate value	\(P(Y=\mathrm{A}\mid X)\)	\(P(Y=\mathrm{B}\mid X)\)	\(P(Y=\mathrm{C}\mid X)\)
\(X=0\)	0.60	0.30	0.10
\(X=1\)	0.20	0.50	0.30

Each row is a valid conditional distribution: the first sums to \(0.60+0.30+0.10=1\), and the second sums to \(0.20+0.50+0.30=1\). Average the entries category by category using the covariate probabilities.

For category A,

P(Y=\mathrm{A})=(0.60)(0.75)+(0.20)(0.25)=0.45+0.05=0.50.

For category B,

P(Y=\mathrm{B})=(0.30)(0.75)+(0.50)(0.25)=0.225+0.125=0.35.

For category C,

P(Y=\mathrm{C})=(0.10)(0.75)+(0.30)(0.25)=0.075+0.075=0.15.

The marginal distribution is therefore \((0.50,0.35,0.15)\). As a check, its probabilities sum to \(0.50+0.35+0.15=1\). This marginal distribution summarizes the population as a whole, while each row in the table summarizes one covariate group.

A Mixture Can Change the Distribution's Form

Averaging conditional probabilities always gives the marginal probabilities, but the marginal response distribution need not have the same model form as each conditional distribution. The next example shows that a mixture of binomial distributions can fail to be binomial.

Worked Example: A Mixture of Binomial Response Models

Suppose \(X\) takes values 0 and 1 with probability \(0.5\) each. Given \(X=0\), the response count \(Y\) has a \(\operatorname{Binomial}(2,0.2)\) distribution. Given \(X=1\), it has a \(\operatorname{Binomial}(2,0.8)\) distribution. Find the marginal probabilities for \(Y=0,1,2\), then check whether the marginal distribution is binomial with two trials.

For \(X=0\), the three conditional probabilities are

P(Y=0\mid X=0)=(0.8)^2=0.64,\quad P(Y=1\mid X=0)=2(0.2)(0.8)=0.32,\quad P(Y=2\mid X=0)=(0.2)^2=0.04.

For \(X=1\), they are

P(Y=0\mid X=1)=(0.2)^2=0.04,\quad P(Y=1\mid X=1)=2(0.8)(0.2)=0.32,\quad P(Y=2\mid X=1)=(0.8)^2=0.64.

Now average each probability with weights \(0.5\) and \(0.5\):

$$ P(Y=0)=(0.64)(0.5)+(0.04)(0.5)=0.34, $$

$$ P(Y=1)=(0.32)(0.5)+(0.32)(0.5)=0.32, $$

$$ P(Y=2)=(0.04)(0.5)+(0.64)(0.5)=0.34. $$

These probabilities sum to \(0.34+0.32+0.34=1\). If the marginal distribution were \(\operatorname{Binomial}(2,q)\), its mean would be \(2q\). The conditional means are \(2(0.2)=0.4\) and \(2(0.8)=1.6\), so the marginal mean is \((0.4)(0.5)+(1.6)(0.5)=1\). A binomial distribution with two trials and mean 1 must have \(q=0.5\). But \(\operatorname{Binomial}(2,0.5)\) assigns probabilities \((0.25,0.50,0.25)\), not \((0.34,0.32,0.34)\). The marginal distribution is therefore not binomial, even though each conditional distribution is.

A Second Averaging Result: The Marginal Mean

The same structure applies to response means. If the conditional mean exists and is integrable, the marginal mean is the average of the conditional means. For discrete \(X\), this follows directly by grouping the possible response values according to the covariate value.

Theorem: If \(X\) and \(Y\) are discrete and \(E[|Y|]<\infty\), then

E[Y]=\sum_x E[Y\mid X=x]P(X=x),

where the sum is over covariate values with positive probability.

Proof. Absolute integrability ensures the expectations and rearrangement of the sums below are valid. Starting from the joint probabilities,

E[Y] =\sum_y yP(Y=y) =\sum_y y\sum_x P(Y=y\mid X=x)P(X=x).

Interchange the sums, then factor out terms that do not depend on \(y\):

E[Y] =\sum_x P(X=x)\sum_y yP(Y=y\mid X=x) =\sum_x P(X=x)E[Y\mid X=x).

Replacing the final unmatched parenthesis with the conditional expectation notation gives \(E[Y]=\sum_x E[Y\mid X=x]P(X=x)\). \(\square\)

This result is useful for interpreting a conditional response model: its fitted means at different covariate values must be combined using a covariate distribution to obtain an overall mean. It does not say that the overall mean equals the conditional mean at an “average” covariate value.

Population Margins and Study Designs

To define a population marginal distribution, one needs a distribution for \(X\). In an observational population, this may be the population distribution of the covariates. In a designed study, investigators may instead set the numbers of units assigned to different covariate groups. Then averaging conditional response probabilities using the study's group proportions gives the average for that design, not necessarily for a target population with different group proportions.

For example, if a study deliberately includes equal numbers from two groups, its sample proportion of responses averages the two conditional rates with weights one-half and one-half. A population with 80% in the first group would use weights 0.8 and 0.2 for its marginal probability. The conditional probabilities can be identical in both calculations, while the two marginal probabilities differ.

Common Mistake: Treating a Conditional Probability as Overall

Wrong reasoning: “The response probability for \(X=1\) is 0.30, so the response probability in the population is 0.30.” This uses one subgroup probability without accounting for the other covariate values.
Correction: Use the law of total probability and the covariate weights. If the probabilities are 0.10 for \(X=0\) and 0.30 for \(X=1\), with group proportions 0.60 and 0.40, the population probability is \((0.10)(0.60)+(0.30)(0.40)=0.18\).
Another error: Assuming that averaging conditional distributions preserves their family. The binomial example shows that a mixture of binomial distributions with different success probabilities need not be binomial.

Key takeaway: A conditional response model describes outcomes at specified covariate values. A marginal response distribution averages those conditional distributions over a stated covariate distribution. Always identify which probability is being requested and which covariate weights define the population or design.

Check Your Understanding

Use the conditional-versus-marginal distinction to answer the following.

If \(P(X=0)=0.7\), \(P(X=1)=0.3\), and the conditional success probabilities are 0.2 and 0.5, respectively, what is the marginal success probability?
Why can two populations with the same conditional response probabilities have different marginal response probabilities?
For a categorical response with three categories, how do you obtain the marginal probability of the second category?
In the binomial mixture example, what value of \(q\) would a two-trial binomial distribution need to have mean 1, and why does that fail to match the marginal distribution?
What additional distribution must be specified to turn conditional response probabilities into a population marginal probability?