Conditional Response Distributions Given Covariates

Why Condition on Covariates?

In many studies, the response of interest is expected to vary with information recorded about each unit. A patient's binary outcome may depend on treatment group; a count may depend on exposure time; a measurement may vary with age. A response model should therefore describe not just the possible values of the response, but how its distribution changes with the covariates.

Write $Y$ for a response and $X$ for a covariate or covariate vector. The central object is the distribution of $Y$ given $X=x$. It answers a conditional question: among units with covariates $x$, what probabilities or density describe the response? It does not, by itself, say how often the covariate value $x$ occurs.

Definition: A conditional response distribution given covariates is a probability distribution for $Y$ indexed by covariate values $x$. For a discrete response, it may be specified by a conditional probability mass function $f_{Y\mid X}(y\mid x)$, whose values sum to one over the response support. For a continuous response, it may be specified by a conditional density $f_{Y\mid X}(y\mid x)$, which integrates to one over that support. The dependence on $x$ describes how the response distribution changes with the covariates.

For discrete covariates, conditioning can be defined by the familiar ratio $P(Y=y\mid X=x)=P(Y=y,X=x)/P(X=x)$ whenever $P(X=x)>0$. If $X$ is continuous, then typically $P(X=x)=0$, so this ratio is not a definition. Instead, the conditional distribution is described by a regular conditional probability distribution: for each relevant $x$, it assigns probabilities to response events, with those probabilities measurable as functions of $x$. Such distributions exist when the variables take values in standard Borel spaces, as real-valued responses and covariates do. They are generally determined only up to sets of covariate values having probability zero. A model may specify a convenient version at particular covariate values, but data alone cannot identify behavior at covariate values outside the model's supported or justified range.

What the Conditional Model Specifies

A conditional response model gives a probability law for the response at specified covariate values. For example, a model may assert that $Y\mid X=x$ is Bernoulli with success probability $p(x)$, or that $Y\mid X=x$ is normal with mean $\mu(x)$ and variance $\sigma^2(x)$. The functions $p(x)$, $\mu(x)$, or $\sigma^2(x)$ describe how the response law depends on covariates. At each fixed $x$, the resulting probabilities or density must still define a valid distribution.

The model is not a probability law for $X$. It does not specify $P(X=x)$, a density for $X$, or how covariates are sampled. Covariates may be deliberately assigned or fixed by a study design, or they may be random measurements. Either way, specifying $Y\mid X=x$ alone does not decide how the values of $X$ are distributed.

Worked Example: A Binary Response by Treatment Group

Let $Y=1$ indicate a treatment success and $Y=0$ a failure. Let $X$ identify treatment group, with $X=0$ for the comparison group and $X=1$ for the new treatment. Suppose the conditional model is Bernoulli, with $P(Y=1\mid X=0)=0.20$ and $P(Y=1\mid X=1)=0.35$.

For the comparison group, the complete conditional probability mass function is $P(Y=0\mid X=0)=1-0.20=0.80$ and $P(Y=1\mid X=0)=0.20$. These probabilities sum to $0.80+0.20=1$.

For the new-treatment group, $P(Y=0\mid X=1)=1-0.35=0.65$ and $P(Y=1\mid X=1)=0.35$, again summing to one. Thus the model specifies the response distribution at each treatment value.

It does not tell us what proportion of study participants receive the new treatment. That proportion concerns the distribution of $X$, not the conditional distribution of $Y$ given $X$.

Worked Example: A Count Response Given Exposure

Suppose $Y$ is a count of events and $X$ is exposure measured in units of time. Consider the conditional model $Y\mid X=x\sim\operatorname{Poisson}(1.5x)$ for $x>0$. At exposure $x=2$, the Poisson parameter is $1.5(2)=3$, so

P(Y=y\mid X=2)=e^{-3}\frac{3^y}{y!},\qquad y=0,1,2,\ldots.

For example, $P(Y=0\mid X=2)=e^{-3}$, which is about $0.0498$, and $P(Y=1\mid X=2)=3e^{-3}$, about $0.1494$. At exposure $x=1$, the parameter is $1.5$, giving $P(Y=0\mid X=1)=e^{-1.5}$, about $0.2231$. The count distribution changes with exposure because its Poisson parameter changes with $x$.

This specification says how counts behave at a given exposure. It does not provide a distribution for exposure itself—for example, it does not state what proportion of units have exposure equal to one or two.

Worked Example: A Continuous Response at a Given Covariate Value

Suppose a measurement $Y$, in millimeters, has the conditional model $Y\mid X=x\sim N(10+x,9)$, where the second argument is the variance. At $x=2$, the conditional mean is $10+2=12$, and the conditional standard deviation is $\sqrt{9}=3$. Therefore,

P(9\leq Y\leq 15\mid X=2) =P\left(\frac{9-12}{3}\leq \frac{Y-12}{3}\leq \frac{15-12}{3}\,\middle|\,X=2\right) =\Phi(1)-\Phi(-1)\approx 0.6827.

Here $\Phi$ is the standard normal cumulative distribution function. The calculation uses the normal response distribution at the specified covariate value. It does not require a probability model for $X$.

Validity of a Conditional Response Law

A conditional probability mass function must be a valid mass function at each covariate value where the model is asserted. Likewise, a conditional density must be nonnegative and normalized over the response support. The following result makes explicit how conditional probabilities for response events are obtained from the mass function or density.

Result: Suppose that, for a fixed covariate value $x$, the conditional response law is discrete with mass function $q(y\mid x)$ on a countable support $\mathcal Y$. Then for any response event $A\subseteq\mathcal Y$, $$ P(Y\in A\mid X=x)=\sum_{y\in A}q(y\mid x). $$ The mass function is valid exactly when $q(y\mid x)\geq 0$ for every $y$ and $\sum_{y\in\mathcal Y}q(y\mid x)=1$. For a continuous response with conditional density $q(y\mid x)$, the corresponding event probability is the integral of that density over the event, and validity requires nonnegativity and integral one.

For the discrete case, the proof follows from countable additivity: the event $Y\in A$ is the disjoint union of the events $Y=y$, for $y\in A$, so its conditional probability is the sum of their conditional probabilities. Taking $A=\mathcal Y$ gives total probability one; taking any singleton shows each mass is nonnegative. For a continuous response, the defining property of a density gives the probability of an event by integration, and applying it to the entire support gives total integral one. These conditions are checked at each covariate value where the conditional model is defined.

A Response Model Is Not a Covariate Model

The distinction can be shown exactly for discrete covariates and responses. Let $p(x,y)$ be a joint probability mass function, and define $p_X(x)=\sum_y p(x,y)$. At any value $x$ with $p_X(x)>0$, the conditional mass function is

q(y\mid x)=\frac{p(x,y)}{p_X(x)}.

Thus the joint probabilities contain both the covariate probabilities and the conditional response probabilities. Conversely, a conditional response model alone supplies only $q(y\mid x)$.

Theorem: Let $\mathcal X$ and $\mathcal Y$ be finite or countable sets. Suppose $q(y\mid x)\geq 0$ and $\sum_{y\in\mathcal Y}q(y\mid x)=1$ for every $x\in\mathcal X$. For any probability mass function $r(x)$ on $\mathcal X$, the formula $p(x,y)=r(x)q(y\mid x)$ defines a joint probability mass function. Its conditional response distribution is $q(y\mid x)$ wherever $r(x)>0$. Consequently, a conditional response model does not determine the distribution of covariates.

Proof. Each joint mass $p(x,y)=r(x)q(y\mid x)$ is nonnegative. Its total sum is

\sum_{x\in\mathcal X}\sum_{y\in\mathcal Y}p(x,y) =\sum_{x\in\mathcal X}r(x)\sum_{y\in\mathcal Y}q(y\mid x) =\sum_{x\in\mathcal X}r(x)=1.

The interchange and grouping of these sums are valid because all terms are nonnegative. Thus $p$ is a joint probability mass function. Summing over responses gives $p_X(x)=\sum_y r(x)q(y\mid x)=r(x)$. If $r(x)>0$, division then yields $p(x,y)/p_X(x)=q(y\mid x)$, as claimed. Since $r$ can be any probability mass function on $\mathcal X$, the same conditional response model can accompany different covariate distributions. $\square$

Worked Example: The Same Conditional Model, Different Covariate Frequencies

Return to the binary treatment example, with $q(1\mid 0)=0.20$ and $q(1\mid 1)=0.35$. Suppose first that $P(X=0)=0.80$ and $P(X=1)=0.20$. The joint probability of $X=1,Y=1$ is

P(X=1,Y=1)=P(X=1)P(Y=1\mid X=1)=0.20(0.35)=0.07.

Now let $P(X=0)=0.30$ and $P(X=1)=0.70$, leaving the conditional response probabilities unchanged. Then $P(X=1,Y=1)=0.70(0.35)=0.245$. The conditional success probability in the new-treatment group remains $0.35$, but the joint probability differs because the covariate distribution differs.

The two specifications agree about the response law at each treatment value. They disagree about how frequently each treatment group occurs. That information is additional to, not implied by, the conditional response model.

Common Mistake and Practical Interpretation

Mistake: “I have specified $P(Y=y\mid X=x)$, so I have specified the probability model for the study.” This confuses a conditional response law with a joint law for the response and covariates. The response law answers what responses are possible and how probable they are at a given $x$; it does not say which $x$ values occur or with what probabilities.

Correction: State clearly which part of the probability structure is being modeled. A conditional response model specifies the distribution of $Y$ given $X=x$. A model for covariate values would be a separate specification. In some designs, covariates are fixed by design and no probability model for them is needed for the conditional question. In other settings covariates are random, but the conditional response model still does not automatically describe their distribution.

The qualification about probability-zero covariate values also matters. For continuously distributed $X$, individual values usually have probability zero; conditional response distributions are not obtained by dividing by $P(X=x)$. They are defined through a conditional distribution, and only values supported by the data-generating setup can be learned without additional assumptions. A formula written for every real $x$ may be useful as a model, but its interpretation beyond the observed or supported covariate range requires care.

Check Your Understanding

Use the definitions and results above to answer the following questions.

A response is Bernoulli with success probability $0.4$ when $X=0$ and $0.6$ when $X=1$. What are the two conditional failure probabilities, and why does this specification not tell you the proportion of units with $X=1$?
For a discrete response, what two conditions must a conditional mass function satisfy at each covariate value where it is defined?
Why is $P(Y=y\mid X=x)=P(Y=y,X=x)/P(X=x)$ not a suitable definition when $X$ is continuous?
Give two different covariate distributions that could accompany one fixed conditional response model. What information differs between the resulting joint models?
In the conditional normal model $Y\mid X=x\sim N(5+2x,4)$, state the conditional mean and standard deviation at $x=3$.