Bernoulli Conditional Means for Binary Outcomes

Why a Binary Outcome Has a Probability as Its Conditional Mean

Many econometric questions concern outcomes recorded as zero or one: whether a customer defaults, whether a patient experiences an event, or whether a worker is employed. For an individual observation, the recorded outcome is one realized value. The model’s target is different: it describes the probability of an outcome, given the observed covariates. Separating these two objects is essential for defining a binary-response regression model.

The conditional mean is useful because the outcome itself is binary. If $Y$ can only be zero or one, its expected value is exactly the probability that it equals one. This fact gives a direct interpretation to the regression target, but it does not mean that an individual’s observed outcome is a probability or that its value must equal the conditional mean.

Definition: A binary response $Y_i$ takes values in $\{0,1\}$, where the coding specifies which event is represented by 1. Let $X_i$ denote the observed covariates for observation $i$. The binary-response conditional mean is $$ m(X_i) = E(Y_i\mid X_i) = P(Y_i=1\mid X_i). $$ It is the probability of the coded event, conditional on the covariates. The observed outcome $Y_i$ is one realized zero or one; $m(X_i)$ is a conditional probability between zero and one.

The coding matters. If $Y_i=1$ means “default,” then $m(X_i)$ is the probability of default, not the probability of avoiding default. Reversing the coding changes the event whose probability is modeled. Throughout, the covariates may be a single variable or a vector; no particular functional form for $m(\cdot)$ is assumed here.

Why the Conditional Mean Equals a Probability

For a zero-one variable, the expectation follows directly from its two possible values. Conditional on $X_i$, the outcome is zero with probability $1-P(Y_i=1\mid X_i)$ and one with probability $P(Y_i=1\mid X_i)$. Thus,

E(Y_i\mid X_i) =0\cdot P(Y_i=0\mid X_i)+1\cdot P(Y_i=1\mid X_i) =P(Y_i=1\mid X_i).

Write $p_i=P(Y_i=1\mid X_i)$. Then, conditional on $X_i$, the outcome has a Bernoulli distribution with success probability $p_i$, written $Y_i\mid X_i\sim\operatorname{Bernoulli}(p_i)$. In this terminology, “success” means the outcome coded as one; it does not imply that the event is beneficial.

The same two-point distribution also gives the conditional variance. Since $Y_i^2=Y_i$ for both possible values,

\operatorname{Var}(Y_i\mid X_i) =E(Y_i^2\mid X_i)-[E(Y_i\mid X_i)]^2 =p_i-p_i^2 =p_i(1-p_i).

This variance belongs to the outcome conditional on the covariates. It is not the variance of $p_i$ across different covariate values. The conditional mean and conditional variance are both determined by the conditional Bernoulli probability.

Worked Example: Interpreting a Conditional Mean

Suppose a hypothetical model assigns a patient with a particular covariate profile a 0.18 probability of experiencing a specified event during follow-up. Let $Y_i=1$ indicate that the event occurs, and $Y_i=0$ indicate that it does not. Find the conditional mean and variance, and distinguish them from the patient’s observed outcome.

For this profile, $p_i=0.18$, so

E(Y_i\mid X_i)=P(Y_i=1\mid X_i)=0.18.

The conditional variance is

\operatorname{Var}(Y_i\mid X_i) =0.18(1-0.18) =0.18(0.82) =0.1476.

The patient’s observed outcome will be either 0 or 1. If the event occurs, the observation is $Y_i=1$; if not, it is $Y_i=0$. The value 0.18 is the event probability for this covariate profile, not a possible observed value of $Y_i$.

Support and the Meaning of a Valid Probability Model

The support of a random variable is the set of values it can take. For a binary response, the outcome support is $\{0,1\}$, not the full interval from zero to one. Its conditional mean has a different support: because it is a probability, $m(X_i)$ must lie in $[0,1]$ for every covariate value under consideration. The mean may be any value in that interval even though the realized outcome can only be zero or one.

This distinction is a basic validity check. A proposed conditional-mean specification that produces a value below zero or above one for a relevant covariate profile cannot represent a probability there. This tutorial does not impose a particular link or functional form to ensure valid probabilities; those modeling choices are separate. The key requirement at this stage is that the target itself be a well-defined conditional probability.

Worked Example: Comparing the Outcome and Its Mean Across Groups

Consider a hypothetical model for whether a loan defaults. Let $Y_i=1$ denote default and $Y_i=0$ denote no default. For two covariate profiles, suppose the conditional default probabilities are 0.04 and 0.11.

For the first profile, the model implies $E(Y_i\mid X_i)=0.04$; for the second, it implies $E(Y_i\mid X_i)=0.11$. Each borrower’s observed outcome is still either zero or one. The differences 0.04 and 0.11 describe conditional probabilities, not fractional default outcomes.

The associated conditional variances are

0.04(1-0.04)=0.0384,\qquad 0.11(1-0.11)=0.0979.

Both conditional means satisfy the required probability range. The larger probability is associated with the larger conditional variance in this comparison, but conditional Bernoulli variance is not simply proportional to the mean: it equals $p(1-p)$, which reaches its maximum at $p=0.5$.

Sampling Assumptions for Bernoulli Regression

A conditional mean is a feature of a conditional distribution. To use a Bernoulli regression framework, the response must have the stated binary support, and the distribution of each response given its covariates must be Bernoulli. For a sample, a commonly used sampling condition is that the responses are conditionally independent given the covariates. This means that after conditioning on the covariate information, the joint conditional distribution factors into the individual Bernoulli distributions.

P(Y_1=y_1,\ldots,Y_n=y_n\mid X_1,\ldots,X_n) =\prod_{i=1}^{n}p_i^{y_i}(1-p_i)^{1-y_i}, \qquad y_i\in\{0,1\},

where $p_i=P(Y_i=1\mid X_i)$. This factorization allows the probabilities to differ across observations; it does not require all observations to have the same event probability. Conditional independence is an assumption about the sampling or dependence structure, not a consequence of the outcome being binary. Repeated observations on the same person, shared clinics, or other clustering can create dependence that requires a different treatment.

The covariates may be treated as fixed by the design, or as observed values from a random sample. With random sampling, one standard formulation assumes independent sampled pairs $(Y_i,X_i)$ from a common population, with the conditional response distribution specified given $X_i$. In either formulation, the key conditional statement is about $Y_i$ given the covariates. If the observations are not independent, or the sampling process selectively includes outcomes in a way that changes their conditional distribution, the simple Bernoulli sampling setup may not apply as stated.

These assumptions do not by themselves guarantee that a chosen regression specification is adequate. One must also choose a conditional-mean model $m(X_i)$ that represents the probabilities of interest. At this point, the general object is simply $P(Y_i=1\mid X_i)$; no causal interpretation follows from writing it as a conditional probability.

Worked Example: Checking a Hypothetical Bernoulli Sample

A hypothetical data set records whether each of four different customers misses a payment in the next month. The outcomes are $Y_1=0$, $Y_2=1$, $Y_3=0$, and $Y_4=0$. Assume their observed covariate profiles imply conditional probabilities $p_1=0.10$, $p_2=0.25$, $p_3=0.10$, and $p_4=0.05$. Suppose, for this example, that the four outcomes are conditionally independent given their covariates.

Each outcome is in the required support $\{0,1\}$, and each stated probability is in $[0,1]$. The corresponding conditional means are $0.10, 0.25, 0.10,$ and $0.05$; they need not be equal because the covariate profiles differ.

Under the stated conditional-independence assumption, the probability of precisely the observed sequence $(0,1,0,0)$, conditional on the covariates, is

$$ (1-p_1)p_2(1-p_3)(1-p_4) =(0.90)(0.25)(0.90)(0.95) =0.192375. $$

The calculation uses the probability of zero for observations 1, 3, and 4, and the probability of one for observation 2. The factorization is justified here by the example’s explicit assumption of conditional independence; binary support alone would not justify multiplying the four probabilities.

Sample Averages and Conditional Probabilities

A realized sample can provide information about a conditional mean, but its observed average is not identical to the underlying probability in every finite sample. For observations sharing a covariate profile $x$, let there be $n_x$ outcomes, each with conditional success probability $p(x)$, and suppose they are conditionally independent. The sample average of their zero-one outcomes is the observed fraction of successes:

\bar{Y}_x=\frac{1}{n_x}\sum_{i:X_i=x}Y_i.

Because each outcome has conditional mean $p(x)$, linearity of expectation gives $E(\bar{Y}_x\mid X_1,\ldots,X_n)=p(x)$. Thus the group fraction is conditionally unbiased for that probability under the stated common-probability setup. It remains a random sample statistic: a particular group may have a fraction above or below $p(x)$.

Worked Example: A Group Fraction Is Not the Individual Probability

In a hypothetical group of five observations with the same covariate profile, suppose the outcomes are $0,1,0,0,1$. The observed group fraction is

\bar{Y}_x=\frac{0+1+0+0+1}{5} =\frac{2}{5} =0.40.

If these five outcomes are independent Bernoulli draws with a common conditional probability $p(x)$, then $E(\bar{Y}_x\mid X_1,\ldots,X_n)=p(x)$. The observed value 0.40 is therefore an estimate based on this particular group, not proof that the underlying conditional probability is exactly 0.40. A different sample from the same conditional distribution could produce a different fraction.

The group average is also not any one observation’s value: each of the five recorded outcomes remains zero or one. The average summarizes the group’s realized outcomes, while $p(x)$ describes the probability governing each outcome under the assumed conditional model.

Common Mistake: Treating a Probability as a Realized Outcome

A frequent error is to say that a person with a predicted conditional probability of 0.18 has an outcome of 0.18, or that a prediction is wrong whenever the observed outcome differs from the probability. This confuses the conditional mean with the realization. Under a Bernoulli model, an observation with probability 0.18 can still realize either zero or one. A single realization is not expected to equal its probability; the probability describes the distribution across repeated outcomes with the same conditional information.

Another related error is to assume that conditional independence follows automatically because each response is binary. It does not. For example, if several patients share an unobserved clinic-level factor, their outcomes may remain dependent even after conditioning on recorded patient covariates. The correction is to state the sampling and dependence assumptions explicitly and assess whether they are plausible for the data.

Key takeaway: Bernoulli regression models the conditional probability $P(Y_i=1\mid X_i)$, which equals the conditional mean of a zero-one outcome. The outcome itself remains binary, the probability must lie between zero and one, and sample-level independence assumptions must be stated rather than presumed.

Check Your Understanding

Use the definitions and assumptions in this tutorial to answer the following.

If $Y_i$ is a zero-one indicator and $P(Y_i=1\mid X_i)=0.30$, what are $E(Y_i\mid X_i)$ and $\operatorname{Var}(Y_i\mid X_i)$?
Explain why a conditional mean of 0.30 does not mean that the observed outcome can equal 0.30.
What is the support of a binary response, and what range must its conditional mean satisfy?
Does binary support alone imply that observations are conditionally independent? Explain.
A group’s observed success fraction is 0.60. Why need this not equal the underlying conditional probability for that covariate profile?