The Conditional Mean and Linear Predictor

Why Separate the Predictor From the Mean?

A generalized linear model (GLM) separates two quantities that can be easy to confuse: a linear predictor, built from covariates and coefficients, and the conditional mean of the response. The predictor is a convenient scale on which to combine covariate effects. The mean is the quantity that describes the response on its original scale. A link function connects the two.

This distinction is especially important for a binary outcome. A linear predictor can be any real number, but the conditional mean of a binary response must be between 0 and 1. The predictor therefore cannot generally be read as a probability. Instead, it is transformed to obtain the probability. The introductory material on GLM (C27) provides the broader GLM context; here the focus is the relationship between these two quantities.

“Linear” refers to the predictor’s form in the coefficients: it is a sum of covariate values multiplied by coefficients. Covariates may themselves be transformed or coded into indicator variables, so the predictor need not be a straight-line function of every original measurement. The vector notation is shorthand for a sum such as \(\eta_i=\beta_0+\beta_1x_{i1}+\cdots+\beta_px_{ip}\).

The conditional mean is defined given the covariates. It is not the observed value \(Y_i\): for a binary response, the observation is either 0 or 1, while the conditional mean is the probability of observing 1. A mean model states how that expected response varies with covariates; it does not determine each individual outcome with certainty.

From Predictor to Conditional Mean

The link \(g\) puts the mean on the predictor scale. For a specified link, first calculate \(\eta_i=x_i^T\beta\), then apply the inverse link to obtain \(\mu_i\). The exact relationship depends on the chosen link. For example, with the identity link \(g(\mu)=\mu\), the mean equals the predictor. With a nonlinear link, the mean is a transformed version of it. These are different model specifications, not two names for the same quantity.

For a binary response coded \(Y_i=1\) when an event occurs and \(Y_i=0\) otherwise, the conditional mean has a particularly direct interpretation:

The logistic link is a common choice for a binary response. It relates the event probability to the predictor by the logit, the logarithm of the odds. Inverting that relationship gives a probability between 0 and 1 for every finite predictor value.

Why the Binary Mean Is a Probability

The probability interpretation follows directly from the definition of expectation for a binary variable. Conditional on \(x_i\), let \(p_i=P(Y_i=1\mid x_i)\); then \(P(Y_i=0\mid x_i)=1-p_i\). Multiplying each possible value by its probability gives the conditional expectation:

Therefore \(\mu_i=p_i\). Under the logistic model, the specified relationship is \(\log\{\mu_i/(1-\mu_i)\}=x_i^T\beta\). Exponentiating both sides yields \(\mu_i/(1-\mu_i)=e^{\eta_i}\). Multiplying by \(1-\mu_i\) and collecting the terms involving \(\mu_i\) gives \(\mu_i(1+e^{\eta_i})=e^{\eta_i}\), hence \(\mu_i=e^{\eta_i}/(1+e^{\eta_i})\). This shows both why the predictor is on a different scale and how it determines the conditional mean.

For a mean model to be coherent, its predicted mean must lie in the allowable range for the response. The logistic inverse satisfies \(0<\mu_i<1\) for every finite \(\eta_i\), as required for a binary event probability. This range property does not guarantee that the selected covariates, coefficients, or overall model are scientifically adequate; it only ensures that the fitted mean is on a possible probability scale.

Coding Covariates and Interpreting the Intercept

The linear predictor also depends on how covariates are represented. For a two-group predictor, one common coding uses \(x_i=0\) for a reference group and \(x_i=1\) for a comparison group. In \(\eta_i=\beta_0+\beta_1x_i\), the reference group’s predictor is \(\beta_0\), and the comparison group’s predictor is \(\beta_0+\beta_1\). Under a logistic link, these quantities are group-specific log odds, while the group-specific conditional means are their logistic transformations.

Assumptions, Diagnostics, and Alternatives

The equations above concern a conditional mean model. For their interpretation, the response and covariates must be defined consistently, the coefficient and link specification must be stated, and the conditional mean must be in the response’s allowable range. In the binary examples, the response is coded 0 or 1 and the logistic inverse supplies a valid probability. The arithmetic does not require independence across observations. However, valid estimation and uncertainty calculations need assumptions about the response distribution and dependence appropriate to the data; a mean equation alone does not settle those issues.

Useful checks focus on whether the specified mean relationship is plausible. For a binary outcome, examine observed event proportions across covariate ranges and compare them with model-based predicted probabilities, accounting for the fact that small groups can have noisy proportions. Look for systematic under- or over-prediction, an implausible shape across a continuous covariate, and predictions that cluster near 0 or 1 where the data do not support such certainty. A predictor can be calculated correctly while the model’s mean structure is still inadequate.

• Confusing scales. A predictor outside the interval from 0 to 1 is not automatically an error for a logistic model; it is on the log-odds scale. Apply the inverse link before interpreting a probability.
• Reading a coefficient as a probability change. Under the logistic link, a slope is a change in log odds per covariate unit. The associated probability change depends on the predictor’s starting value.
• Extrapolating carelessly. The logistic inverse always returns a mathematically valid probability, but predictions far beyond the observed covariate range may be unreliable because the assumed relationship is being extended without data support.
• Using a mean equation as a complete data model. Specifying \(E(Y_i\mid x_i)\) does not by itself specify the full conditional distribution or the dependence among observations.

A direct linear model for a binary response is a possible alternative mean specification, but it can predict values below 0 or above 1 and imposes a constant probability change per covariate unit. The logistic mean model avoids predictions outside the probability range and instead has covariate effects that are linear on the log-odds scale. No link makes a model automatically correct: choice depends on the response scale, the intended interpretation, and how well the mean pattern describes the data.

The central distinction is practical as well as notational. The linear predictor is assembled from covariates and coefficients on a chosen scale; the link determines how that scale maps to the conditional mean. In a binary logistic model, \(\eta_i\) is a log odds and \(\mu_i\) is an event probability. Keeping those roles separate prevents invalid interpretations and helps identify which part of a model’s specification or prediction needs scrutiny.