When an Identity Link Produces Invalid Probabilities

Why the Identity Link Needs Special Care

For a binary outcome, a fitted mean is also a fitted probability. The identity link makes that mean a linear function of the predictors. This gives coefficients a direct additive interpretation, but unlike links such as the logit or probit, a linear function is not automatically restricted to the interval from zero to one. A computed value outside that interval is not a small interpretive inconvenience: it cannot be a probability.

The key distinction is between the mean model and the parameter values and predictions allowed by the binomial likelihood. An identity-link binomial model has to satisfy probability constraints wherever its likelihood is evaluated. A prediction at a new covariate value may nevertheless be invalid, and a fitting procedure that ignores the constraints can return inadmissible fitted values.

Definition: For independent binomial observations $Y_i\sim\mathrm{Binomial}(n_i,p_i)$, an identity-link model specifies $$ \operatorname{E}(Y_i\mid x_i)=n_i p_i,\qquad p_i=\mu_i=\eta_i=x_i^\mathsf{T}\beta. $$ Here $x_i$ is the predictor vector, $\beta$ is the coefficient vector, and $\eta_i$ is the linear predictor. The model is a valid binomial probability model only when $0\le x_i^\mathsf{T}\beta\le1$ for every covariate value at which the model is used. For interior likelihood calculations, $0<x_i^\mathsf{T}\beta<1$.

This setup assumes the response counts follow the stated binomial distributions, with the specified numbers of trials $n_i$, and that observations are independent under the model. The conditional success probability is represented by the stated linear mean structure. These assumptions do not make the linear predictor a probability automatically: the range restriction must still be checked.

A Valid Fit Can Give an Invalid Extrapolation

An identity-link fit can be valid for the observed predictor values and invalid for a new value. The reason is that fitting only checks the likelihood at the covariate values included in the data; it does not make the fitted linear function stay between zero and one everywhere.

Worked Example: A Valid Grouped Fit and an Invalid Prediction

Suppose hypothetical independent binomial groups have $n_i=10$ trials each. At predictor value $x=0$, there are 2 successes; at $x=1$, there are 8. Fit the identity-link model $p(x)=\beta_0+\beta_1x$.

With two distinct predictor values and two coefficients, the model can match the two observed proportions. The group proportions are $2/10=0.2$ and $8/10=0.8$, so $$ \hat\beta_0=0.2,\qquad \hat\beta_0+\hat\beta_1=0.8,\qquad \hat\beta_1=0.8-0.2=0.6. $$ At the observed predictor values, the fitted means are $p(0)=0.2$ and $p(1)=0.8$. Both are valid probabilities. In this saturated two-group fit, these are also the binomial maximum-likelihood estimates of the two group probabilities.

Now consider predicting at $x=1.5$. Substitution gives $$ \hat p(1.5)=0.2+0.6(1.5)=0.2+0.9=1.1. $$ This exceeds one, so it is not a valid probability. The invalid value is an extrapolation beyond the observed predictor range, not evidence that either observed group had an invalid fitted mean. The model’s additive extension has no probability-range protection at the new value.

Why an Unrestricted Linear Fit Can Fail

The issue is not confined to predictions far outside the observed range. A line fitted without binomial likelihood constraints can produce an invalid value at an observed predictor value. Ordinary least squares is one such unrestricted fitting method. It may be used for a linear probability model, but its estimates are not the maximum-likelihood estimates of a binomial identity-link GLM.

Worked Example: An Unrestricted Fit Gives a Negative Fitted Mean

Consider hypothetical binomial groups, each with 10 trials: at $x=0$, there are 0 successes; at $x=1$, 0 successes; and at $x=2$, 10 successes. Fit a straight line to the three observed group proportions using ordinary least squares.

The proportions are $0,0,1$. With equally sized groups, fitting these group proportions by least squares gives the same line as fitting all 30 individual binary outcomes by least squares. The mean predictor is $\bar x=1$, and the mean response proportion is $\bar y=1/3$. The slope is $$ \hat\beta_1 =\frac{\sum_i (x_i-\bar x)(y_i-\bar y)} {\sum_i (x_i-\bar x)^2} =\frac{(-1)(-1/3)+(0)(-1/3)+(1)(2/3)} {(-1)^2+0^2+1^2} =\frac{1}{2}=0.5. $$ Thus the intercept is $\hat\beta_0=\bar y-\hat\beta_1\bar x=1/3-0.5=-1/6$. The fitted mean at the observed value $x=0$ is $-1/6$, approximately $-0.167$, which is not a possible probability.

This calculation illustrates a limitation of the unrestricted least-squares fit, not a valid binomial maximum-likelihood fit. A binomial likelihood cannot use a negative success probability. The example shows why merely writing a linear mean function does not make every method of estimating its coefficients suitable for binomial data.

What the Binomial Likelihood Requires

For a group with $y_i$ successes in $n_i$ trials, the binomial log-likelihood contribution, apart from a constant, is $$ \ell_i(p_i)=y_i\log(p_i)+(n_i-y_i)\log(1-p_i). $$ This expression is defined for $0<p_i<1$; the likelihood has well-defined limiting values at the boundary when the observed count supports them. It is not a real-valued binomial log-likelihood for $p_i<0$ or $p_i>1$. Consequently, an identity-link binomial fit must restrict its coefficient estimates so that all fitted means at the observed predictor values remain in $[0,1]$. An algorithm that steps outside the feasible region cannot evaluate the binomial likelihood there.

This gives a useful short justification for why invalid output needs careful interpretation. If a program reports an in-sample fitted mean outside $[0,1]$, that output cannot be the result of maximizing the ordinary binomial likelihood over its valid parameter space. Possible explanations include an unconstrained fitting criterion, an approximation or algorithm that failed to respect the feasible region, or confusion between observed-design fitted values and predictions at new covariates. Check which explanation applies before treating the coefficients as a binomial GLM fit.

Specify the domain.
State the predictor values for which the model must give probabilities. The observed values and the intended prediction range may differ.

Check the fitted means.
Compute $x_i^\mathsf{T}\hat\beta$ for each observed row and for relevant prediction values. Verify every value is between zero and one, inclusive.

Identify the fitting method.
Determine whether the estimates came from a binomial likelihood with valid constraints or from a method such as unrestricted least squares. An out-of-range in-sample value is a warning that the reported fit is not an admissible binomial likelihood fit.

Limit interpretation to valid predictions.
Do not report an out-of-range fitted mean as a probability. Reconsider the prediction domain or use a model and fitting procedure that provide valid probabilities for the intended use.

Constraints Over a Bounded Predictor Range

For one predictor $x$, suppose predictions are required over a bounded interval $a\le x\le b$, with $a<b$. The identity-link model is $p(x)=\beta_0+\beta_1x$. Because this function is linear, its values between the endpoints lie between its endpoint values. Therefore it is enough to require $$ 0\le\beta_0+\beta_1a\le1,\qquad 0\le\beta_0+\beta_1b\le1. $$ These endpoint constraints ensure valid predictions throughout the interval. They do not ensure validity beyond it.

The same reasoning extends to a bounded predictor region with a finite set of vertices: a linear function reaches its minimum and maximum over that region at vertices. Checking the predictions at all vertices therefore checks the entire region. This is a way to define the coefficient constraints for a specified use domain, not a guarantee that the model will extrapolate validly outside it.

Worked Example: Checking Constraints on a Bounded Range

Suppose a hypothetical identity-link model is $p(x)=0.10+0.45x$, and predictions are needed only for $0\le x\le2$. At the endpoints, $$ p(0)=0.10+0.45(0)=0.10,\qquad p(2)=0.10+0.45(2)=1.00. $$ Both endpoints are valid. Since $p(x)$ is linear, every value for $0\le x\le2$ lies between $0.10$ and $1.00$, so the model gives valid probabilities throughout this specified interval.

At $x=2.2$, however, the prediction becomes $$ p(2.2)=0.10+0.45(2.2)=0.10+0.99=1.09. $$ The model is invalid at that new value. The endpoint check established validity only on $[0,2]$; it did not establish global validity.

Why the Link Cannot Bound a Nonconstant Line

The source of the problem can be seen directly from the model form. If $p(x)=\beta_0+\beta_1x$ and $\beta_1>0$, then $p(x)$ eventually exceeds one as $x$ increases without bound. If $\beta_1<0$, it eventually becomes negative as $x$ increases. Thus, over the entire real line, a nonconstant identity-link line cannot stay in $[0,1]$. Only a constant line with $\beta_1=0$ and $0\le\beta_0\le1$ remains valid everywhere.

This is why the identity link can be useful when additive probability changes are central and the prediction domain is limited, but it requires particular attention to the domain and fitting constraints. The logit and probit links, by contrast, map any finite linear predictor to a value strictly between zero and one. Their response-scale effects are not generally constant additive changes, so the choice involves both interpretation and validity, not a claim that one link is always appropriate.

Diagnostics and Common Mistakes

Before interpreting an identity-link binomial fit, inspect fitted means at the observed design points and at the covariate values where predictions will be used. Values very close to zero or one also deserve attention: they may indicate a boundary fit, and small changes in estimates or covariates can move predictions outside the valid range. Review convergence and warnings, and check whether the estimation procedure explicitly enforced the probability constraints.

Mistake: treating every linear prediction as a probability. The linear predictor is a probability only when its value is between zero and one.
Mistake: concluding the binomial likelihood produced an invalid in-sample probability. An out-of-range observed-design value is not an admissible fitted mean under the ordinary binomial likelihood; investigate the fitting procedure or implementation.
Mistake: checking only observed covariate values. A fit valid at the observed values can fail at a new value, as the first worked example showed.
Mistake: clipping values below zero to zero or above one to one and proceeding as if nothing changed. Clipping changes the model’s predictions and does not repair the fitted likelihood or its interpretation.

The main alternatives are to keep the identity link but use a procedure that enforces probability constraints on a clearly defined covariate domain, or to choose a link that maps the real line into the probability interval. A constrained fit may have boundary estimates or be sensitive to the chosen domain. A bounded link avoids out-of-range predictions at finite predictors, but changes how coefficients translate into response-scale differences. Whichever approach is used, assess predictions where they will actually be interpreted.

Key takeaway: The identity link makes the binomial mean linear but does not keep it in the probability range. A valid binomial fit must respect the likelihood’s constraints at the design points, and predictions must be checked separately over their intended covariate domain.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions below.

A fitted model is $p(x)=0.15+0.40x$. What is the predicted probability at $x=2$, and is it valid?
Why does an identity-link binomial likelihood not accept an observed-design fitted mean of $1.08$?
For $p(x)=\beta_0+\beta_1x$ on $1\le x\le3$, which two values must be checked to ensure the model stays in $[0,1]$ throughout this interval?
Does a valid fit at every observed predictor value guarantee valid predictions at new predictor values? Explain.

Solutions. (1) $p(2)=0.15+0.40(2)=0.95$, which is a valid probability. (2) A binomial success probability must be between zero and one; $1.08$ is not a possible probability, so the ordinary binomial likelihood is not defined there. (3) Check $p(1)=\beta_0+\beta_1$ and $p(3)=\beta_0+3\beta_1$. Since the function is linear, valid endpoint values imply valid values between them. (4) No. The linear predictor may leave $[0,1]$ outside the observed covariate values, so predictions at new values must be checked separately.