Feasibility Constraints In Log-Binomial Models

Why Feasibility Needs Attention

A binary regression model must assign a probability between zero and one to every covariate value at which it is used. With the log link, the model writes the logarithm of the risk as a linear predictor. Exponentiating that predictor guarantees a positive value, but it does not guarantee a value at most one. A positive linear predictor produces a number greater than one, which cannot be a probability.

The restriction is simple to state but important in practice: at every relevant covariate value, the linear predictor must be nonpositive. This tutorial develops that condition for a finite design, explains its geometry, and distinguishes valid fitted risks from the separate question of whether a likelihood estimate lies on the boundary.

The Feasible Coefficient Set

For finite coefficients, the exponential is strictly positive. Thus the lower probability bound is automatic, and the upper bound is the one that imposes a constraint. For each observed covariate vector, the equivalences are

Collecting these inequalities gives the design-specific feasible set. If \(X\) is the matrix whose \(i\)th row is \(x_i^\mathsf{T}\), then

where the vector inequality applies row by row. This condition is about the covariate values under consideration. It does not automatically guarantee valid predictions at covariate values outside that set.

Proof. The row-wise equivalence above shows that all fitted means are at most one if and only if \(X\beta\leq 0\). They are already strictly positive for finite \(\beta\), so this is precisely the set of valid fitted means. The set is the intersection of finitely many closed half-spaces, one for each inequality \(x_i^\mathsf{T}\beta\leq 0\). Each half-space is closed and convex, so their intersection is closed and convex; it is a polyhedron because it is described by finitely many linear inequalities. Finally, \(\beta=0\) satisfies every inequality, since \(X0=0\). Therefore the set is nonempty. It is a cone because, if \(\beta\in\mathcal{C}(X)\) and \(c\geq 0\), then \(X(c\beta)=cX\beta\leq 0\). Nothing in the inequalities requires the set to be bounded. \(\square\)

The zero vector always gives the constant risk one, so the finite-design feasible set cannot be empty under this model as stated. Additional requirements can change the problem: for example, an offset \(o_i\) changes the inequalities to \(x_i^\mathsf{T}\beta\leq -o_i\), and an imposed coefficient restriction may also remove zero. Such modified constraints can be infeasible, but that is not the homogeneous finite-design constraint \(X\beta\leq 0\).

Boundary Risks and the Likelihood

The feasible set includes equality. If \(x_i^\mathsf{T}\beta=0\), then \(p_i=1\), a valid boundary probability. This differs from a zero risk: with finite coefficients, \(\exp(x_i^\mathsf{T}\beta)\) is strictly positive, so a fitted risk of exactly zero is not attained. It can only be approached as the linear predictor tends to negative infinity.

Whether a boundary fitted risk is compatible with observed outcomes depends on the data. For a Bernoulli observation \(y_i\in\{0,1\}\), its likelihood contribution is \(p_i^{y_i}(1-p_i)^{1-y_i}\). At \(p_i=1\), this contribution is one if \(y_i=1\), but zero if \(y_i=0\). Thus feasibility alone does not say that a boundary value is a good fit; the observed response matters.

Observed Design Versus Prediction Range

A model may be feasible at every observed covariate value and still produce an invalid prediction elsewhere. If predictions are required over an entire covariate region, the inequalities must hold throughout that region. For a single predictor on an interval, it is enough to check the endpoints.

Proof. Necessity follows because both endpoints belong to the interval. For sufficiency, any \(x\in[a,b]\), with \(a<b\), can be written as \(x=(1-t)a+tb\) for \(t=(x-a)/(b-a)\in[0,1]\). Since \(\eta\) is affine, \(\eta(x)=(1-t)\eta(a)+t\eta(b)\). If both endpoint values are nonpositive, this weighted average is nonpositive. If \(a=b\), the single endpoint condition is already sufficient. \(\square\)

Common Mistakes and Practical Checks

• Assuming exponentiation enforces a probability. It enforces positivity only. Check the upper bound by checking that every relevant linear predictor is nonpositive.
• Checking only the intercept. A nonpositive intercept does not prevent a positive slope or other covariate combination from making the predictor positive.
• Treating equality as invalid. A fitted risk of one is a legitimate probability. It may, however, assign zero likelihood to observed failures at that covariate value.
• Confusing the observed design with a prediction domain. Constraints at the observed rows ensure valid fitted means there, not necessarily at new covariate values.
• Assuming the constraint set can be empty in the unmodified finite-design model. The zero coefficient vector satisfies \(X\beta\leq0\). Extra offsets or coefficient restrictions define a different constraint problem.

In computation, feasibility and likelihood optimization are related but distinct tasks. A numerical optimizer should respect the linear inequalities, and predictions should be checked over the covariate values where they will be interpreted. A boundary estimate can be valid and data-supported, as in the all-success group, but it should not be mistaken for an interior solution or silently treated as evidence that arbitrary extrapolations are valid.