Why Represent a Predictor Nonlinearly?
A GLM’s linear predictor is linear in its coefficients, but it need not be a straight-line function of every original predictor. If a response changes at a diminishing or increasing rate as a predictor grows, a polynomial or piecewise-linear term can represent that pattern while retaining the chosen GLM family and link. These terms change the predictor structure; they do not, by themselves, change the response distribution or link.
For an observation \(i\), write the linear predictor as \(\eta_i\), and let the conditional mean be \(\mu_i=E[Y_i\mid x_i]\). The GLM link \(g\) relates them by \(g(\mu_i)=\eta_i\). A linear effect uses a term proportional to \(x_i\). A nonlinear effect adds transformed terms, such as \(x_i^2\) or \((x_i-k)_+\), to \(\eta_i\). The resulting mean is obtained by applying the inverse link, not by assuming that the mean itself has the same shape as \(\eta_i\).
The family, link, and predictor terms should be specified separately. For example, a Poisson GLM with a log link and a quadratic predictor has a Poisson response conditional on the predictors, a log link for its conditional mean, and a linear predictor containing \(x\) and \(x^2\). Its mean is not a quadratic function of \(x\).
Polynomial Terms in the Linear Predictor
A quadratic specification is
where \(\beta_0,\beta_1,\beta_2\) are fitted coefficients. The \(x_i^2\) term allows the rate of change in \(\eta_i\) to vary with \(x_i\). More generally, terms through degree \(r\) give \(\eta_i=\beta_0+\beta_1x_i+\cdots+\beta_rx_i^r\). A higher degree can fit more complicated shapes, but it also creates greater risk of unstable estimates and implausible behavior, especially near the ends of the observed range and when extrapolating.
In an R formula, write a squared term as `I(x^2)`, for example `y ~ x + I(x^2)`. The `I()` function tells the formula parser to treat the expression inside as arithmetic. Writing `x^2` without `I()` does not reliably specify the intended squared numeric predictor in an R formula. To use a cubic term, include `I(x^3)` as well as the lower-order terms. Keeping lower-order terms makes the polynomial structure explicit and usually gives a more interpretable, hierarchical model specification.
Worked Example: A Quadratic Effect With an Identity Link
A hypothetical study models a continuous outcome \(Y\) using a Gaussian GLM with identity link. Assume observations are independent, and the conditional errors have mean zero and constant variance. The fitted linear predictor is \(\eta(x)=10+2x-0.25x^2\). Calculate the fitted mean at \(x=0,2,4\), compare it with the linear-only model \(\eta_L(x)=10+2x\), and verify the direction of the quadratic bend.
Method and assumptions. With the identity link, \(g(\mu)=\mu\), so the fitted conditional mean equals \(\eta(x)\). The Gaussian model assumptions stated above apply; these computations describe fitted means and do not establish that the model is appropriate.
At \(x=0\), the quadratic contribution is \(0^2=0\), so \(\hat\mu(0)=10+2(0)-0.25(0)=10\). The linear-only model also gives \(10+2(0)=10\).
At \(x=2\), \(x^2=4\), so the quadratic contribution is \(-0.25(4)=-1\). Thus \(\hat\mu(2)=10+4-1=13\). The linear-only fitted mean is \(10+4=14\).
At \(x=4\), \(x^2=16\), so the quadratic contribution is \(-0.25(16)=-4\). Thus \(\hat\mu(4)=10+8-4=14\). The linear-only fitted mean is \(10+8=18\).
Check. The quadratic term subtracts an increasingly large amount as \(x\) grows: the fitted means are 10, 13, and 14, rather than the linear model’s 10, 14, and 18. The derivative of the predictor is \(2-0.5x\), which decreases as \(x\) increases, confirming the downward bend. This is a calculation of fitted values, not evidence that the quadratic form should be selected.
Common mistake avoided. The quadratic term is \(-0.25x^2\), not \((-0.25x)^2\). Squaring the coefficient and predictor together would give a different model.
Piecewise-Linear Effects and Hinge Terms
A hinge term provides a direct way to change slope at a chosen knot \(k\). The model
has the equivalent piecewise form
For \(x_i<k\), the slope in \(x_i\) is \(\beta_1\). For \(x_i>k\), it is \(\beta_1+\beta_2\). At \(x_i=k\), the hinge is zero, so the two pieces meet at the same value, \(\beta_0+\beta_1k\). The left-hand slope at the knot is \(\beta_1\), and the right-hand slope is \(\beta_1+\beta_2\). The derivative at \(k\) exists only if these slopes agree, which requires \(\beta_2=0\).
This slope result follows directly from differentiating on each side of the knot. Below \(k\), \((x-k)_+=0\), so its derivative with respect to \(x\) is zero and the slope is \(\beta_1\). Above \(k\), \((x-k)_+=x-k\), whose derivative is one, so the slope is \(\beta_1+\beta_2\). At the knot, the hinge remains continuous because both side values are zero, even though the one-sided slopes can differ.
In R, a one-knot linear predictor can be written as `y ~ x + I(pmax(x - k, 0))`, using the numerical knot value in place of `k`. Multiple knots can be represented by adding one hinge term for each knot. This creates a continuous piecewise-linear predictor; it does not create a jump in the predictor at a knot. With a non-identity link, the mean is the inverse-link transformation of this piecewise-linear predictor.
Worked Example: A Poisson Log-Link Model With a Hinge
A hypothetical count response is modeled with a Poisson GLM and log link. Assume counts are conditionally independent and, given \(x\), have a Poisson distribution with conditional mean \(\mu(x)\). The specified linear predictor is \(\eta(x)=\log(3)+0.1x+0.2(x-5)_+\). Find the fitted mean at \(x=3,5,7\).
Method and assumptions. The log link gives \(\log\mu(x)=\eta(x)\), hence \(\mu(x)=\exp(\eta(x))\). The conditional Poisson variance is also \(\mu(x)\); that distributional assumption is distinct from the predictor calculation.
At \(x=3\), the hinge is \((3-5)_+=0\). Therefore \(\eta(3)=\log(3)+0.1(3)+0.2(0)=\log(3)+0.3\), and \(\hat\mu(3)=3e^{0.3}\approx4.0496\).
At \(x=5\), the hinge is \((5-5)_+=0\). Therefore \(\eta(5)=\log(3)+0.1(5)+0.2(0)=\log(3)+0.5\), and \(\hat\mu(5)=3e^{0.5}\approx4.9462\).
At \(x=7\), the hinge is \((7-5)_+=2\). The linear contribution is \(0.1(7)=0.7\), and the hinge contribution is \(0.2(2)=0.4\). Thus \(\eta(7)=\log(3)+0.7+0.4=\log(3)+1.1\), and \(\hat\mu(7)=3e^{1.1}\approx9.0125\).
Check. The predictor’s slope is \(0.1\) below 5 and \(0.1+0.2=0.3\) above 5. The fitted mean is continuous at 5, while its rate of change on the response scale need not be the same on either side.
Common mistake and correction. A tempting but incorrect calculation at \(x=7\) is to use only the hinge contribution \(0.2(2)=0.4\) and report \(\log(3)+0.5+0.4=\log(3)+0.9\). That carries forward the linear term at \(x=5\), rather than recalculating it at \(x=7\). The correct linear contribution is \(0.1(7)=0.7\), giving \(\log(3)+1.1\) and a fitted mean of \(9.0125\) to four decimal places.
Worked Example: Two Knots and the Slopes Between Them
A hypothetical continuous outcome is modeled with a Gaussian GLM and identity link, assuming independent errors with mean zero and constant variance. Its fitted predictor is \(\eta(x)=12+1.5x-0.5(x-4)_++0.75(x-8)_+\). Find the fitted means at \(x=2,4,6,8,10\), and state the slopes on the three intervals.
Method. Substitute each value into both hinge terms. A hinge is zero at and to the left of its knot, and equals the excess above the knot to its right. The identity link means the calculated predictor is the fitted conditional mean.
At \(x=2\), both hinges are zero. Thus \(\hat\mu(2)=12+1.5(2)=12+3=15\).
At \(x=4\), both hinges are zero, including the hinge at its knot. Thus \(\hat\mu(4)=12+1.5(4)=12+6=18\).
At \(x=6\), \((6-4)_+=2\) and \((6-8)_+=0\). Thus \(\hat\mu(6)=12+9-0.5(2)=21-1=20\).
At \(x=8\), the hinge values are \((8-4)_+=4\) and \((8-8)_+=0\). Thus \(\hat\mu(8)=12+12-0.5(4)=24-2=22\).
At \(x=10\), the hinge values are \((10-4)_+=6\) and \((10-8)_+=2\). Thus \(\hat\mu(10)=12+15-0.5(6)+0.75(2)=27-3+1.5=25.5\).
For \(x<4\), neither hinge changes with \(x\), so the slope is \(1.5\). Between 4 and 8, the first hinge contributes slope \(-0.5\), giving \(1.5-0.5=1.0\). Above 8, both hinges contribute, giving \(1.5-0.5+0.75=1.75\). The values agree at each knot from either adjacent segment, as required by continuity.
Check and common mistake avoided. At \(x=8\), the second hinge is still zero; it does not contribute until \(x>8\). Including it at \(x=8\) would be numerically harmless because its value there is zero, but treating it as a nonzero contribution would be wrong. At \(x=10\), both hinges must be included, giving 25.5 rather than a value based only on the most recent segment.
Choosing Complexity and Checking the Specification
A more flexible predictor is not automatically a better model. A polynomial can turn sharply near the edge of the observed data, and a piecewise-linear model can make abrupt changes in slope. A knot selected without adequate observations on both sides may produce an unstable estimate of the slope change. Predictions far outside the observed predictor range are especially sensitive to the chosen form.
Compare candidate structures using a consistent fitting criterion and the intended predictive task. Check whether the fitted shape is plausible over the range where predictions will be used, whether the added terms have adequate data support, and whether the model is unnecessarily complicated. A small improvement in fit alone may not justify a fragile shape. The assumed GLM family and link must still be appropriate; changing the predictor terms does not resolve a family or link mismatch.
For an aligned-with-the-SOA Exam PA (Predictive Analytics) syllabus analysis, be prepared to translate a verbal description such as “the relationship changes slope after a threshold” into a hinge term, and to distinguish that from a quadratic bend. Before interpreting or comparing fitted means, write the full linear predictor, evaluate every transformed term, and apply the inverse link only after obtaining \(\eta\).
Check Your Understanding
For each problem, identify the predictor structure and carry the calculation through to the requested fitted value or conclusion.
- A Gaussian GLM with identity link has \(\eta(x)=6+1.2x-0.1x^2\). Find its fitted mean at \(x=3\). (Answer: \(6+3.6-0.9=8.7\))
- A Poisson GLM with log link has \(\log\mu(x)=\log(2)+0.2x+0.1(x-4)_+\). Find the fitted mean at \(x=6\), to four decimal places. (Answer: \(2e^{1.6}\approx9.9061\))
- For \(\eta(x)=4+2x-0.5(x-3)_+\), state the slopes for \(x<3\) and \(x>3\), and whether the derivative exists at \(x=3\). (Answer: Left slope \(2\); right slope \(1.5\); the derivative does not exist at \(x=3\))
- In an R formula, specify a quadratic predictor effect using both the linear and squared terms. (Answer: `y ~ x + I(x^2)`)
- A fitted linear predictor has a hinge at \(k=10\), but the observed predictor values all lie between 1 and 7. What is the hinge contribution for every observed record, and what limitation follows? (Answer: Every hinge value is zero; the observed data provide no information about the post-knot slope change.)