Linearity in the Predictor Versus Linearity in Parameters

Two Different Meanings of “Linear”

In regression, “linear” can describe two different things. A conditional mean is linear in the predictor when its graph against the original predictor is a straight line. A model is linear in its parameters when its unknown coefficients enter as a linear combination, without being squared, multiplied together, or placed inside a nonlinear function. These meanings are related, but they are not interchangeable.

The distinction matters because transforming a predictor, such as replacing $x$ with $x^2$, can produce a curved relationship with the original predictor while keeping the unknown coefficients linear. Such a model can be treated as a regression on transformed predictor values, but it is not the usual simple linear regression model in $x$.

Definition: The standard simple linear regression conditional-mean model is $$ E[Y\mid X=x]=\beta_0+\beta_1x, $$ where $\beta_0$ and $\beta_1$ are unknown parameters. It is linear in the predictor $x$, because the conditional mean is a straight-line function of $x$, and linear in the parameters, because the coefficients occur only as $\beta_0$ and $\beta_1$, each multiplied by a known quantity.

A useful general form is

E[Y\mid X=x]=\beta_0g_0(x)+\beta_1g_1(x)+\cdots+\beta_pg_p(x),

where each $g_j(x)$ is a known function of the predictor. This is linear in the parameters $\beta_0,\ldots,\beta_p$: the unknown coefficients are combined by addition and multiplication by known values. But the conditional mean need not be a straight line in $x$. For example, $g_2(x)=x^2$ allows curvature.

For the mean model to be specified, the functions $g_j$ must be fixed rather than unknown functions. At each $x$, their values are known numbers. If instead an unknown parameter appears inside a function such as an exponential, or is multiplied by another unknown parameter, the model is generally not linear in its parameters.

What the Standard Model Requires

For observations $Y_i$ at predictor values $X_i$, a standard error representation of simple linear regression is

Y_i=\beta_0+\beta_1X_i+\varepsilon_i.

The zero conditional mean condition $E[\varepsilon_i\mid X_i]=0$ gives $E[Y_i\mid X_i]=\beta_0+\beta_1X_i$. The model for the conditional mean does not, by itself, require normally distributed errors. If the additional normal-error assumption holds, with $\varepsilon_i\mid X_i\sim N(0,\sigma^2)$, then $Y_i\mid X_i=x\sim N(\beta_0+\beta_1x,\sigma^2)$. The normal assumption supports distributional conclusions; it does not determine whether the mean is linear in $x$.

A transformed-predictor model instead might have conditional mean $\beta_0+\beta_1x^2$. It can be written as $Y_i=\beta_0+\beta_1X_i^2+\varepsilon_i$, with zero conditional mean errors. The coefficients remain linear, but the mean is generally curved as a function of the original predictor. This is often called a linear model in the parameters with a transformed predictor.

Worked Examples

Worked Example: A Squared Predictor and Expected Claim Counts

A hypothetical insurer studies the expected annual number of small equipment claims, $Y$, in relation to a machine-age index $X=x$. Consider the model $E[Y\mid X=x]=12+3x+0.5x^2$, where $x$ is measured in index units. For a distributional specification, assume $Y\mid X=x\sim N(12+3x+0.5x^2,4)$; the variance is $4$. No interest rate or mortality assumption is involved. Determine the conditional means at $x=0,2,4$, compare the increases, and classify the model’s linearity.

Method. Substitute each predictor value into the stated conditional mean. The coefficients are $\beta_0=12$, $\beta_1=3$, and $\beta_2=0.5$, multiplying the known functions $1,x,x^2$.

At $x=0$, the conditional mean is $12+3(0)+0.5(0^2)=12+0+0=12$ claims.

At $x=2$, it is $12+3(2)+0.5(2^2)=12+6+0.5(4)=12+6+2=20$ claims.

At $x=4$, it is $12+3(4)+0.5(4^2)=12+12+0.5(16)=12+12+8=32$ claims.

The increase from $x=0$ to $x=2$ is $20-12=8$ claims. The increase from $x=2$ to $x=4$ is $32-20=12$ claims. The increases differ, so the conditional mean is not a straight line in $x$. The model is nevertheless linear in its parameters: each unknown coefficient multiplies a known function of $x$.

Check. The three means are $12,20,32$, and the changes over equal two-unit steps are $8$ and $12$. The changing increments are consistent with curvature. A common mistake would be to call the model nonlinear in its parameters merely because it contains $x^2$; $x^2$ is a known predictor function, not an unknown coefficient.

Worked Example: Classifying Parameter Structure

For a hypothetical portfolio study, let $Y\mid X=x\sim N(m(x),9)$, where the conditional mean under consideration is $m(x)=5+2\sqrt{x}-0.1x^2$, for $x\geq 0$. No interest rate or mortality assumption is involved. Compute the means at $x=1,4,9$ and determine whether this is linear in the predictor and in the parameters.

Method. Evaluate the fixed predictor functions $1,\sqrt{x},x^2$, then multiply by their specified coefficients. The parameter values are $5,2,-0.1$, and the conditional variance is $9$.

At $x=1$, $\sqrt{1}=1$ and $1^2=1$, so $m(1)=5+2(1)-0.1(1)=5+2-0.1=6.9$.

At $x=4$, $\sqrt{4}=2$ and $4^2=16$, so $m(4)=5+2(2)-0.1(16)=5+4-1.6=7.4$.

At $x=9$, $\sqrt{9}=3$ and $9^2=81$, so $m(9)=5+2(3)-0.1(81)=5+6-8.1=2.9$.

The mean is linear in the parameters because it has the form $\beta_0(1)+\beta_1\sqrt{x}+\beta_2x^2$, with known functions multiplying the coefficients. It is not a straight line in the original predictor: the square-root and squared terms change its shape.

Check. Direct substitution gives $6.9,7.4,2.9$. The decrease from $x=4$ to $x=9$, despite the earlier increase, is possible because the squared term has a negative coefficient and grows more rapidly at larger $x$. A common mistake is to assume that any model with one raw predictor must be a simple straight-line regression; transformed terms can create a curved mean.

Worked Example: A Squared Coefficient Is Not Linear in Parameters

A hypothetical maintenance study considers $m(x)=4+2x+\gamma^2x^2$, with $\gamma=3$. Assume $Y\mid X=x\sim N(m(x),1)$; the conditional variance is $1$. No interest rate or mortality assumption is involved. Calculate the means at $x=1$ and $x=2$, then decide whether the displayed model is linear in its parameters.

Method. Substitute $\gamma=3$ and the predictor values. Then inspect how the unknown coefficient appears in the model before its value is substituted.

Since $\gamma^2=3^2=9$, at $x=1$ the mean is $m(1)=4+2(1)+9(1^2)=4+2+9=15$.

At $x=2$, $x^2=4$, so $m(2)=4+2(2)+9(4)=4+4+36=44$.

The displayed model is not linear in its parameters as written, because the unknown $\gamma$ appears as $\gamma^2$. Substituting the numerical value $3$ does not change that classification: linearity is assessed by the model’s form in its unknown parameters, not by its form after the parameters have been assigned values. One could define a new coefficient $\theta=\gamma^2$, in which case the mean is $4+2x+\theta x^2$, linear in $4,2,\theta$. But if the original interpretation requires $\theta=\gamma^2$, then $\theta$ is constrained to be nonnegative and is not an unrestricted replacement for $\gamma$.

Check. The computed means are $15$ and $44$. The squared unknown coefficient is the reason the original specification is nonlinear in parameters. A common mistake is to treat $\gamma^2x^2$ as linear in $\gamma$ because it is a product with $x^2$; $x^2$ is known at a given predictor value, but $\gamma^2$ is still a nonlinear function of an unknown parameter.

Why a Transformed-Predictor Model Is Linear in Its Coefficients

Suppose the conditional mean is $m(x)=\beta_0+\beta_1x+\beta_2x^2$. For any fixed $x$, the quantities $1,x,x^2$ are known. The mean is therefore a sum of known numbers multiplied by the unknown coefficients. More generally, if $m(x)=\sum_{j=0}^{p}\beta_jg_j(x)$, then at fixed $x$ each $g_j(x)$ is just a number, so changing a coefficient changes the mean in direct proportion to that number.

This gives a practical test: hold the predictor value fixed and inspect the expression as a function of the unknown coefficients. If each coefficient appears only to the first power and is not multiplied by another unknown coefficient or placed inside a nonlinear function, the model is linear in its parameters. Then separately ask whether the resulting function of the original $x$ is a straight line.

Identify the original predictor.
Decide whether “linear in the predictor” is being judged against $x$, rather than against a transformed variable such as $x^2$.

Separate coefficients from predictor functions.
Write the mean as coefficients multiplied by known functions, if possible. Terms such as $x^2$ are predictor functions when $x$ is observed.

Inspect how unknowns enter.
A coefficient squared, a product of unknown coefficients, or a coefficient inside a nonlinear function means the specification is not linear in those parameters as written.

Classify both properties separately.
A model may be linear in its coefficients but curved in the original predictor. State both conclusions rather than calling it simply “linear” or “nonlinear.”

Common Exam Trap and Why the Distinction Matters

Consider $m(x)=\beta_0+\beta_1x^2$. A mistaken classification says, “This is nonlinear in the parameters because $x$ is squared.” That confuses a predictor transformation with a parameter transformation. For fixed $x$, the expression is $\beta_0+\beta_1$ multiplied by the known number $x^2$, so it is linear in $\beta_0,\beta_1$. The correction is: it is not linear in the original predictor $x$, but it is linear in its parameters.

By contrast, $m(x)=\beta_0+\beta_1^2x$ is linear in the predictor $x$ once the coefficient values are fixed, but it is nonlinear in the parameter $\beta_1$. The two classifications can therefore differ in either direction.

This distinction helps describe what a regression equation assumes about the conditional mean. A straight-line model says equal changes in $x$ correspond to equal changes in the conditional mean. A transformed-predictor model permits a different pattern while retaining coefficients that enter linearly. Neither choice is automatically appropriate for a particular application; it is essential to name the form being assumed and to interpret it against the original predictor.

Key takeaway: “Linear in the predictor” concerns the shape of the conditional mean as $x$ changes. “Linear in the parameters” concerns how the unknown coefficients appear in the equation. A squared or otherwise transformed predictor can create curvature without making the model nonlinear in its coefficients.

Check Your Understanding

For each problem, distinguish the shape in the original predictor from the parameter structure. Assume the stated conditional distributions and use the given parameter values.

A hypothetical inspection model has $Y\mid X=x\sim N(6+4x^2,4)$. Find the conditional means at $x=0$ and $x=2$, and classify the model in the predictor and parameters. (Answer: $6$ and $22$; curved in $x$, linear in the parameters.)
A hypothetical service model has $m(x)=3+5\sqrt{x}$, for $x\geq0$. Find $m(1)$ and $m(9)$. Is the model linear in the original predictor and in its parameters? (Answer: $8$ and $18$; not a straight line in $x$, but linear in its parameters.)
For $m(x)=2+\alpha x+\delta x^2$, take $\alpha=1$, $\delta=-0.5$, and $x=0,2,4$. Compute the three means and classify the form. (Answer: $2,2,-2$; curved in $x$, linear in $\alpha,\delta$.)
A hypothetical model is $m(x)=7+\eta^2x$, with $\eta=2$. Calculate the means at $x=1$ and $x=3$. Is it linear in the predictor and in the displayed unknown parameter? (Answer: $11$ and $19$; linear in $x$, nonlinear in $\eta$.)
A model is $m(x)=\beta_0+\beta_1x+\beta_2x^3$. At $x=2$, take $\beta_0=1,\beta_1=3,\beta_2=0.25$. Compute the conditional mean and classify the model. (Answer: $1+6+2=9$; not generally a straight line in $x$, but linear in its parameters.)

What You'll Learn