Building Candidate Curves from Subject Knowledge

From Scientific Expectations to Regression Forms

A scientific description often tells us more than whether a response should increase or decrease. It may suggest that an effect begins only after a threshold, that each additional unit has a smaller effect than the previous one, or that the response decreases and then increases. These expectations can guide the construction of a small, interpretable set of candidate regression curves before any coefficients are estimated.

A candidate curve is a proposed form for the mean response as a function of a predictor. It is not a claim that every observation will lie on that curve, nor does choosing a form prove a scientific mechanism. The aim here is to translate a stated expectation into a mathematical shape, specify the domain where that shape makes sense, and interpret what its terms imply.

For the examples, suppose observations follow \(Y_i=m(x_i)+\varepsilon_i\), where \(x_i\) is the predictor value for observation \(i\), and \(\varepsilon_i\) is its error around the mean function. Assume the errors are independent, have conditional mean zero, and have a common finite variance \(\sigma^2\): \(E(\varepsilon_i\mid x_i)=0\) and \(\operatorname{Var}(\varepsilon_i\mid x_i)=\sigma^2\). These assumptions define a familiar regression setting; they do not determine the shape of \(m(x)\). The scientific expectation guides the candidate mean function, while the error assumptions describe variation around it. The examples below are hypothetical illustrations, not results from real studies.

Recognizing the Shape Implied by Subject Knowledge

Begin by describing the expected response in ordinary language: Does the response change gradually, change rate at a particular point, level off, or reverse direction? Then choose a function whose behavior matches that description over the predictor range that matters. The coefficients give the form flexibility, but the form itself imposes important restrictions.

Here \(\beta_0,\beta_1,\beta_2\) are coefficients and \(x_0\) is a positive reference value with the same units as \(x\). The hinge term is defined by \((x-c)_+=\max(x-c,0)\). A logarithm in a regression expression should have a dimensionless argument; the ratio \(x/x_0\) provides one. These forms are not interchangeable descriptions of “nonlinearity”: each encodes a different pattern of change.

The hinge construction separates a baseline trend from a change in trend. The coefficient \(\beta_2\) is the change in slope at \(c\), not the slope after \(c\) by itself. If the expected response instead starts only after a threshold, subject knowledge might support a different form, such as a hinge term without the baseline linear trend. The choice should reflect the stated expectation rather than convenience alone.

The logarithmic form is one way to express diminishing increases, not a universal model of saturation. It keeps increasing for every positive \(x\), rather than approaching a finite upper limit. If scientific knowledge requires a strict upper bound or a plateau, that expectation calls for a candidate with a corresponding limiting behavior. The form should not be assigned a stronger interpretation than its mathematics supports.

Encoding a U-Shaped Expectation

A quadratic mean function can express a response that decreases and then increases, provided its curvature and turning point are appropriate. For \(m(x)=\beta_0+\beta_1x+\beta_2x^2\), the coefficient of \(x^2\) controls the direction of curvature. If \(\beta_2>0\), the curve bends upward. Completing the square shows where its minimum occurs:

Because a square is nonnegative and \(\beta_2>0\), the first term is smallest when \(x=-\beta_1/(2\beta_2)\); that is the quadratic’s minimum. A U-shaped curve over the predictor values of interest requires more than a positive quadratic coefficient: the minimum must also fall within that range. Otherwise the curve may appear only increasing or only decreasing over the observed region.

A Practical Construction Process

The following sequence keeps the curve tied to subject knowledge and makes its implications explicit before coefficients are estimated.

It can be useful to retain two or three scientifically plausible forms when the verbal expectation does not identify a unique function. For instance, “diminishing gains” does not specify exactly how quickly gains decline. Candidate forms should differ for a reason that can be explained, rather than being collected simply because they are mathematically available. At this stage, the task is to formulate the possibilities—not to declare which curve fits best.

Common Mistakes and Why Formulation Matters

• Using a threshold coefficient as the post-threshold slope. In the hinge model, the post-threshold slope is the baseline slope plus the hinge coefficient. At the threshold itself, the curve is continuous but may have no single derivative.
• Calling every increasing concave curve “saturation.” The logarithmic form has diminishing increases but does not approach a finite ceiling. Its actual limiting behavior matters.
• Assuming a positive quadratic coefficient guarantees a U shape in the data range. It guarantees upward curvature; the minimum must also be within the relevant predictor range for the curve to decrease and then increase there.
• Ignoring domains and units. A logarithmic predictor must be positive, and a log argument should be dimensionless. A slope’s units are response units per predictor unit.
• Reading a candidate as a causal claim. A mean-function shape describes an association under a regression model. By itself, it does not establish that changing the predictor causes the response to change.