Why Compare These Models?
A concentration-effect data set may be described adequately by a straight line over the observed range, even when a saturating Emax curve is biologically plausible. Conversely, a fitted Emax curve can appear to improve the in-sample fit simply because it has more flexibility. The practical question is not which curve looks more sophisticated, but whether the available data support the extra curvature and whether that choice improves predictions in the concentration range that matters.
The comparison requires particular care when the estimated maximum effect is close to zero. In that situation, the data may provide little information about the concentration at which half of the maximum effect occurs. A numerical EC50 estimate can still be printed, but it may not be a meaningful basis for preferring the Emax model.
Models and Assumptions
Both models describe a direct concentration-effect relationship: the response at a measurement is modeled as depending on the concentration at that measurement, without adding a delay or an indirect response mechanism. Concentration and response units must be specified. For example, if \(C\) is in mg/L and response is measured in units, then \(\beta\) has units of response units per mg/L, while \(EC_{50}\) has units of mg/L and \(E_{\max}\) has units of response units.
For a standard comparison, assume the same observations and response scale are used for both fits. The conditional mean is given by the selected model, residuals are independent, and the error distribution and variance model are appropriate. Under constant-variance normal errors, \(Y_i\) has mean given by its model and a common residual variance \(\sigma^2\). If variability changes with concentration, a suitable weighted or otherwise specified likelihood should be used for both models; comparing criteria computed under incompatible error assumptions is not informative.
The linear model has two mean parameters, \(E_0\) and \(\beta\); the Emax model has three, \(E_0\), \(E_{\max}\), and \(EC_{50}\). When estimating residual variance as well, information-criterion parameter counts include that variance parameter. Concentrations should cover the range relevant to the intended inference, and the design must contain enough information to distinguish curvature from a nearly straight segment.
Why an Emax Curve Can Resemble a Line
At concentrations much smaller than \(EC_{50}\), the Emax effect is approximately proportional to concentration. Dividing the Emax term by \(EC_{50}\) gives:
Thus, over a low-concentration range, the linear slope can approximate the ratio \(E_{\max}/EC_{50}\), even when neither Emax parameter is precisely estimated. This explains why a small difference in fit statistics is not, by itself, persuasive evidence that the observed data establish saturation. The high-concentration behavior also differs: the Emax effect approaches a plateau, whereas a nonzero linear slope continues to change without bound.
The problem is especially acute when \(E_{\max}\) is near zero. The sensitivity of the Emax mean to \(EC_{50}\) is:
As \(E_{\max}\) approaches zero, this derivative approaches zero at every finite concentration. The predicted response then changes very little as \(EC_{50}\) changes, so the data cannot sharply locate it. This is a parameter-identification problem, not evidence that the reported EC50 is a precise estimate.
Residuals, Predictions, and Information Criteria
A residual-versus-concentration plot can reveal systematic lack of fit. A linear model that misses a plateau may show residuals of one sign at intermediate concentrations and the opposite sign at the extremes; an Emax model that imposes unsupported curvature may show its own systematic pattern. Check the residual scale and the variance assumptions as well as the pattern. The earlier material on residual diagnostics for direct concentration-effect fits gives the general diagnostic framework; here the emphasis is on comparing the two candidate mean functions.
Predictive checks ask how well each fitted model predicts observations not used to estimate it. A holdout set or cross-validation can be useful, especially when prediction in a particular concentration range is the objective. The split must preserve the intended use: randomly withholding observations can test interpolation, while withholding an entire high-concentration range tests extrapolation more directly. A favorable prediction check does not prove that a model is mechanistically true.
For fits to the same data under independent normal errors with a common estimated variance, the Akaike information criterion with small-sample correction is:
where \(\ell\) is the maximized log likelihood, \(k\) is the number of estimated parameters including residual variance, and \(n\) is the number of observations. Smaller AICc indicates a more favorable balance of fit and parameter count among the candidate models, for the specified data and likelihood. It is a relative comparison, not a test that establishes the true model.
Under the stated normal, constant-variance assumptions, the maximized likelihood for a model with residual sum of squares \(SSE\) has:
This follows because the normal log likelihood is maximized at the residual variance estimate \(SSE/n\). Substituting that estimate into the likelihood yields the expression above. When \(n\) and the error model are shared, the constants cancel in an AICc difference, but the fit term, parameter penalty, and small-sample correction remain. AICc comparisons can be unstable or unavailable when the sample is too small relative to \(k\).
Worked Example: Comparing AICc Values
Suppose a hypothetical experiment records 12 responses, two at each of six concentrations from 0 to 16 mg/L. The observed responses, in response units, are: at 0 mg/L, 10.1 and 9.9; at 1 mg/L, 11.2 and 11.8; at 2 mg/L, 12.0 and 12.6; at 4 mg/L, 13.3 and 14.1; at 8 mg/L, 15.0 and 15.6; and at 16 mg/L, 16.2 and 17.0. Hypothetical fits using the same normal, constant-variance error model give \(SSE=9.36\) for the linear model and \(SSE=1.55\) for Emax. These are fitted summaries, rounded to three significant figures.
The linear fit estimates \(E_0\), \(\beta\), and residual variance, so \(k=3\). The Emax fit estimates \(E_0\), \(E_{\max}\), \(EC_{50}\), and residual variance, so \(k=4\). For the linear model:
For Emax:
The displayed-value difference is \(23.21-40.07=-16.86\). In this hypothetical comparison, AICc strongly favors Emax among these two fitted models under the chosen likelihood. This result does not establish that Emax is the true mechanism; residual patterns, parameter uncertainty, and predictive performance still matter. The example also assumes the stated error model is appropriate for both fits.
Worked Example: Checking Predictions at a Held-Out Concentration
Suppose two hypothetical models have been fitted to training data: a linear model with \(E_0=10\) response units and \(\beta=0.40\) response units per mg/L, and an Emax model with \(E_0=10\) response units, \(E_{\max}=8\) response units, and \(EC_{50}=4\) mg/L. At a held-out concentration of 20 mg/L, the observed response is 16.4 response units.
The linear prediction is \(10+0.40(20)=18.0\) response units, so its prediction error, defined as observed minus predicted, is \(16.4-18.0=-1.6\) response units. The Emax prediction is:
Its prediction error is \(16.4-16.67=-0.27\) response units, to two decimal places. The squared errors are \(2.56\) and approximately \(0.071\) response-units squared for the linear and Emax models, respectively. Emax predicts this held-out value better. A single held-out observation is weak evidence, however; repeated or cross-validated predictive checks are more informative, and this result concerns prediction at 20 mg/L rather than all possible concentrations.
Worked Example: Interpreting a Near-Zero Emax Estimate
Consider a hypothetical data set whose responses remain close to 5.0 units across concentrations from 0 to 16 mg/L. A fitted Emax model estimates \(E_{\max}=0.10\) response units. At 16 mg/L, an estimate \(EC_{50}=3\) mg/L predicts an effect of \(0.10(16)/(3+16)=0.0842\) response units. If instead \(EC_{50}=30\) mg/L, the predicted effect is \(0.10(16)/(30+16)=0.0348\) response units. Both changes are small relative to the 5.0-unit baseline; whether the data can distinguish them depends on residual variability and the full concentration design.
The derivative with respect to \(EC_{50}\) is also small when \(E_{\max}=0.10\). At \(C=16\) mg/L and \(EC_{50}=3\) mg/L it is \(-0.10(16)/(19^2)=-0.00443\) response units per mg/L. A broad range of EC50 values may therefore give similar predictions and likelihoods. Report the weak identification and examine a profile or prediction uncertainty; do not interpret one optimizer-selected EC50 as precisely determined.
Choosing and Diagnosing the Comparison
Fit both mean functions to the same response observations using the same defensible error structure and likelihood. Confirm that the units, concentration range, and baseline treatment match.
Look for concentration-related patterns, changing spread, influential observations, or systematic curvature that remains under either fit. A lower SSE alone does not diagnose the error model.
Use held-out data or cross-validation where feasible, focusing on the concentration range relevant to the scientific or clinical decision. Distinguish interpolation from extrapolation.
Calculate the criterion using the correct \(k\), then examine whether the apparent preference depends on a weakly identified \(EC_{50}\), a boundary estimate, or a few observations.
A standard likelihood-ratio test should not be used automatically to compare these models. Although Emax can approximate a line over a low-concentration range, the linear model is not simply obtained by fixing one ordinary Emax parameter at a regular interior value. At \(E_{\max}=0\), \(EC_{50}\) is unidentified, and boundary or unidentified-parameter conditions invalidate the usual reference distribution for a routine likelihood-ratio statistic. Information criteria and predictive checks avoid claiming that such a test has its standard calibration, though they have their own limitations.
AICc is useful for balancing fit and complexity, but it depends on the candidate set, likelihood, and data. A small difference should not be treated as decisive. Cross-validation estimates predictive performance but can be noisy, particularly with few observations, and the split must match the prediction task. Residual plots can reveal lack of fit but cannot reliably choose between models when data are sparse or noisy. None of these approaches repairs a design that has no concentrations near the transition or plateau needed to identify Emax parameters.
Common Pitfall
A second overinterpretation is to treat the smaller AICc as proof of a biological mechanism. It indicates relative support within the specified candidate models and assumptions; it does not exclude other response shapes, error structures, or design limitations. If the Emax fit wins but its plateau lies far beyond observed concentrations, the data may support improved interpolation without establishing the plateau itself.
Check Your Understanding
Answer each question, then compare with the solutions below.
- At concentrations small relative to \(EC_{50}\), what linear slope does the Emax model approximately imply?
- Why can \(EC_{50}\) be poorly identified when \(E_{\max}\) is near zero?
- For \(n=15\) observations, a model estimates three mean parameters and residual variance. What is \(k\), and what is its AICc correction term?
- A linear fit has smaller residuals at every observed concentration than Emax, but its residual plot shows a systematic pattern. Does this alone establish that linear is preferable?
- Why is a routine likelihood-ratio test not automatically justified for comparing linear and Emax fits?
Solutions. (1) The approximate slope is \(E_{\max}/EC_{50}\), since \(E_{\max}C/(EC_{50}+C)\approx(E_{\max}/EC_{50})C\) when \(C\ll EC_{50}\). (2) The sensitivity is \(-E_{\max}C/(EC_{50}+C)^2\), which approaches zero as \(E_{\max}\) approaches zero, so changing \(EC_{50}\) barely changes predictions. (3) The count is \(k=4\), including residual variance. The correction is \(2(4)(5)/(15-4-1)=40/10=4\). (4) No. Smaller residuals are an in-sample fit comparison, while systematic residual structure signals possible lack of fit; also examine the appropriate likelihood, predictions, and uncertainty. (5) At \(E_{\max}=0\), \(EC_{50}\) is unidentified, so the usual regularity conditions for the standard likelihood-ratio reference distribution do not hold automatically.