Why Compare Emax With Sigmoid Emax?
An Emax curve is often a useful starting point for a direct concentration-effect relationship. A sigmoid Emax curve allows an additional parameter, the Hill coefficient \(\gamma\), to change the curve’s steepness. That flexibility can capture a sharper transition, but it can also make the fitted curve more sensitive to noise and the available concentration range. The question is not simply whether the sigmoid model can fit the data at least as well: it can, because it includes Emax as a special case. The question is whether the data support the extra flexibility and whether it improves predictions that matter.
This comparison is especially difficult when observations do not cover both the low-concentration and high-concentration limbs. Earlier material on identifiability and concentration selection explains why parameter estimates depend on where observations lie relative to the transition. Here, the focus is how that limitation should shape a model comparison.
The Models and the Comparison
In these expressions, \(E_0\) is the baseline response, \(E_{\max}\) is the maximum effect above baseline, and \(EC_{50}\) and \(\gamma\) govern the curve’s concentration scale and shape. For a comparison to isolate the question of curve shape, both models should use the same response scale, baseline specification, residual-error model, and data. Otherwise, a difference in fit might be due to those choices rather than the Hill coefficient.
Under the usual nonlinear least-squares assumptions, responses have independent errors with mean zero and a common variance. Under a likelihood comparison, the assumed residual distribution must also be appropriate; for example, independent normally distributed errors with a common variance lead to the ordinary Gaussian likelihood. If the analysis uses weights or a different residual-error structure, fit and compare the models using that same structure in both. A comparison is only as credible as its shared assumptions.
Because the models are nested, fitting sigmoid Emax cannot increase the minimum residual sum of squares. This fact alone does not establish that the added parameter is useful. A model with more parameters can follow random variation as well as systematic curve shape. Assess evidence for \(\gamma\ne1\) using parameter uncertainty, penalized fit criteria, predictive performance, and the concentration range represented by the data.
What Does the Hill Coefficient Change?
The additional parameter changes how quickly the response rises around \(EC_{50}\). A short derivative makes this connection explicit. Write the effect above baseline as \(g(C)=E_{\max}C^\gamma/(EC_{50}^\gamma+C^\gamma)\), and define \(q=(C/EC_{50})^\gamma\). Then \(g(C)=E_{\max}q/(1+q)\), and \(dq/d\log C=\gamma q\). Applying the chain rule gives:
At \(C=EC_{50}\), \(q=1\), so the slope with respect to log concentration is \(E_{\max}\gamma/4\). Thus, with the same \(E_{\max}\) and \(EC_{50}\), increasing \(\gamma\) makes the transition steeper at that point. This is a description of the model’s shape, not evidence that the data estimate that shape precisely. If the observed concentrations are far from the transition, or cover only a narrow range, quite different values of \(\gamma\) may produce similar fitted responses.
Worked Example: Interpreting Uncertainty in Gamma
Suppose hypothetical data include concentrations below and above the estimated transition. With the same baseline and residual model, a sigmoid Emax fit gives \(\hat{\gamma}=1.42\). An approximate standard error for \(\log(\hat{\gamma})\) is \(0.300\). Use a normal approximation on the log scale to assess whether these results clearly distinguish the fitted curve from Emax.
Calculation. A 95% approximate interval for \(\log(\gamma)\) is \(\log(1.42)\pm1.96(0.300)\). Since \(\log(1.42)=0.351\), the interval is \(0.351\pm0.588\), or \((-0.237,0.939)\). Exponentiating the endpoints gives the interval for \(\gamma\):
The interval includes \(\gamma=1\), so this approximate analysis does not clearly distinguish the sigmoid model from Emax at the 95% level. It also allows values below and substantially above one. It would be incorrect to conclude that \(\gamma\) is exactly one, or that the sigmoid model has no value. The interval indicates that the data have not estimated the shape parameter precisely enough to rule out Emax under this approximation.
This Wald interval relies on an approximately normal estimate of \(\log(\gamma)\) and adequate local identification. If the likelihood is asymmetric or nearly flat, a profile-likelihood interval may describe the uncertainty better. Either way, the interval is conditional on the model and error assumptions.
Compare Fit and Prediction, Not Parameter Count Alone
For independent normal errors with common variance, the likelihood-ratio statistic compares the maximized likelihoods of the nested models. In regular conditions, its reference distribution is approximately chi-squared with one degree of freedom, because sigmoid Emax adds one mean parameter. Those conditions include an identifiable parameter, a well-behaved likelihood, and a suitable error model. Weak identification, boundary behavior, or a nearly flat profile can make the usual approximation unreliable. In those circumstances, an apparently precise test result should not replace inspection of the profile and fit diagnostics.
Information criteria provide another way to penalize extra parameters. For Gaussian models with the same observations and an estimated common residual variance, a convenient difference in AIC is:
where \(RSS\) is the residual sum of squares and \(n\) is the number of observations. The \(+2\) is the penalty for the one additional mean parameter; the variance parameter is common to both models and cancels in the difference. A negative difference favors sigmoid Emax by this criterion, but it is not proof that the sigmoid curve is biologically correct. AIC summarizes a fit-versus-complexity trade-off under its likelihood assumptions; it does not guarantee good predictions at concentrations absent from the data.
Predictive performance can be assessed using a held-out set or cross-validation, provided the validation design reflects the intended prediction task. Randomly holding out observations may test prediction within the observed range, whereas predicting beyond that range is extrapolation and needs separate caution. A more complex model can improve the training fit yet perform worse on held-out data.
Worked Example: An AIC Gain With Worse Held-Out Error
In a hypothetical data set of \(n=24\) observations, both models use the same Gaussian residual assumptions. The fitted residual sums of squares are \(30.0\) for Emax and \(22.0\) for sigmoid Emax. On six held-out observations, the prediction errors (observed minus predicted response, in response units) are \(1,-1,2,-2,1,-1\) for Emax and \(1.5,-1.5,2.2,-2.2,1.5,-1.5\) for sigmoid Emax. Compare the training AIC difference and held-out root mean squared error.
AIC difference. Substituting the residual sums of squares gives:
The negative difference favors sigmoid Emax on training-data AIC, because the improved likelihood more than offsets the added-parameter penalty.
Held-out error. For Emax, the sum of squared errors is \(1+1+4+4+1+1=12\), giving \(RMSE=\sqrt{12/6}=\sqrt{2}=1.414\) response units. For sigmoid Emax, the sum is \(2.25+2.25+4.84+4.84+2.25+2.25=18.68\), giving \(RMSE=\sqrt{18.68/6}=\sqrt{3.113}=1.764\) response units. Here, sigmoid Emax has the better penalized training fit but worse held-out RMSE. The small held-out set does not establish that Emax is generally superior; it does show why a training-fit advantage should not automatically be treated as predictive improvement.
When Concentrations Do Not Cover Both Limbs
A comparison is most informative when the data constrain the baseline region, the transition, and the approach to the upper plateau. When observations lie in a narrow region, the models may look nearly identical there even if they imply different curves elsewhere. In particular, if only one limb is observed, the fit may compensate for a change in \(\gamma\) by changing \(E_{\max}\), \(EC_{50}\), or other estimated parameters. A narrow interval around a fitted \(\gamma\) is not automatically reassuring if it depends strongly on constraints, starting values, or the assumed residual model.
Worked Example: Similar Fits, Different Extrapolations
Consider hypothetical baseline-adjusted responses observed only near concentrations \(C=0.2\) and \(C=0.5\), in consistent concentration units. For illustration, take two sigmoid-family parameter combinations, with \(E_0=0\). One is an Emax curve with \(E_{\max}=100\), \(EC_{50}=2\), and \(\gamma=1\). The other has \(E_{\max}=21.9\), \(EC_{50}=0.224\), and \(\gamma=3\). Compare their predictions at the observed concentrations and at \(C=3\).
For the Emax curve, at \(C=0.2\), the predicted response is \(100(0.2)/(2+0.2)=9.09\); at \(C=0.5\), it is \(100(0.5)/(2+0.5)=20.0\). For the \(\gamma=3\) curve, the predictions are:
The predictions are nearly the same at the two observed concentrations. At \(C=3\), however, the Emax curve predicts \(100(3)/(2+3)=60.0\), while the \(\gamma=3\) curve predicts \(21.9(27)/(0.224^3+27)\approx21.9\). The example does not establish that both parameter combinations fit a larger data set equally well; it demonstrates that agreement over a limited observed range need not imply agreement in extrapolation. More observations across the transition and toward a plateau would help distinguish these shapes.
Diagnostics, Alternatives, and Limitations
Before deciding that \(\gamma\) adds useful structure, inspect the fitted curves over the observed concentration range and the residuals from both fits. Look for systematic residual patterns that remain under Emax but are reduced by sigmoid Emax. Also check convergence from multiple plausible starting values, sensitivity to constraints, and whether the profile for \(\gamma\) is well-defined. Large changes in other parameter estimates when \(\gamma\) is freed, or very different curves with nearly equal objective values, are warnings of weak identification rather than evidence for a uniquely established shape.
The main alternatives answer related but distinct questions:
| Approach | What it contributes | Important limitation |
|---|---|---|
| Uncertainty interval or likelihood-ratio comparison | Assesses whether the data distinguish \(\gamma\) from one under a stated model. | Can be unreliable with weak identification or a poor likelihood approximation. |
| AIC or another penalized fit criterion | Balances likelihood improvement against added model complexity. | Does not guarantee accurate prediction outside the observed range. |
| Held-out prediction or cross-validation | Checks predictive performance for the validation design used. | Results can be noisy and do not automatically assess extrapolation. |
| Retain Emax for parsimony | Uses the simpler curve when the extra flexibility is not supported or needed. | May miss a genuinely steeper transition if informative data are lacking. |
No single criterion should be treated as a universal decision rule. If uncertainty includes one, penalized fit results are mixed, and predictions differ chiefly outside the observed range, the defensible conclusion is that the data do not resolve the choice. Report that limitation and avoid claiming the selected curve is validated for unobserved concentrations. Conversely, a reproducible improvement in fit and prediction, a well-behaved uncertainty profile, and adequate concentration coverage together provide stronger evidence for estimating \(\gamma\) rather than fixing it at one.
A Practical Decision Sequence
Use the same observations, baseline specification, response scale, and residual-error assumptions for both models.
Inspect uncertainty and the profile for \(\gamma\), along with convergence and sensitivity to constraints or starting values.
Consider a suitable penalized criterion and validation performance, not just the reduction in training residuals.
When one or both limbs are poorly represented, state that the model comparison is inconclusive for curve shape or extrapolation.
Common Mistake
A common mistake is to reason, “Sigmoid Emax has a smaller residual sum of squares, so the data prove \(\gamma>1\).” The smaller sum of squares follows from allowing an additional parameter; it does not by itself show that the parameter is estimated away from one, that the improvement exceeds what complexity can explain, or that predictions improve. Check uncertainty, account for complexity, assess prediction, and consider whether the data span the curve features that distinguish the models.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- Why can sigmoid Emax never have a larger minimum residual sum of squares than Emax when both are fit to the same data by the same method?
- A 95% interval for \(\gamma\) is \((0.94,1.80)\). What does this say about evidence distinguishing the fitted curve from Emax?
- For \(n=20\), suppose \(RSS_{\text{Emax}}=25\) and \(RSS_{\text{sigmoid}}=20\). Calculate the AIC difference using the stated formula and identify which model it favors.
- Why can observations confined to one side of the transition make estimates of \(\gamma\), \(EC_{50}\), and \(E_{\max}\) difficult to distinguish?
Solutions. 1. Emax is obtained from sigmoid Emax by setting \(\gamma=1\), so the sigmoid model’s set of possible curves contains every Emax curve. Minimizing over that larger set cannot produce a larger minimum residual sum of squares. 2. The interval includes one, so this approximate interval does not clearly distinguish \(\gamma\) from the Emax value; it does not prove that \(\gamma=1\). 3. \(\Delta AIC=20\log(20/25)+2=20\log(0.8)+2\approx20(-0.2231)+2=-2.46\). The negative difference favors sigmoid Emax by AIC, subject to the shared likelihood assumptions. 4. Over a narrow range, changing \(\gamma\) can be offset by changes in the other curve parameters, leaving similar fitted responses. Without observations that constrain the transition and limbs, the data may not identify which parameter combination describes the broader curve.