Why Concentration-Response Calibration Needs More Than a Curve Fit
In vitro concentration-response measurements connect an experimental input to a pathway-level output, such as a signaling readout or a functional response. Calibration uses those measurements to estimate parameters in a model that predicts the output across concentrations. The challenge is not simply to draw a smooth curve through the observations: replicate variation, assay precision, and the range of concentrations tested all affect what the data can support.
A mechanistic pathway model may calculate its predicted output by evaluating a steady-state system at each concentration, or by passing the concentration through a reduced response function. In either case, the calibration target is the model-predicted measurement on the scale actually reported by the assay. The saturable curve used below is a useful example of a reduced response function; its fitted parameters are not automatically direct measurements of molecular binding or other isolated biological processes.
For the examples, use an increasing saturable response:
Here \(E_0\) is the baseline response, \(A\) is the response increment at saturation, \(K\) is the concentration giving half of that increment, and \(h\) controls the curve's steepness. The response approaches \(E_0\) as \(C\) approaches zero and \(E_0+A\) as \(C\) becomes large. In an application, some of these quantities may be fixed from design information or calibrated alongside other parameters. Any fixed value should be justified; fixing a parameter does not make it known with certainty.
Observation Error and the Calibration Objective
A simple observation model assumes that replicate measurements are independent conditional on the model and have additive normal error:
The error standard deviation \(\sigma_j\) can vary by concentration or assay condition if that is supported by assay characterization. Under this model, maximum likelihood is equivalent to minimizing the variance-weighted sum of squared residuals:
If the reported response is positive and its variability is approximately proportional to its magnitude, an additive normal model may be inappropriate. A log-scale model can instead assume \(\log(y_{jr})\) is normal around a model prediction on the log scale. The choice must follow the assay's error behavior, not whichever scale produces the most convenient fit. When observations within a condition are correlated, their covariance must also enter the likelihood; treating correlated replicates as independent exaggerates their information.
With independent replicates having a common variance at a concentration, using the replicate mean is equivalent to retaining all replicates for estimating the mean response, provided the variance and replicate count are handled correctly. The variance of that mean is \(\sigma_j^2/n_j\), where \(n_j\) is the number of replicates. Retaining replicate-level data is often preferable because it preserves information about variability and makes the error assumptions easier to inspect.
Worked Example: Estimating a Saturable Curve From Replicates
Suppose a hypothetical assay reports responses in arbitrary assay units. At concentrations \(C=1,2,4\) nM, three replicate responses are respectively \(28,30,32\); \(38,40,42\); and \(48,50,52\). Assume independent additive errors with known standard deviation \(2\) units, and fix \(E_0=10\) units and \(h=1\). Estimate \(A\) and \(K\).
The replicate means are \(30,40,50\) units. Subtracting the fixed baseline gives increments \(d=20,30,40\) units. For \(h=1\), the model is \(d=A C/(K+C)\). Rearrangement gives:
The three observed \(C/d\) values are \(1/20=0.0500\), \(2/30=0.0667\), and \(4/40=0.100\). Their slope against \(C\) is \((0.100-0.0500)/(4-1)=0.0167=1/A\), giving \(A=60.0\) units. The intercept is \(0.0500-0.0167(1)=0.0333=K/A\), so \(K=0.0333(60.0)=2.00\) nM.
The fitted means are \(10+60(1/3)=30\), \(10+60(1/2)=40\), and \(10+60(2/3)=50\) units. Across the nine replicate observations, residuals at each concentration are \(-2,0,2\) units, so the residual sum of squares is \(3(2^2+0^2+2^2)=24\) units squared. With known \(\sigma=2\), the corresponding weighted sum is \(24/2^2=6.00\). These estimates describe this hypothetical dataset under the stated fixed-parameter and error assumptions; they do not establish that the fitted curve is a unique biological mechanism.
What Partial Concentration Coverage Can and Cannot Identify
A concentration series is informative about a parameter only if changes in that parameter produce response changes distinguishable from changes in other parameters and from assay noise. When the tested concentrations cover both the rising portion and the plateau, the response increment \(A\) and the transition scale \(K\) can often be estimated more separately. If all concentrations lie well below \(K\), the model has the approximation:
In that range, the observations mainly constrain the ratio \(A/K\), not each quantity on its own. Many pairs of \(A\) and \(K\) can therefore yield nearly indistinguishable predictions. This is a practical identifiability problem: a numerical optimizer may return precise-looking values even when the data only support a broad trade-off between parameters.
Worked Example: A Low-Concentration Series Leaves a Parameter Trade-Off
Consider the same model with \(E_0=10\) units and \(h=1\), but suppose measurements are available only at \(0.1\) and \(0.2\) nM. Compare \((A,K)=(60\text{ units},2\text{ nM})\) with \((A,K)=(120\text{ units},4\text{ nM})\).
For the first pair, the predicted increments are \(60(0.1)/(2+0.1)=2.86\) units and \(60(0.2)/(2+0.2)=5.45\) units. The total predictions are therefore \(12.86\) and \(15.45\) units. For the second pair, the increments are \(120(0.1)/(4+0.1)=2.93\) units and \(120(0.2)/(4+0.2)=5.71\) units, giving predictions of \(12.93\) and \(15.71\) units.
The predictions differ by only \(0.07\) and \(0.26\) units, respectively, despite the two parameter pairs having different maximum increments and half-response concentrations. If assay error is of comparable or greater size, these data will have little power to distinguish the pairs. In the low-concentration approximation, both pairs have \(A/K=30\) units per nM, explaining their similar predictions. Adding concentrations near the transition and, where feasible, near the plateau can help separate the parameters. If the experiment cannot reach those ranges, report the trade-off rather than interpreting either parameter as independently well determined.
Using Replicate Precision in the Fit
Replicates serve two roles: they help estimate the mean response at a concentration, and they inform the measurement variability. If the assay's standard deviation is already characterized, that information should influence the fit. A response with higher assay variance should generally contribute less than an equally sized residual from a more precise measurement. Conversely, a large number of replicates can improve precision in the estimated mean.
Worked Example: Weighting Residuals by Assay Error
At \(C=1,10,100\) nM, hypothetical replicate responses are \(17,18,19\), \(46,50,54\), and \(78,80,82\) units. Suppose independent replicate errors have known standard deviations \(1,4,2\) units at these three concentrations. Evaluate the weighted residual sum for the candidate curve \(E_0=10\), \(A=80\), \(K=10\) nM, \(h=1\).
The model predictions are \(10+80(1/11)=17.27\), \(10+80(10/20)=50.0\), and \(10+80(100/110)=82.73\) units. The replicate means are \(18,50,80\) units. Since each mean averages three independent replicates, its variance is \(\sigma_j^2/3\). Thus the weighted residual sum based on means is:
The first condition receives more weight because its assay error is smaller; the second has zero mean residual, and the third contributes about \(5.58\) to \(Q\). This calculation evaluates one candidate parameter set; it does not prove it is the best-fitting set. Calibration requires minimizing the chosen objective over the parameters allowed to vary, then checking that the resulting fit and error model are adequate.
Fitting, Diagnostics, and Alternatives
For a nonlinear pathway model, the fitting procedure typically evaluates its predicted response at each tested concentration, calculates the likelihood or weighted residual objective, and searches for parameter values that minimize it. Before interpreting the result, confirm that the model is defined over the tested concentrations, parameters obey biologically and mathematically justified bounds, and the chosen error model matches the assay scale. A good optimizer cannot compensate for a misspecified observation model or a response function that cannot represent the data.
After fitting, plot residuals against concentration and fitted response, and inspect replicate values rather than only their means. Residuals should not show a systematic trend, a concentration-dependent spread inconsistent with the error model, or repeated deviations in one direction. Check whether estimates change substantially with starting values or plausible bounds, and whether profile likelihood or local sensitivity diagnostics reveal a long parameter trade-off. Boundary estimates, near-singular curvature, or very different parameter values with nearly identical predictions signal weak identification, not necessarily an optimizer failure.
The main alternatives differ in how they represent uncertainty and prior information:
| Approach | Useful when | Important caution |
|---|---|---|
| Unweighted least squares | Errors are approximately additive and have similar variance across observations. | It gives imprecise and precise observations equal influence even when their error variances differ. |
| Weighted least squares or likelihood | The assay error distribution and its variance structure are characterized. | Incorrect variance assumptions can distort estimates and uncertainty. |
| Bayesian calibration | Defensible prior information is available for weakly informed parameters. | A narrow posterior can reflect an influential prior as well as the in vitro data; distinguish those contributions. |
The introductory material on assay-specific observation error and replicate correlation provides relevant foundations; here the essential point is to use an observation model consistent with the actual concentration-response measurements. A fitted response curve is also limited by the experimental design: no fitting method can recover a plateau that was never observed without relying on assumptions or external information. In addition, a Hill-shaped response may summarize a pathway's net behavior while failing to uniquely identify its internal steps. Treat its parameters as mechanistic only to the extent that the model structure and data justify that interpretation.
Check Your Understanding
Work through the questions, then compare your reasoning with the solutions.
- For \(\mu(C)=E_0+A C/(K+C)\), what does \(K\) represent when \(h=1\), and what is the limiting response as \(C\) becomes large?
- Why can measurements only at concentrations much smaller than \(K\) fail to distinguish \(A\) from \(K\)?
- At one concentration, three independent replicates have known standard deviation \(\sigma=3\) units. What is the variance of their mean?
- A calibration has residuals that increase systematically with concentration. Give one model-level concern and one error-model concern to investigate.
Solutions. (1) At \(C=K\), \(C/(K+C)=1/2\), so the response is \(E_0+A/2\); as \(C\) becomes large it approaches \(E_0+A\). (2) For \(C\ll K\), the increment is approximately \((A/K)C\), so the data mainly constrain the ratio \(A/K\). (3) The variance is \(\sigma^2/n=3^2/3=3\) units squared. (4) The pathway response function may be missing concentration-dependent behavior, or the assumed constant-variance error model may be inappropriate if assay variability changes with response. Inspect the observations and assay characteristics before choosing between these explanations.