Why an End-to-End Assessment Is Needed
An experiment can produce a smooth, precisely measured response and still provide weak evidence about one or more model parameters. The relevant question is not simply whether the model fits, but whether the planned observations can distinguish plausible parameter values. An end-to-end assessment connects the biological model, the measurement process, local information, and likelihood-based uncertainty to a concrete design decision.
Consider a hypothetical receptor-pathway experiment in which a stimulus is applied at time zero and a normalized downstream response is measured over time. We use this deliberately simple model to make each step visible; it is not intended to represent every receptor mechanism. The parameters are \(A\), the asymptotic response amplitude in normalized response units, and \(k\), the response rate in hours\(^{-1}\).
These assumptions make the likelihood and its local information calculable. Independence means that one measurement's error does not predict another's; constant variance means all observations receive the same error variance in the likelihood. If measurements are correlated, the information calculation must reflect that covariance. If the assay's error changes with response magnitude, a constant \(\sigma\) is not appropriate. Neither a large sample nor a numerical optimizer repairs a misspecified observation model.
From the Observation Model to Local Information
For a candidate parameter pair, define the residual \(r_i=Y_i-f(t_i;A,k)\). Under the stated Gaussian model, minimizing the negative log-likelihood is equivalent to minimizing the weighted residual sum of squares. A convenient deviance-like objective, omitting constants that do not depend on the parameters, is
The local sensitivity matrix \(J\) has one row for each observation and one column for each parameter. Its entries are derivatives of the mean response with respect to the parameters:
With independent errors of common known standard deviation \(\sigma\), the local Fisher information is \(I=J^\mathsf{T}J/\sigma^2\). This is a local approximation evaluated at a specified parameter point; it is not a guarantee that the model is globally identifiable or that the likelihood is well approximated by a quadratic over a wide range.
Worked Example: Fit and Local Information at Four Sampling Times
Suppose the hypothetical measurements are \(0.50000,\ 0.78000,\ 1.10000,\) and \(1.19794\) normalized units at \(t=1,\ 2,\ 4,\) and \(8\) hours. Evaluate the candidate values \(A=1.20\) normalized units and \(k=0.50\) hours\(^{-1}\).
The predictions from \(f(t;A,k)\) are \(0.4722,\ 0.7585,\ 1.0376,\) and \(1.1780\) normalized units. Using the unrounded model predictions, the residuals (observation minus prediction) are approximately \(0.02784,\ 0.02146,\ 0.06240,\) and \(0.01992\) normalized units. In particular, at four hours, \(1.20(1-\exp(-2))=1.0375977\), so the residual is \(1.10000-1.0375977=0.0624023\).
The sum of squared residuals is approximately
Thus \(D\approx 0.00552603/[2(0.10)^2]=0.2763\). This value describes the candidate's residual discrepancy under the assumed error scale; by itself it is not an estimability diagnostic or a test of the model.
At the same candidate parameters, the sensitivity columns, in time order, are approximately \((0.3935,0.6321,0.8647,0.9817)\) for \(A\), and \((0.7278,0.8829,0.6496,0.1758)\) for \(k\). For example, at two hours the second derivative is \(1.20(2)\exp(-1)=0.8829\); at eight hours it is \(1.20(8)\exp(-4)=0.1758\).
The unscaled sums of squares and cross-product are approximately \(2.2657,\ 1.5788,\) and \(1.7622\), respectively. Therefore
The determinant of \(J^\mathsf{T}J\) is approximately \(2.2657(1.7622)-(1.5788)^2=1.50\), which is positive: the two sensitivity columns are not exactly proportional at this point. Their normalized inner product is about \(1.5788/\sqrt{2.2657(1.7622)}=0.789\), showing substantial similarity in the response patterns. Under the local Gaussian approximation, the inverse-information standard errors are approximately \(0.108\) normalized units for \(A\) and \(0.123\) hours\(^{-1}\) for \(k\). These are local approximations, not profile intervals.
Profiles Add a Nonlinear Check
A profile likelihood examines uncertainty in one parameter while allowing the other parameter to adjust. For example, at each fixed value \(k_0\), minimize \(D(A,k_0)\) over the allowed values of \(A\). Write the resulting minimum as \(D_{\mathrm{prof}}(k_0)\), and write the joint minimum as \(D_{\min}\). For a regular one-parameter profile, an approximate 95% likelihood-ratio interval includes values satisfying \(2[D_{\mathrm{prof}}(k_0)-D_{\min}]\leq 3.84\). This cutoff is an asymptotic reference, not an exact finite-sample guarantee.
For this model, the conditional best-fitting amplitude can be calculated directly at fixed \(k_0\). Let \(g_i(k_0)=1-\exp(-k_0t_i)\). Minimizing \(\sum_i[Y_i-A g_i(k_0)]^2\) over \(A\) gives
provided the denominator is nonzero. This simple conditional calculation is useful here; in larger QSP models, profiling generally requires numerical optimization over the nuisance parameters at each fixed parameter value.
Worked Example: Constructing a Profile Interval for the Rate
Use the four measurements in the preceding example. First minimize \(D(A,k)\) jointly over positive \(A\) and \(k\); for this hypothetical data set, numerical minimization gives values near \(\widehat A=1.23\) normalized units and \(\widehat k=0.52\) hours\(^{-1}\). The local standard error for \(k\) from the candidate-point information matrix was about \(0.123\) hours\(^{-1}\), so it suggests the scale of uncertainty, but it does not define the profile interval.
For a profile calculation, fix a trial rate, such as \(k_0=0.50\) hours\(^{-1}\), calculate each \(g_i(k_0)\), substitute the observations into the formula for \(\widehat A(k_0)\), and then evaluate \(D(\widehat A(k_0),k_0)\). Repeat over a grid of \(k_0\) values, refining near the points where the likelihood-ratio difference reaches its cutoff. For these illustrative data, the resulting approximate 95% profile interval is about \(0.325\) to \(0.84\) hours\(^{-1}\). The interval is obtained by locating where \(2[D_{\mathrm{prof}}(k_0)-D_{\min}]\) reaches \(3.84\), not by interpreting \(0.52\pm1.96(0.123)\) as an exact profile calculation.
A profile can reveal asymmetry, a long tail, or a boundary that a local standard error hides. Its numerical endpoints must be checked for optimizer convergence: at each fixed \(k_0\), nuisance-parameter optimization should be repeated from sensible starting values, and the profile should be stable to a finer grid and tighter tolerances.
Revise the Design for the Weakly Distinguished Parameters
The Fisher matrix indicates how the specified observations change the predicted responses near the candidate parameters. If two columns of \(J\) are nearly parallel, parameter changes can compensate for one another, making their effects difficult to distinguish. A small determinant relative to the scale of the matrix, a high sensitivity-column correlation, or a very small singular value can flag this problem. Parameter units and scaling matter when comparing raw matrix entries, so inspect scaled sensitivities as well as the unscaled matrix.
The sensitivity-column correlation in the four-time example is about \(0.789\). This is not the correlation of the parameter estimates: under the local approximation, the off-diagonal element of \(I^{-1}\) has the opposite sign, and the corresponding parameter correlation is about \(-0.789\). Both facts describe local geometry, not a causal relationship between biological parameters.
Worked Example: Assessing a Late Additional Sample
Suppose the experiment can add one measurement at 12 hours with the same error standard deviation, \(0.10\) normalized units. At \(A=1.20\) and \(k=0.50\), its sensitivities are
The added row contributes its squared sensitivities and their product to the unscaled information. The updated sums are approximately \(3.2608\) for the \(A\) column, \(1.6144\) for the cross-product, and \(1.7635\) for the \(k\) column. Thus the updated unscaled determinant is about \(3.2608(1.7635)-(1.6144)^2=3.144\), and the sensitivity-column correlation falls to approximately \(1.6144/\sqrt{3.2608(1.7635)}=0.673\).
Under these local calculations, the late sample adds strong information about \(A\) while contributing little direct sensitivity to \(k\), and it improves the separation of the two sensitivity patterns. It is therefore a plausible design revision if estimating both parameters is important. The determinant and correlation alone do not establish that this is the best design: the cost of the sample, the actual measurement error at 12 hours, biological relevance of late response, and performance across plausible parameter values also matter.
Diagnostics, Alternatives, and Limitations
A defensible assessment checks the chain from model to design rather than reporting a single information number. Inspect residuals against time for systematic patterns; such patterns can indicate a wrong response shape or time origin. Check that the assumed error scale is credible and that measurements are genuinely independent under the experimental procedure. Recalculate derivatives by finite differences or automatic differentiation and compare them with analytic sensitivities. For profile calculations, inspect convergence, multiple starting values, and whether the likelihood continues to rise toward a parameter boundary.
Fisher diagnostics and profile likelihood answer related but different questions. Fisher information gives a local, relatively inexpensive summary around a parameter point and is useful for comparing nearby design choices. A profile likelihood allows nonlinear movement in nuisance parameters and can expose asymmetric or extended uncertainty, but it costs more computation and depends on successful optimization. A Wald interval based on an inverse-information standard error is a still faster local summary; it is least persuasive when the likelihood is curved, parameters are near constraints, or the sensitivity matrix is nearly rank deficient.
None of these calculations proves that the biological mechanism is correct. The one-exponential model omits possible delays, feedback, multiple response phases, and assay effects. The constant known error assumption may be unrealistic, and the local results depend on the parameter values at which sensitivities were evaluated. If the fitted parameters are poorly determined, design comparisons should be repeated over scientifically plausible parameter values rather than relying on one nominal point.
State what is measured, when, in what units, and how measurement errors are represented.
Check fit and residual patterns without treating a small residual sum as proof of estimability.
Check derivatives, matrix rank and scaling, and translate inverse-information results into parameter units.
Use profiles to examine nonlinear uncertainty, then evaluate candidate samples under the same observation assumptions.
Check Your Understanding
Answer each question using the model and assumptions stated above. Brief solutions follow the questions.
- What is the expected response at \(t=0\), and what does that imply about the information in a measurement at that time for \(A\) and \(k\)?
- Why does the 12-hour sample increase information about \(A\) more directly than information about \(k\) at the candidate parameter values?
- If the sensitivity columns are highly correlated, what does that suggest about estimating the two parameters separately?
- What quantity is compared with \(3.84\) when constructing the approximate 95% profile interval for one parameter?
- Name two diagnostics that should be checked before trusting a profile interval or local Fisher calculation.
Solutions. (1) The expected response is \(A(1-\exp(0))=0\), and both sensitivities are zero at \(t=0\), so that observation contributes no local information about either parameter in this model. (2) At 12 hours, the amplitude sensitivity is about \(0.9975\), while the rate sensitivity is only about \(0.0357\); the added observation therefore has a much larger direct sensitivity to \(A\). (3) Similar sensitivity patterns allow changes in one parameter to be offset by changes in the other, making separate estimates less precise. (4) Compare \(2[D_{\mathrm{prof}}(k_0)-D_{\min}]\) with \(3.84\). (5) Suitable checks include verifying derivatives against finite differences, inspecting residual patterns and the error assumptions, testing optimizer convergence from multiple starting values, and checking profile stability under grid refinement.