From Structural Identifiability to Data-Based Uncertainty
A model can have parameters that are uniquely determined by ideal, noise-free observations and still provide little information about them when fitted to real data. In practice, observations are finite and noisy. Two substantially different parameter values may then produce predictions that are difficult to distinguish at the sampled times or doses. The fitted model may describe the observations well, even though the data leave some parameters highly uncertain.
Structural identifiability asks whether ideal observations, under specified conditions, uniquely determine a parameter. Practical identifiability asks a different question: given this particular finite data set and its observation-error assumptions, how tightly are the parameters constrained? Structural identifiability is a property of the model and observation setup under idealized conditions. Practical identifiability is conditional on the data, their precision, the sampling design, and the analysis assumptions.
The distinction matters in QSP because a complex mechanistic model can have many structurally identifiable parameters, while a particular experiment provides little information about some of them. A fitted value alone does not reveal this limitation. Uncertainty estimates, considered alongside the model and data, help show which parameters are well constrained and which are not.
What Parameter Uncertainty Tells Us
An uncertainty interval summarizes the range of parameter values compatible with the data under stated assumptions and a specified inferential procedure. A narrow interval relative to the scientifically relevant range indicates that the data constrain the parameter more strongly. A wide interval indicates that substantially different values remain plausible under the model and error assumptions. The interval's units are the parameter's units.
For example, a 95% confidence interval is produced by a procedure that, under its assumptions, covers the true parameter in 95% of repeated data sets. It does not mean there is a 95% probability that the fixed parameter lies in the particular interval calculated from the observed data. Nor does a broad interval prove structural nonidentifiability: it can reflect sparse or noisy observations, an uninformative design, or a parameter near a boundary.
A practical assessment should therefore state the assumed observation model and examine the fitted uncertainty in context. The examples below use independent Gaussian errors with known standard deviation and fixed observation settings. These assumptions make the calculations transparent; they are not automatically appropriate for every QSP data set.
Worked Uncertainty Calculations
Worked Example: A Precisely Estimated Concentration Scale
Suppose four hypothetical measurements, in nM, are modeled as independent observations \(Y_i=\theta+\varepsilon_i\), where \(\theta\) is a concentration scale and \(\varepsilon_i\sim N(0,1^2)\) nM. The error standard deviation of 1 nM is assumed known. The observed measurements are 8.8, 9.4, 9.0, and 8.8 nM. Estimate \(\theta\) and calculate a 95% confidence interval.
Check the assumptions. The model assumes a common mean, independent Gaussian errors, and a known standard deviation of 1 nM. For this hypothetical calculation, suppose those assumptions are appropriate; the four measurements are treated as replicates under the same conditions.
The maximum-likelihood estimate is the sample mean:
Because the error standard deviation is known, the standard error of the mean is \(SE=1/\sqrt{4}=0.500\) nM. Under the assumed Gaussian model, the 95% interval is \(\bar{x}\pm1.96SE\):
The estimate is 9.00 nM, and the interval is relatively narrow on this measurement scale. Under the assumptions, this data set constrains the common concentration scale more closely than it would if the interval were much wider. The interval describes uncertainty in \(\theta\), not the range in which 95% of individual future measurements must fall.
The next example shows why structural identifiability alone does not guarantee useful precision. Consider a simple response model with two parameters and a controlled setting \(q\). Different settings produce different sensitivities to the parameters; the model can distinguish them in ideal data even when a particular finite, noisy design does so poorly.
Worked Example: A Structurally Identifiable but Weakly Constrained Parameter
Suppose the response, in arbitrary assay units, follows \(Y=\theta_1+q\theta_2+\varepsilon\), with independent errors \(\varepsilon\sim N(0,1^2)\). There are two observations at \(q=0\), with values 9.8 and 10.2, and two at \(q=0.1\), with values 10.4 and 10.6. Assume the error standard deviation is known to be 1 assay unit and that the parameters are not restricted by bounds.
Check structural identifiability and assumptions. At \(q=0\), the noise-free response is \(\theta_1\); at \(q=0.1\), it is \(\theta_1+0.1\theta_2\). These two distinct equations uniquely determine both parameters, so this model is structurally identifiable under these settings. For the uncertainty calculation, assume the observations are independent Gaussian errors with the stated common standard deviation.
The mean response at \(q=0\) is \((9.8+10.2)/2=10.0\), and at \(q=0.1\) it is \((10.4+10.6)/2=10.5\). Thus:
so \(\hat{\theta}_2=(10.5-10.0)/0.1=5.00\) assay units. Each mean is based on two observations with variance 1, so each mean has variance \(1/2=0.5\). Since the two means use disjoint independent observations, their errors are independent. Therefore:
Using the normal 1.96 multiplier gives the illustrative 95% interval \(5.00\pm1.96(10.0)=[-14.6,24.6]\) assay units. This interval is very wide relative to the estimate. The data identify \(\theta_2\) in the structural sense, but the small change in setting \(q\) makes its contribution difficult to separate from observation noise. The interval crossing zero also means these data do not tightly establish the sign of \(\theta_2\) under the unconstrained model.
Why the Design Changes Practical Identifiability
The preceding example can be understood directly from the estimating equation. If the two settings are \(q=0\) and \(q=d\), and there are \(m\) independent observations at each setting with error standard deviation \(\sigma\), then the difference between the two sample means estimates \(d\theta_2\). Each mean has variance \(\sigma^2/m\), so their difference has variance \(2\sigma^2/m\). Dividing by \(d\) to estimate \(\theta_2\) gives:
This derivation requires \(d\ne0\), independent errors of equal known variance, and the stated linear response model. It shows two general design principles in this example: greater measurement noise increases uncertainty, while a larger separation between settings decreases it. Replication also decreases uncertainty, but it may do so less effectively than choosing settings that distinguish the parameters.
Worked Example: A More Informative Setting
Keep the same model, error standard deviation, and two replicates per setting, but use settings \(q=0\) and \(q=1\). Suppose the observations are 9.8 and 10.2 at \(q=0\), and 14.8 and 15.2 at \(q=1\). The two means are 10.0 and 15.0 assay units, giving \(\hat{\theta}_1=10.0\) and \(\hat{\theta}_2=(15.0-10.0)/1=5.00\) assay units.
With \(m=2\), \(\sigma=1\), and \(d=1\), the derived standard error is:
The corresponding 95% interval is \(5.00\pm1.96(1.00)=[3.04,6.96]\) assay units. The measurements have the same error level and replication as in the preceding example, but the larger separation of settings produces a much narrower interval for \(\theta_2\). In an actual QSP application, settings would have to remain scientifically and experimentally relevant; larger perturbations are not automatically feasible or appropriate.
Assessing Results and Recognizing Failure Modes
A fitted uncertainty estimate should be read together with the fit and the scientific question. A small residual error does not imply that every parameter is well constrained: different parameter values can give similar predictions at the observed settings. Conversely, a wide uncertainty interval does not by itself show that the model is wrong. It may show that the data do not distinguish among plausible parameter values.
Useful warning signs include very wide intervals relative to the parameter's relevant range, estimates that change markedly when the data are slightly perturbed, and uncertainty that is strongly affected by imposed parameter bounds. If an estimate lies at a bound, an interval calculated as if the estimate were well inside the allowed range may give a misleading summary. Strongly asymmetric uncertainty or a poor fit to the assumed error model also makes a simple symmetric interval less informative. These signs motivate examining the assumptions and the data rather than treating one uncertainty number as definitive.
The uncertainty calculation above used a known Gaussian error scale and an exactly linear model, so its intervals have a clear interpretation under those assumptions. In more complex models, uncertainty may be assessed using likelihood-based or simulation-based procedures, each with its own assumptions and limitations. A local approximation can be convenient, but it may not capture asymmetry, parameter boundaries, or distant regions of similarly good fit. No uncertainty method repairs structural nonidentifiability, misspecification, or a data design that contains little information about a parameter.
Practical identifiability is also purpose-dependent. An interval could be adequate for predicting a response at a tested dose but too broad for interpreting a biological mechanism or selecting a parameter-specific intervention. Whether the uncertainty is “small enough” must therefore be judged against the decision, parameter scale, and scientific range of interest—not by a universal numerical cutoff.
Structural analysis and practical uncertainty assessment answer complementary questions. Structural analysis checks whether ideal observations can distinguish parameters at all. Data-based uncertainty asks how much the available observations distinguish them in this application. If uncertainty is too large, possible responses include collecting more precise or replicated observations, choosing more informative sampling conditions, or reconsidering which parameters the data can support. These choices should be guided by the scientific objective and model assumptions.
Check Your Understanding
Answer each question using the definitions and assumptions in this tutorial. Short solutions follow.
- In the two-parameter example, why does a small nonzero value of \(q\) permit structural identification but still yield a wide interval for \(\theta_2\)?
- For the model in that example, if \(d\) doubles while \(m\) and \(\sigma\) stay fixed, by what factor does \(SE(\hat{\theta}_2)\) change?
- A model fit has small residuals, but one parameter has a very wide uncertainty interval. Does the small residual error establish that the parameter is practically identifiable? Explain.
- State the repeated-sampling interpretation of a 95% confidence interval for a fixed parameter.
Solutions. (1) Any \(d\ne0\) gives two distinct noise-free equations and hence a unique parameter pair, but \(\operatorname{Var}(\hat{\theta}_2)=2\sigma^2/(m d^2)\) becomes large when \(d\) is small. (2) Since \(SE(\hat{\theta}_2)=\sqrt{2}\sigma/(\sqrt{m}|d|)\), doubling \(d\) halves the standard error. (3) No. A good fit does not show that the observations distinguish nearby or compensating parameter values; the wide interval indicates weak practical constraint under the stated analysis. (4) Across repeated data sets generated under the assumptions, 95% of intervals constructed by that procedure would contain the fixed true parameter.