Flat Likelihoods and Practical Nonidentifiability

Why a Converged Estimate Can Still Be Unstable

An estimation algorithm can converge to a parameter vector and still leave an important scientific question unanswered: how much do the data constrain each parameter? If a wide range of values gives nearly the same likelihood, the optimizer may report one precise-looking estimate even though nearby alternatives fit almost as well. In pharmacometric models, this can occur when measurements are sparse, noisy, or uninformative over the range needed to distinguish parameter values.

A parameter profile makes this weakness visible. It varies one parameter across a chosen range and, at each value, reestimates the remaining parameters. A shallow profile means the data provide little information about that parameter after allowing the other parameters to adjust. This is evidence of practical nonidentifiability: estimates may be highly sensitive to small changes in data, model specification, or numerical details, even when the likelihood calculation has converged.

Definition: Let \(\theta\) be the parameter of interest and \(\boldsymbol{\eta}\) denote all other parameters. The profile objective at \(\theta\) is the minimum objective-function value obtained while holding \(\theta\) fixed and reoptimizing \(\boldsymbol{\eta}\). If \(OFV_{\min}\) is the minimum over all parameters, the profile increase is \(\Delta OFV(\theta)=OFV_{\mathrm{profile}}(\theta)-OFV_{\min}\). For an objective equal to \(-2\) times the log likelihood, \(\Delta OFV\) is the likelihood-ratio statistic comparing the constrained and unconstrained fits.

A flat profile has small \(\Delta OFV\) over a broad range of \(\theta\). The range is meaningful only relative to a stated objective scale and likelihood definition. If the objective is \(-2\) log likelihood, constants that do not depend on parameters cancel in the difference. A profile is not a plot of the optimizer's path, and the nuisance parameters must be reoptimized at each fixed value; otherwise it is a conditional slice, not a profile.

Deriving and Reading a Profile

For independent observations \(X_i\sim N(\theta,\sigma^2)\), suppose \(\sigma\) is known. Ignoring terms that do not depend on \(\theta\), the objective is the sum of squared residuals divided by the variance. Its minimum occurs at \(\bar{x}\). The identity below shows exactly how the objective increases away from that estimate.

\sum_{i=1}^{n}(x_i-\theta)^2 = \sum_{i=1}^{n}(x_i-\bar{x})^2+n(\theta-\bar{x})^2

To verify the identity, write \(x_i-\theta=(x_i-\bar{x})+(\bar{x}-\theta)\), square, and sum. The cross-product is zero because \(\sum_i(x_i-\bar{x})=0\). Thus, after dividing by \(\sigma^2\), the profile increase is \(n(\theta-\bar{x})^2/\sigma^2\). The curvature is proportional to \(n/\sigma^2\): fewer observations or greater measurement variability produce a shallower profile.

\Delta OFV(\theta)=\frac{n(\theta-\bar{x})^2}{\sigma^2}

For a single parameter of interest, a common large-sample likelihood-ratio reference uses the 95th percentile of a chi-square distribution with one degree of freedom, approximately 3.84. Under regularity conditions, values with \(\Delta OFV\leq 3.84\) form an approximate 95% profile-likelihood confidence set. The conditions include a correctly specified likelihood, an interior parameter value, and adequate asymptotic approximation. This is not a universal cutoff for every model, especially at boundaries or when the likelihood approximation is poor.

Worked Example: A Broad Profile for a Mean Concentration

Suppose three hypothetical independent concentration measurements, in mg/L, are 9.8, 10.4, and 9.9. For this illustration, assume they follow a normal model with unknown mean \(\theta\), known standard deviation \(\sigma=2.0\) mg/L, and no additional parameters. The observations are assumed independent and the normal model is assumed appropriate; with only three observations, the example is primarily an illustration of the profile calculation.

The sample mean is \(\bar{x}=30.1/3=10.0333\) mg/L. The objective, up to a parameter-independent constant, is the residual sum of squares divided by \(2^2=4\). The derivation above gives the profile increase while retaining the unrounded mean \(30.1/3\):

\Delta OFV(\theta)=\frac{3(\theta-30.1/3)^2}{4}

At \(\theta=9.0\) mg/L, \(\Delta OFV=3(9.0-30.1/3)^2/4=0.8008\). At \(\theta=10.0\) mg/L, it is \(0.0008333\); at \(\theta=11.0\) mg/L, it is \(0.7008\); and at \(\theta=12.0\) mg/L, it is \(2.901\). These values are all below 3.84. The approximate likelihood-ratio interval solves \(3(\theta-30.1/3)^2/4\leq3.84\), giving \(|\theta-10.0333|\leq2.263\), or approximately 7.771 to 12.296 mg/L. Under the stated model and approximation, this broad interval indicates that these few noisy observations do not locate the mean precisely.

Limited Information After Reoptimizing Other Parameters

In a pharmacometric model, a parameter may be weakly constrained because other parameters can adjust to preserve much of the fit. A profile accounts for this adjustment. The following constructed objective illustrates the distinction between a profile and a slice that fixes a nuisance parameter. It is a mathematical illustration, not a real study or a general pharmacometric model.

Worked Example: Reoptimizing a Nuisance Parameter Reveals a Flat Profile

Let \(\theta\) be a parameter of interest and \(\eta\) a nuisance parameter. Suppose a hypothetical two-parameter objective is

Q(\theta,\eta)=\frac{(\theta+\eta-10)^2}{0.04}+\frac{(\theta-5)^2}{25}

The first term strongly constrains the combination \(\theta+\eta\); the second supplies only weak information about \(\theta\). The global minimum is \(Q_{\min}=0\) at \((\theta,\eta)=(5,5)\). When \(\theta\) is fixed, the first term is minimized by \(\hat{\eta}(\theta)=10-\theta\), so the profile is \(Q_{\mathrm{profile}}(\theta)=(\theta-5)^2/25\). At \(\theta=7\), the nuisance estimate is \(\eta=3\), and the profile increase is \(Q_{\mathrm{profile}}(7)=4/25=0.16\). A change of 2 units in \(\theta\) therefore costs little objective value in this example.

If instead \(\eta\) is incorrectly held at 5, then \(Q(7,5)=(2)^2/0.04+(2)^2/25=100.16\). That very large increase describes a conditional slice, not the profile: it prevents the nuisance parameter from adapting. The example shows why a fixed-nuisance scan can falsely suggest that the parameter of interest is sharply determined.

A profile table or plot should record the fixed value, the reoptimized objective, and whether the nuisance-parameter optimization converged. When the likelihood is regular and the parameter is interior, compare \(\Delta OFV\) with an appropriate likelihood-ratio threshold. Also inspect the width and shape of the profile itself: a long, shallow region is a practical warning even if a point estimate and standard error are available.

Worked Profile from Sparse Concentration-Time Data

A second calculation illustrates how a profile can quantify weak information from sparse measurements. Suppose hypothetical log concentrations \(y_i=\log(C_i)\), at times \(t_i\) in hours, follow the model \(y_i=a-kt_i+\epsilon_i\), where the errors are independent normal with known standard deviation 0.5 log units. Here \(a\) is an intercept and \(k\) is an elimination-rate parameter in h\(^{-1}\). This simple model assumes a straight line on the log-concentration scale and constant error variance.

Worked Example: Profiling an Elimination Rate

Take the hypothetical data \((t_i,y_i)=(0,4.0),(1,3.1),(2,2.2)\). They lie exactly on the line with \(\hat{a}=4.0\) and \(\hat{k}=0.9\) h\(^{-1}\). With \(k\) fixed, refit \(a\) as the mean of \(y_i+kt_i\). The profile objective, apart from a constant, is the minimized sum of squared log-scale residuals divided by \(0.5^2=0.25\).

The mean time is 1 hour and the mean log concentration is 3.1. After refitting the intercept, the residuals at a fixed \(k\) are \((k-0.9)(t_i-1)\), so their squared sum is \(2(k-0.9)^2\). Therefore,

\Delta OFV(k)=\frac{2(k-0.9)^2}{0.25}=8(k-0.9)^2

At \(k=0.4\) h\(^{-1}\), the increase is \(8(0.4-0.9)^2=2.00\); at \(k=1.4\) h\(^{-1}\), it is also 2.00. The approximate 95% profile set satisfies \(8(k-0.9)^2\leq3.84\), so \(|k-0.9|\leq\sqrt{0.48}=0.6928\) h\(^{-1}\), or approximately 0.2072 to 1.593 h\(^{-1}\). In this deliberately small, noisy data set, a wide range of elimination rates remains compatible with similar likelihood. The interval is an asymptotic illustration, not a claim that three observations guarantee a reliable normal approximation.

Diagnostics, Alternatives, and Limitations

A useful profile is smooth enough to interpret, has a clearly identified minimum, and uses a successful nuisance-parameter optimization at every grid point. Diagnostics include repeating fits from more than one starting value, checking that the reported profile objective is the best converged value at each fixed parameter, and refining the grid where the profile bends rapidly. A jagged or discontinuous trace can indicate numerical integration error, optimizer failures, or inconsistent convergence tolerances rather than genuine information in the data. Compare objective changes with the numerical precision and estimation behavior already assessed in the model workflow.

A flat profile should not be diagnosed from a single standard error alone. Hessian-based standard errors summarize local curvature near the optimum and can be unreliable when the likelihood is highly asymmetric or flat over a broad region. The profile directly explores departures from the optimum while allowing nuisance parameters to adapt. Conversely, a profile can be computationally expensive, and coarse grids may miss narrow features or misleading local minima. Profile endpoints based on a chi-square cutoff also rely on regularity assumptions; interpret them cautiously when those assumptions are doubtful.

The distinction from a numerical failure matters. If different starts produce substantially different estimates but nearly identical objective values, the data may support a broad region of solutions. If the objective values also vary unpredictably, or the profile is irregular, numerical instability is a competing explanation that must be investigated. A small gradient at one solution establishes local stationarity, not that the data identify every parameter well. Parameter-correlation diagnostics and Hessian curvature provide complementary clues, but neither substitutes for checking the profile.

Choose a parameter and scale.
Use a scientifically interpretable range; for positive parameters, a logarithmic grid may show relative changes more clearly than an equally spaced raw-scale grid.

Fix one value at a time.
At every grid point, hold the selected parameter fixed and reoptimize all remaining parameters using suitable starting values.

Subtract the global minimum.
Calculate \(\Delta OFV=OFV_{\mathrm{profile}}-OFV_{\min}\) using objective values from the same likelihood definition and settings.

Check reliability before interpreting.
Inspect convergence, repeat questionable fits, refine the grid, and compare the profile width with relevant likelihood-ratio thresholds only when their assumptions are reasonable.

The main alternative is a local curvature summary, such as a Hessian-based standard error. It is faster and useful when the likelihood is smooth and approximately quadratic near a well-behaved interior optimum. A profile is more informative about asymmetry and broader flat regions, but requires repeated optimization and remains dependent on the fitted model and data. Neither method can create information absent from the measurements. A broad profile is therefore not proof that a particular estimate is wrong; it is evidence that the available data do not sharply distinguish the alternatives examined.

Common Pitfalls

Holding nuisance parameters fixed. A steep conditional slice can be mistaken for a well-informed parameter. Refit the other parameters at every profile point.
Calling every flat trace a data problem. Failed optimization or inaccurate likelihood calculations can also produce irregular profiles. Verify the numerical fit before interpreting the shape.
Treating one estimate as uniquely supported. A converged optimizer returns a solution, but a broad profile may show that many nearby parameter values fit nearly as well.
Applying a likelihood-ratio cutoff mechanically. The familiar one-parameter chi-square threshold is approximate and depends on regularity conditions; report the assumptions and use caution when they are doubtful.

Key takeaway: Practical nonidentifiability is indicated when a parameter can move across a scientifically meaningful range while the profiled objective changes little. Reoptimizing nuisance parameters, checking numerical reliability, and interpreting objective differences in context are essential before drawing conclusions about parameter precision.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions below.

In a profile, why must nuisance parameters be reoptimized at each fixed value of the parameter of interest?
For independent normal observations with known standard deviation, what happens to profile curvature if the sample size is halved while the standard deviation stays the same?
In the sparse concentration example, calculate \(\Delta OFV\) at \(k=1.2\) h\(^{-1}\).
A profile has a broad, shallow region but different starting values yield erratic objective values. What should be checked before concluding that the parameter is weakly identified?

Solutions. (1) Reoptimization lets the other parameters take their best values at each fixed point, so the profile measures the best likelihood available there rather than an artificially restricted slice. (2) Curvature is proportional to \(n/\sigma^2\), so halving \(n\) halves the curvature and makes the profile flatter. (3) Using \(\Delta OFV(k)=8(k-0.9)^2\), the increase is \(8(0.3)^2=0.7200\). (4) Check optimizer convergence and starting-value sensitivity at each grid point, refine the profile grid, and verify consistent likelihood evaluation before attributing the apparent flatness to limited information.