Why Reoptimize Nuisance Parameters?
A QSP model may contain many parameters, while an analysis asks about only one quantity, such as a pathway rate or a receptor turnover parameter. The remaining parameters are nuisance parameters for that question: they are not the target, but their values affect how well the model fits the observations. If a target value changes, the best-fitting nuisance values may change as well. A likelihood analysis that allows this compensation answers a different question from one that freezes the nuisance estimates at their unrestricted fitted values.
The distinction matters when parameters are correlated or when different mechanisms can compensate for one another. Holding nuisance parameters fixed can make the target appear more precisely estimated than it is when the full model is allowed to adjust. Profiling makes that adjustment explicit.
Let \((\hat{\psi},\hat{\eta})\) be an unrestricted maximum-likelihood estimate and let \(\hat{\ell}=\ell(\hat{\psi},\hat{\eta})\). The profile deviance is
It measures the loss in maximized log likelihood when \(\psi\) is fixed at the value under consideration and the nuisance parameters are still optimized. A profile-likelihood confidence set commonly retains target values for which \(D_p(\psi)\) is no larger than a chosen cutoff. Under suitable regularity conditions, a one-parameter likelihood-ratio statistic is compared with a chi-square distribution with one degree of freedom. The cutoff's calibration is a separate issue; the examples below use 3.84 to illustrate how nuisance reoptimization changes the likelihood set.
Why the Profile Deviance Is No Larger Than a Fixed-Slice Deviance
A fixed-nuisance calculation evaluates \(\ell(\psi,\hat{\eta})\), keeping the unrestricted nuisance estimate unchanged. Define its deviance as \(D_f(\psi)=2\{\hat{\ell}-\ell(\psi,\hat{\eta})\}\). This comparison is valid only when \((\psi,\hat{\eta})\) is feasible. A sufficient condition is a Cartesian-product parameter space \(\Theta=\Psi\times H\), in which every nuisance value in \(H\) is allowed at every target value in \(\Psi\). More generally, the argument applies only at target values where the fixed-nuisance point is feasible.
The justification is simply maximization over a larger set of candidates: the best feasible nuisance value cannot produce a lower log likelihood than a particular feasible candidate. If feasibility fails, that candidate is not in the maximization, so the comparison cannot be made this way. This is why parameter-space constraints must be checked rather than silently assumed away.
Worked Example: A Linear Model With a Compensating Nuisance Parameter
Suppose two hypothetical independent observations are \(y_1=10\) and \(y_2=14\). Their model means are \(\psi+\eta\) and \(\psi+2\eta\), respectively, and both observations have known standard deviation 1. Assume \(\psi\) and \(\eta\) may take any real values. Apart from an additive constant, the log likelihood is minus one half the residual sum of squares, so the deviance from the unrestricted maximum is the increase in that sum of squares.
The unrestricted fit has zero residuals: solving \(\psi+\eta=10\) and \(\psi+2\eta=14\) gives \((\hat{\psi},\hat{\eta})=(6,4)\). Write \(\delta=\psi-6\) and \(\gamma=\eta-4\). At these displaced values, the residuals are \(-(\delta+\gamma)\) and \(-(\delta+2\gamma)\), so $$ D(\psi,\eta)=(\delta+\gamma)^2+(\delta+2\gamma)^2. $$
For a fixed \(\psi\), differentiate with respect to \(\gamma\): $$ \frac{\partial D}{\partial\gamma}=2(\delta+\gamma)+4(\delta+2\gamma)=6\delta+10\gamma. $$ Setting this derivative to zero gives the minimizing nuisance displacement \(\hat{\gamma}(\psi)=-0.6\delta\). Substitution yields $$ D_p(\psi)=(0.4\delta)^2+(-0.2\delta)^2=0.20\delta^2. $$ At the fixed unrestricted nuisance value \(\eta=4\), however, \(\gamma=0\), so \(D_f(\psi)=\delta^2+(2\delta)^2=5\delta^2\). For the illustrative cutoff 3.84, the profile set has \(|\delta|\leq\sqrt{3.84/0.20}=4.382\), or approximately \([1.618,10.382]\). The fixed-nuisance set has \(|\delta|\leq\sqrt{3.84/5}=0.8764\), or approximately \([5.124,6.876]\).
The nuisance adjustment allows the model to fit better as \(\psi\) moves away from 6, so the profile deviance rises more slowly. This small example is for geometric comparison, not a claim that the cutoff is calibrated for this two-observation model.
Computing a Profile and Comparing Endpoints
In practice, calculate a profile by fixing \(\psi\) at a sequence of values, maximizing the likelihood over every nuisance parameter at each value, and recording \(D_p(\psi)\). The unrestricted estimate is a useful starting point, but each constrained optimization must be allowed to find its own nuisance solution. A profile confidence set is formed by retaining values satisfying the specified likelihood-ratio cutoff.
Find \((\hat{\psi},\hat{\eta})\) and \(\hat{\ell}\) under the stated parameter space.
Choose a value \(\psi\) and constrain the optimization to that value.
Maximize \(\ell(\psi,\eta)\) over all feasible \(\eta\), then record \(\ell_p(\psi)\).
Compute \(D_p(\psi)=2\{\hat{\ell}-\ell_p(\psi)\}\); repeat across target values and locate the cutoff crossings.
Worked Example: A Nonlinear Profile Versus a Fixed Nuisance Slice
Consider a hypothetical two-parameter likelihood whose deviance from its unrestricted maximum is $$ D(\psi,\eta)=4\delta^2+\{\eta-3-2\delta-0.4\delta^2\}^2,\qquad \delta=\psi-1. $$ The unrestricted minimum is zero at \((\hat{\psi},\hat{\eta})=(1,3)\). The parameter space is the Cartesian product of the real line for each parameter, so the fixed value \(\eta=3\) is feasible for every \(\psi\) considered.
At fixed \(\psi\), the squared term is minimized by \(\hat{\eta}(\psi)=3+2\delta+0.4\delta^2\). Hence the profile deviance is \(D_p(\psi)=4\delta^2\). With cutoff 3.84, its endpoints satisfy \(|\delta|=\sqrt{3.84/4}=0.9797959\ldots\), giving the profile interval \([0.0202,1.9798]\) to four decimal places.
For the fixed-nuisance slice \(\eta=3\), the deviance is $$ D_f(\psi)=4\delta^2+(2\delta+0.4\delta^2)^2. $$ Solving \(D_f(\psi)=3.84\) numerically gives a negative crossing near \(\delta=-0.74627\) and a positive crossing near \(\delta=0.64936\). For checks on the rounding, at \(\delta=-0.746\), \(D_f\approx2.226064+1.611361=3.837425\); at \(\delta=-0.747\), \(D_f\approx2.232036+1.614923=3.846959\). On the positive side, at \(0.649\), \(D_f\approx4(0.649)^2+(1.4664804)^2=3.835369\); at \(0.650\), \(D_f=1.690+1.469^2=3.847961\). Thus the fixed-slice interval is approximately \([0.2537,1.6494]\). It is narrower than the profile interval because it prevents the nuisance parameter from moving to its best-fitting value.
Optimization Checks, Alternatives, and Limitations
A profile is only as reliable as its constrained optimizations. For each target value, check that the reported nuisance solution is feasible and that the optimizer has converged to a maximum, not merely a nearby local optimum. Useful checks include starting from multiple nuisance values, comparing the optimized likelihood across starts, and refining the target grid near apparent cutoff crossings. A profile deviance should be nonnegative up to numerical tolerance and should equal zero at the unrestricted estimate when that estimate is included. If a constrained fit has a higher log likelihood than the purported unrestricted fit, the unrestricted optimization needs investigation.
A further diagnostic is the nuisance trajectory \(\hat{\eta}(\psi)\). Large or systematic changes can reveal compensation between mechanisms. A flat or slowly rising profile may indicate that the data constrain the target weakly after nuisance adjustment, even if the likelihood slice with nuisance parameters held fixed appears sharply peaked. Abrupt jumps can reflect multiple competing optima, numerical instability, or a genuinely irregular likelihood; they should not be smoothed away without checking the fits.
| Approach | What varies as \(\psi\) changes? | Interpretation and caution |
|---|---|---|
| Profile likelihood | Nuisance parameters are reoptimized at each fixed \(\psi\). | Measures fit loss after best feasible adjustment; requires reliable constrained optimization. |
| Fixed-nuisance slice | Nuisance parameters stay at \(\hat{\eta}\). | Can be useful as a diagnostic, but generally overstates target precision when the fixed point is feasible and nuisance compensation is possible. |
| Wald interval | Uses local curvature and an approximate standard error at the fitted estimate. | Can be convenient near a well-behaved, approximately quadratic likelihood; may poorly describe asymmetry or a long profile tail. |
These methods answer different questions. A Wald interval is a local approximation, while profiling follows the likelihood as the target moves and allows nuisance reoptimization. A fixed-nuisance slice is not an alternative profile calculation: it describes the likelihood along one particular slice through parameter space. Profile intervals also have limitations. Their interpretation depends on the likelihood model and the calibration of the cutoff; numerical maximization can fail; and a likelihood-based interval does not establish that the model's mechanisms are biologically correct.
Common Pitfall: Treating the Fitted Nuisance Values as Fixed Truth
The incorrect reasoning is: “The nuisance estimates are \(\hat{\eta}\), so to assess another target value I will leave them at \(\hat{\eta}\) and calculate the loss in fit.” That calculation answers only how well the model fits on the fixed-nuisance slice. It does not answer how well it fits when nuisance parameters are allowed to respond to the changed target value. The correction is to remaximize over feasible nuisance values for every target value, and then use \(\ell_p(\psi)\) to construct the profile deviance.
Also avoid asserting the profile-versus-fixed deviance inequality at target values where \((\psi,\hat{\eta})\) is infeasible. The inequality follows from candidate-set inclusion, not from a universal rule that ignores parameter-space constraints.
Worked Example: Checking a Profile Optimization at One Target Value
Suppose a hypothetical QSP model has three standardized residuals at fixed \((\psi,\eta_1,\eta_2)\), summarized by the deviance $$ D(\psi,\eta_1,\eta_2)=(\psi-1)^2+\{\eta_1-2(\psi-1)\}^2+\{\eta_2+(\psi-1)\}^2. $$ Assume all three parameters are unconstrained. The unrestricted minimum is zero at \((1,0,0)\). At the target value \(\psi=1.5\), \(\delta=\psi-1=0.5\). Minimizing the second and third squared terms gives \(\hat{\eta}_1(1.5)=1\) and \(\hat{\eta}_2(1.5)=-0.5\). Substitution gives \(D_p(1.5)=0.5^2=0.25\).
If the nuisance parameters were incorrectly held at their unrestricted values \((0,0)\), the deviance would instead be $$ D(1.5,0,0)=0.5^2+(-1)^2+(0.5)^2=1.50. $$ The profile result is no larger, as expected. In a numerical fit, this calculation is a useful check: the nuisance solution at the fixed target should not have a worse objective than the feasible fixed-nuisance candidate.
Check Your Understanding
Answer each question, then compare with the solutions below.
- In one sentence, distinguish a profile deviance from a fixed-nuisance deviance.
- Why is \(\ell_p(\psi)\geq\ell(\psi,\hat{\eta})\) not guaranteed when \((\psi,\hat{\eta})\) is infeasible?
- In the linear worked example, why is the profile interval wider than the fixed-nuisance interval?
- If a constrained optimization reports a higher log likelihood than the reported unrestricted maximum, what should be checked?
- Does a smaller profile deviance prove that the QSP mechanism is biologically correct?
Solutions. (1) The profile deviance allows nuisance parameters to be reoptimized at each target value; the fixed-nuisance deviance evaluates only the slice through \(\hat{\eta}\). (2) The fixed nuisance point is not a candidate in the constrained maximization, so candidate-set inclusion does not apply. (3) Reoptimizing \(\eta\) lets the model compensate as \(\psi\) changes, reducing the likelihood loss and retaining more target values under the same cutoff. (4) Check convergence, feasible parameter values, likelihood calculations, and whether the purported unrestricted fit found the global maximum. (5) No; it describes relative fit under the specified likelihood model and does not validate the biological mechanism.