Profiling Nuisance Parameters for Selection Effects

Why Profile a Selection Effect?

In a selection model, the outcome distribution and the response mechanism jointly determine the observed-data likelihood. A response-effect parameter describes how response depends on an outcome that may be missing. Because that outcome is not observed for everyone, the likelihood can permit different combinations of outcome and response parameters to fit the observed data similarly. The earlier material on likelihood ridges describes this instability; profiling provides a direct way to examine it while focusing on one response-effect value at a time.

The key operation is to re-optimize the nuisance parameters for every fixed value of the response effect. Holding them at their unrestricted estimates answers a different question: it evaluates the likelihood along one slice of parameter space, rather than finding the best fit compatible with the fixed response effect.

Definition: Write the parameter as $\theta=(\psi,\lambda)$, where $\psi$ is the response-effect parameter of interest and $\lambda$ collects all nuisance parameters, including outcome-model parameters and any other response-model parameters. For a fixed value $\psi_0$, let $\Lambda(\psi_0)$ be the set of nuisance values allowed by the full parameter space when $\psi=\psi_0$. The profile log likelihood is $$ \ell_p(\psi_0)=\sup_{\lambda\in\Lambda(\psi_0)}\ell(\psi_0,\lambda). $$ The supremum is the least upper bound of the constrained log likelihood. It need not be attained; when it is attained, it is a constrained maximum.

For example, a selection model may factor the complete-data model into an outcome distribution $f(y;\eta)$ and a response model $P(R\mid Y=y;\psi,\gamma)$. Here $\eta$ and $\gamma$ are nuisance parameters and $\psi$ is the response effect of interest. If $Y_{\mathrm{mis}}$ denotes unobserved outcomes, an observed-data contribution has the form

\int f(y_{\mathrm{obs}},y_{\mathrm{mis}};\eta) P(R\mid y_{\mathrm{obs}},y_{\mathrm{mis}};\psi,\gamma) \,dy_{\mathrm{mis}}.

For discrete outcomes the integral is replaced by a sum. Profiling is applied to the resulting observed-data likelihood, not to a likelihood that treats missing outcomes as observed. It requires a specified model, a well-defined observed-data likelihood, and a parameter space that states which nuisance values are permitted at each $\psi_0$. It does not require the nuisance optimum to be unique or the supremum to be attained.

What the Optimization Does

At each chosen $\psi_0$, regard $\psi_0$ as fixed and search over every allowed component of $\lambda$. In a smooth model, if the constrained optimum is interior and the nuisance parameterization is unconstrained locally, a necessary condition is that the nuisance score vanishes:

\left.\frac{\partial \ell(\psi_0,\lambda)} {\partial \lambda}\right|_{\lambda=\hat{\lambda}(\psi_0)}=0.

This condition is not enough by itself to establish a global maximum: the solution must also be compared with other stationary points and relevant boundaries. If the optimum lies on a boundary, the nuisance score need not equal zero in every direction. The profile value is still defined by the supremum over the permitted set.

Justification: Fix $\psi=\psi_0$. Any parameter pair $(\psi_0,\lambda)$ is a candidate fit under that constraint, so the best achievable log likelihood is the supremum over all such $\lambda$. Repeating this constrained optimization for different values of $\psi_0$ traces the best-fitting likelihood compatible with each response effect. Since the nuisance values are allowed to change, the profile does not penalize $\psi_0$ for being incompatible with nuisance estimates obtained under a different value of $\psi$.

Worked Calculations

The next two examples use a hypothetical log-likelihood surface with nuisance parameters $\alpha$ and $\beta$. It is a transparent numerical illustration of the profiling operation; it is not presented as an empirical clinical analysis. The constant $C$ does not affect which parameter values maximize the likelihood.

Worked Example: Re-optimizing Two Nuisance Parameters

Suppose the hypothetical fitted log likelihood is

\ell(\psi,\alpha,\beta) =C-\frac12\left[ \frac{(\alpha+2\psi-1)^2}{0.16} +\frac{(\beta-0.2\psi)^2}{0.09} +\frac{(\psi-0.2)^2}{0.36} \right].

Assume initially that $\psi,\alpha,\beta$ are unrestricted real parameters. For any fixed $\psi$, the first two squared terms are minimized by setting $\alpha=1-2\psi$ and $\beta=0.2\psi$. The third term cannot be changed by nuisance parameters. Therefore,

\ell_p(\psi)=C-\frac12\frac{(\psi-0.2)^2}{0.36}.

At $\psi=0.5$, the nuisance optima are $\hat{\alpha}(0.5)=0$ and $\hat{\beta}(0.5)=0.10$. Substitution gives

\ell_p(0.5) =C-\frac12\left[ 0+0+\frac{(0.5-0.2)^2}{0.36} \right] =C-\frac12(0.25) =C-0.125.

The unrestricted maximum occurs at $\hat{\psi}=0.2,\hat{\alpha}=0.6,\hat{\beta}=0.04$, where $\ell=C$. If the nuisance parameters were incorrectly held at those unrestricted values while setting $\psi=0.5$, the value would instead be

\ell(0.5,0.6,0.04) =C-\frac12\left[ \frac{(0.6+2(0.5)-1)^2}{0.16} +\frac{(0.04-0.2(0.5))^2}{0.09} +\frac{(0.5-0.2)^2}{0.36} \right] =C-\frac12(2.25+0.0400+0.250) =C-1.27.

The profile value is higher because $\alpha$ and $\beta$ have been re-optimized under the fixed effect. The difference is not a discrepancy: $C-1.27$ is the likelihood at one particular nuisance setting, whereas $C-0.125$ is the best likelihood available at $\psi=0.5$.

Worked Example: A Constrained Nuisance Parameter

Keep the same hypothetical likelihood, but now suppose the parameter space requires $0\leq\alpha\leq1$, while $\beta$ is unrestricted and $\psi\geq0$. The unconstrained nuisance optimum is $\alpha=1-2\psi$. It is permitted when $0\leq\psi\leq0.5$; for $\psi>0.5$, the nearest permitted value is $\alpha=0$. The optimum for $\beta$ remains $0.2\psi$.

At $\psi=0.8$, the constrained nuisance values are therefore $\alpha=0$ and $\beta=0.16$. The three contributions inside the brackets are

\frac{(0+2(0.8)-1)^2}{0.16}=2.25,\qquad \frac{(0.16-0.2(0.8))^2}{0.09}=0,\qquad \frac{(0.8-0.2)^2}{0.36}=1.

Thus $\ell_p(0.8)=C-\frac12(2.25+0+1)=C-1.625$. The nuisance optimum is on the boundary, so it would be incorrect to require the unconstrained nuisance score to vanish there. For $\psi>0.5$, the profile expression is

\ell_p(\psi) =C-\frac12\left[ \frac{(2\psi-1)^2}{0.16} +\frac{(\psi-0.2)^2}{0.36} \right].

This example shows why the parameter space is part of the profile definition. Ignoring a nuisance constraint can produce an unattainable profile value and an incorrect account of how well the model fits at the fixed response effect.

Worked Example: Coupled Nuisance Parameters

In a second hypothetical surface, let $u$ and $v$ be nuisance parameters and define

\ell(\psi,u,v) =C-\frac12\left[(u+2\psi-1)^2+(v-u)^2+(v-0.3)^2\right].

At fixed $\psi=0.4$, maximizing the log likelihood is equivalent to minimizing the sum of squares. Differentiating that sum with respect to $u$ and $v$ gives the interior equations

(u-0.2)+(u-v)=0,\qquad (v-u)+(v-0.3)=0,

or $2u-v=0.2$ and $2v-u=0.3$. Solving yields $u=7/30\approx0.2333$ and $v=4/15\approx0.2667$. The three residuals are $1/30,1/30,-1/30$, so the minimized sum of squares is $3(1/30)^2=1/300$. Consequently,

\ell_p(0.4)=C-\frac12\left(\frac{1}{300}\right) =C-\frac{1}{600} \approx C-0.001667.

The calculation illustrates that nuisance parameters may be coupled: optimizing each one independently while holding the other fixed would not generally give the joint constrained optimum. The nuisance score equations must be solved together, and the resulting candidate must be checked against other solutions or boundaries.

Computation, Diagnostics, and Interpretation

In a realistic selection model, evaluating the observed-data likelihood may itself require summation or numerical integration over missing outcomes. Profiling adds an optimization at every fixed $\psi$. A practical computation therefore separates the tasks: evaluate the observed-data log likelihood accurately, then optimize all nuisance parameters subject to the fixed response effect and the model’s constraints.

Specify the target and parameter space.
Identify $\psi$, list all nuisance parameters, and record constraints that apply at each fixed value.

Fix a response-effect value.
Set $\psi=\psi_0$ in the observed-data log likelihood. Do not substitute nuisance estimates from the unrestricted fit as if they were fixed by definition.

Optimize over the nuisance set.
Search jointly over outcome and other response parameters, including permitted boundaries, to obtain the constrained supremum or maximum.

Check and record the fit.
Compare starts and candidate solutions, verify constraints, and record the profile value and nuisance estimates. Repeat over the response-effect values of interest.

Useful diagnostics include convergence from multiple starting values, small nuisance scores at an interior solution, agreement between independent optimization runs, and a likelihood value that does not decrease when a better feasible nuisance setting is found. For a boundary solution, check the relevant one-sided feasible directions rather than demanding zero scores in directions that are not available. If numerical integration is part of the likelihood, verify that the likelihood calculations are stable enough that optimization is not responding mainly to integration error.

A flat or irregular profile can indicate weak information about the response effect, a ridge involving outcome and response parameters, or numerical difficulty. A flat profile is not by itself proof of a particular failure: inspect nuisance estimates across fixed $\psi$, constraints, integration accuracy, and optimizer behavior. Large changes in nuisance estimates with little change in profile log likelihood are especially consistent with weak separation of the effects. Profiling does not remove weak identification; it makes the best attainable fit across the nuisance values visible.

How Profiling Differs From Alternatives

Approach	What is done with nuisance parameters?	What it answers
Profile likelihood	Maximize them separately at each fixed $\psi$.	Best observed-data fit compatible with each response effect.
Fixed-nuisance likelihood slice	Hold them at selected values, often unrestricted estimates.	Fit along that particular slice; it may understate the best fit at a fixed $\psi$.
Joint maximum likelihood	Maximize over $\psi$ and all nuisance parameters together.	One overall maximizing fit, not the constrained fit for every $\psi$.
Bayesian marginalization	Integrate over nuisance parameters using a specified prior distribution.	A marginal distribution for $\psi$, which depends on the prior and is not a profile likelihood.

Profiling is useful when the question concerns how the likelihood supports different response-effect values after allowing other model parameters to adapt. It is not a cure for model misspecification, nonidentification, or poor numerical integration. Nor does a profile value alone provide a confidence interval or a calibrated test; those require additional inferential procedures and assumptions.

Common Mistake

Pitfall: “The unrestricted fit gives $\hat{\lambda}$, so the likelihood at $\psi_0$ is $\ell(\psi_0,\hat{\lambda})$.” This uses a fixed-nuisance slice and generally does not give the profile likelihood. The correction is to optimize over all $\lambda\in\Lambda(\psi_0)$ separately for each $\psi_0$, using a supremum unless attainment has been established. Another related error is to claim a nuisance maximum always exists; without suitable conditions, only the supremum is guaranteed by the definition.

Check Your Understanding

Answer each question, then compare with the solutions below.

In one sentence, distinguish a profile likelihood from a likelihood slice that holds nuisance parameters fixed.
In the first worked example, what are the nuisance optima at $\psi=0.5$, and what is the profile log likelihood relative to $C$?
Why is the constrained nuisance optimum in the second example at $\psi=0.8$ not required to have an unconstrained nuisance score of zero?
Suppose a numerical optimizer returns different profile values from different starts at the same fixed $\psi$. Name two checks that should be made before reporting the result.

Solutions. (1) A profile likelihood maximizes over nuisance parameters at each fixed response effect, whereas a slice evaluates the likelihood at a selected, held-fixed nuisance value. (2) They are $\alpha=0$ and $\beta=0.10$; the profile value is $C-0.125$. (3) The constraint $\alpha\geq0$ places the optimum at the boundary, so not every infinitesimal direction used in the unconstrained score condition is feasible. (4) Check whether multiple local solutions or boundaries are involved, verify constraints and nuisance scores where appropriate, and compare objective values across starts; also check likelihood integration accuracy if integration is used.

Approach	What is done with nuisance parameters?	What it answers
Profile likelihood	Maximize them separately at each fixed \(\psi\).	Best observed-data fit compatible with each response effect.
Fixed-nuisance likelihood slice	Hold them at selected values, often unrestricted estimates.	Fit along that particular slice; it may understate the best fit at a fixed \(\psi\).
Joint maximum likelihood	Maximize over \(\psi\) and all nuisance parameters together.	One overall maximizing fit, not the constrained fit for every \(\psi\).
Bayesian marginalization	Integrate over nuisance parameters using a specified prior distribution.	A marginal distribution for \(\psi\), which depends on the prior and is not a profile likelihood.