Likelihood Profiles For Weakly Identified Parameters

Why Profile a Parameter?

A joint model can contain parameters that are difficult to estimate separately. For example, an association parameter linking a subject’s latent longitudinal trajectory to the event hazard may be weakly informed by the observed data. A single estimate and standard error can conceal important features: the likelihood may change little over a broad range, uncertainty may be much larger on one side of the estimate, or separated parameter values may fit nearly as well.

A one-parameter profile likelihood makes these features visible. It asks how well the model can fit the data at each fixed value of a parameter of interest, after allowing the other parameters to take their best-fitting values. The profile is a diagnostic summary of the likelihood, not a cure for weak identification and not a probability distribution for the parameter.

Definition and Construction

Definition: Let $\ell(\psi,\boldsymbol{\eta})$ be the log likelihood, where $\psi$ is a scalar parameter of interest and $\boldsymbol{\eta}$ denotes all remaining model parameters. The profile log likelihood for $\psi$ is $$ \ell_p(\psi)=\max_{\boldsymbol{\eta}\in\mathcal{H}(\psi)}\ell(\psi,\boldsymbol{\eta}), $$ where $\mathcal{H}(\psi)$ is the set of nuisance-parameter values allowed by the model when $\psi$ is fixed. The normalized profile log likelihood is $\ell_p(\psi)-\ell_p(\hat{\psi})$, where $\hat{\psi}$ is a global maximizer of the profile over the allowed values of $\psi$.

For a joint model, $\psi$ might be an association parameter, while $\boldsymbol{\eta}$ includes longitudinal fixed effects, random-effects covariance parameters, baseline hazard parameters, and other survival-model parameters. The exact nuisance set depends on the parameter constraints and model specification. Every point on the profile must use the same observed data and the same likelihood definition.

A practical construction selects a grid of fixed values for $\psi$. At each grid point, the analyst maximizes the log likelihood over $\boldsymbol{\eta}$, using valid parameter constraints and numerical settings adequate for the model. The resulting maximum is one point on the profile. The maximization must be sufficiently reliable: an unsuccessful nuisance-parameter optimization can make the profile appear artificially low, jagged, or multimodal.

Likelihood-Ratio Scale and Its Interpretation

It is convenient to show the profile relative to its maximum using the likelihood-ratio statistic $$ D(\psi)=2\{\ell_p(\hat{\psi})-\ell_p(\psi)\}. $$ The best-fitting value has $D(\hat{\psi})=0$; larger values indicate a worse maximized fit when $\psi$ is fixed. A flat stretch, where $D(\psi)$ remains small across a wide range, indicates that the data distinguish those values poorly after the nuisance parameters have been re-optimized.

Under regular likelihood theory, if the model is correctly specified, the true parameter is an interior point, the maximum is well behaved, and identification is adequate, $D(\psi)$ is approximately distributed as chi-squared with one degree of freedom when $\psi$ is the true value. This motivates the approximate 95% profile-likelihood confidence set $$ \{\psi:D(\psi)\leq 3.84\}. $$ The approximation is not guaranteed in weakly identified or otherwise nonregular problems. A profile is still informative as a shape diagnostic, but a cutoff-derived interval may have poor coverage. In such cases, use the profile to describe the likelihood evidence and consider whether more appropriate calibration or sensitivity analysis is needed; do not present the nominal cutoff as automatically reliable.

The profile is not the same as a marginal likelihood or a Bayesian posterior. Profiling maximizes over nuisance parameters. Marginal likelihoods and posterior distributions instead integrate over nuisance parameters, with the latter also depending on the prior. A broad profile does not itself assign probabilities to values of $\psi$.

Worked Profiles

Worked Example: A Flat Profile for an Association Parameter

Suppose a hypothetical joint-model analysis estimates an association parameter $\alpha$, measured as the change in log event hazard per one-unit increase in the latent longitudinal marker. The profile log likelihood has its maximum at $\hat{\alpha}=0.40$, with $\ell_p(0.40)=-120.00$. At $\alpha=0.00$, the maximized profile log likelihood is $-120.60$. Calculate the likelihood-ratio difference and interpret the shape information.

The likelihood-ratio statistic at zero is $$ D(0)=2\{\ell_p(0.40)-\ell_p(0.00)\} =2\{-120.00-(-120.60)\}=1.20. $$

Thus, fixing the association parameter at zero gives a maximized fit only 1.20 likelihood-ratio units worse than the best fit. Because $1.20<3.84$, zero is inside the conventional approximate 95% profile-likelihood set, if the regularity approximation is suitable. This is not evidence that zero is true; it indicates that these data do not sharply distinguish zero from the fitted value on this comparison.

Now suppose the profile values remain close to the maximum throughout $\alpha=0.10$ to $0.75$. That broad, nearly flat region signals weak information about the precise association strength. The estimate $0.40$ alone would hide this uncertainty. Since the parameter is on the log-hazard scale, a difference of one unit in the latent marker corresponds to a hazard ratio of $\exp(\alpha)$; the profile concerns uncertainty in that log-hazard association, not a causal effect.

Worked Example: Asymmetric Profile-Likelihood Uncertainty

In a second hypothetical analysis, $\hat{\alpha}=0.50$ and $\ell_p(0.50)=-80.00$. A grid calculation gives $\ell_p(0.10)=-82.00$ and $\ell_p(0.90)=-81.50$. Find the likelihood-ratio statistics and use linear interpolation to estimate where the profile crosses the conventional 95% cutoff of 3.84.

At the lower grid point, $$ D(0.10)=2\{-80.00-(-82.00)\}=4.00. $$ At the upper grid point, $$ D(0.90)=2\{-80.00-(-81.50)\}=3.00. $$ The lower value is just beyond the 3.84 cutoff, while the upper value is below it. These grid points show that the profile does not fall at the same rate on both sides of its maximum.

For an illustrative lower crossing, suppose the neighboring grid point is $\alpha=0.20$, with $D(0.20)=2.00$. Linear interpolation between $(0.10,4.00)$ and $(0.20,2.00)$ gives $$ \alpha_{\mathrm{lower}}\approx 0.10+ \frac{4.00-3.84}{4.00-2.00}(0.20-0.10) =0.108. $$ For an upper crossing, suppose $D(1.00)=4.20$. Interpolating between $(0.90,3.00)$ and $(1.00,4.20)$ gives $$ \alpha_{\mathrm{upper}}\approx 0.90+ \frac{3.84-3.00}{4.20-3.00}(1.00-0.90) =0.970. $$

The resulting approximate set is $[0.108,0.970]$, conditional on the chi-squared approximation and the interpolation being adequate. Relative to the maximum at 0.50, the lower side extends about 0.392 units and the upper side about 0.470 units. The different profile curvature and crossing locations warn against assuming symmetric uncertainty around the estimate. These interpolated endpoints are illustrative grid approximations, not exact roots.

Worked Example: Two Competitive Parameter Values

Consider a third hypothetical profile for an association parameter $\gamma$. Its largest value is $\ell_p(0.30)=-50.00$, but a separated local peak occurs at $\gamma=1.10$, where $\ell_p(1.10)=-50.70$. Calculate the relative likelihood-ratio difference and explain what this profile says.

The difference at the second peak is $$ D(1.10)=2\{-50.00-(-50.70)\}=1.40. $$ Because $1.40$ is small relative to the conventional 3.84 cutoff, the separated value has a profile fit that is not dramatically worse than the global maximum. The appropriate description is that the observed likelihood supports competing parameter values to a similar degree in this comparison. It would be misleading to report only $0.30$ and a local, symmetric standard error as though the profile had one simple peak.

The global maximum remains at $0.30$ in the stated calculation. The second peak is not a second estimate with equal formal status, nor does the profile assign probability to either value. Check that the nuisance maximizations at both regions actually converged, then refine the grid around each peak. If the competing shape persists under reliable optimization, report it and avoid summarizing the uncertainty with a single symmetric interval alone.

Why the Profile Works and What It Does Not Establish

Profiling is useful because a fixed target value should be judged against its best possible fit under the remaining model parameters. If one instead holds nuisance parameters at their full-model estimates while varying $\psi$, the resulting curve generally answers a different question and may exaggerate how sharply the data identify $\psi$. The profile makes the nuisance parameters adapt to each proposed target value.

This is also why a profile is a diagnostic of joint-model estimation. A broad or irregular profile can reveal that the likelihood provides limited information about the target parameter even when an optimizer returns a numerical estimate. In a joint model, changes in the target may be partly offset by changes in other parameters. The profile records the best fit after those adjustments, but it does not by itself explain the underlying source of weak information or prove that the model is identifiable.

Choose the target.
State the scalar parameter and its scale and units, such as a log-hazard association per one-unit increase in a longitudinal marker.

Fix a grid value.
Set the target parameter to one allowed value and retain the specified joint-model likelihood and constraints.

Re-optimize nuisance parameters.
Maximize over all remaining parameters, checking convergence and retaining a reliable solution.

Compare and inspect.
Subtract the profile log likelihood from its global maximum, multiply by two, and inspect the full curve for flat areas, asymmetry, or separated high regions.

Diagnostics, Alternatives, and Limitations

A profile is only as trustworthy as the likelihood evaluations that compose it. Use a grid fine enough to resolve curvature and any suspected separate peaks. Repeat nuisance optimization from different starting values, especially where the profile unexpectedly jumps or declines. Verify that the reported nuisance solution satisfies parameter constraints and that the attained likelihood is reproducible. If an isolated grid point is much lower than both neighbors, determine whether the shape is genuine or the maximization failed. Extend the grid when the profile has not clearly declined on both sides; otherwise, a reported interval may be truncated by the grid rather than by the data.

The principal alternative summaries answer different questions. A Wald interval uses a local curvature or standard-error approximation around the estimate and is often compact, but can be unreliable when the profile is asymmetric, flat, or multimodal. A likelihood-ratio interval uses the profile shape and can reflect asymmetry, though its usual cutoff depends on regularity conditions. Bayesian posterior summaries incorporate a prior and integrate over uncertainty in other parameters; they can be valuable but should not be described as profile-likelihood results. None of these summaries makes weak information disappear.

Profiles also have limitations. They depend on the specified model, the data, and correct numerical maximization. A profile for one parameter can reveal poor precision without identifying the cause, and a one-dimensional curve cannot display every feature of a high-dimensional likelihood. When parameters lie on boundaries, the likelihood is nonregular, or weak identification undermines the usual approximation, the chi-squared cutoff may not provide the advertised coverage. Report the profile shape and the conditions behind any interval, rather than treating a numerical endpoint as self-validating.

Common Pitfall

Common mistake: “The likelihood-ratio value is below 3.84, so there is a 95% probability that the parameter equals this value.” This confuses a frequentist likelihood-ratio comparison with a probability statement about a fixed parameter. The correction is that, under stated regularity conditions, the cutoff defines an approximate confidence set with a long-run coverage interpretation. It does not give the probability that any particular parameter value is true. For a weakly identified parameter, even that approximate coverage may be unreliable.

Key takeaway: A profile likelihood is formed by fixing the parameter of interest and maximizing over the nuisance parameters. Its shape can expose flatness, asymmetry, and competing values that a point estimate can hide; its interpretation must account for numerical reliability and the limits of standard likelihood-ratio approximations.

Check Your Understanding

Answer each question, then compare with the solutions below.

In defining a profile log likelihood for $\psi$, what is fixed and what is maximized?
If the maximum profile log likelihood is $-40.0$ and the value at a candidate parameter is $-41.2$, calculate $D(\psi)$.
What does a broad, nearly flat profile indicate, and what does it not establish?
Why can a symmetric Wald interval be a poor summary when the profile is asymmetric or has separated peaks?
Give two checks that can help distinguish a genuine irregular profile shape from a numerical optimization problem.

Solutions. 1. Fix $\psi$ at the candidate value and maximize the log likelihood over all allowed nuisance parameters. 2. $D(\psi)=2\{-40.0-(-41.2)\}=2.4$. 3. It indicates that a range of values gives similar maximized fits, hence limited information about the precise value; it does not assign probabilities to parameter values or identify the cause of weak information. 4. A symmetric Wald interval relies on a local, approximately quadratic shape around one estimate, so it can misrepresent unequal uncertainty on either side or conceal a separated competitive region. 5. For example, repeat nuisance optimization from different starting values and verify convergence and reproducibility; also refine the grid around abrupt changes or peaks and check parameter constraints.