Constructing a Profile Likelihood for a Scalar Target

Fix the Target, Refit the Nuisance Parameters

A likelihood describes how well each possible parameter value accounts for the observed data. When the parameter of interest is only one component of a larger GLM parameter vector, the likelihood at a proposed target value still depends on the other components. The profile likelihood addresses this by fixing the target and maximizing over all remaining, nuisance parameters. The nuisance estimates can therefore change as the target changes.

This is different from simply substituting one fixed nuisance-parameter value into the likelihood. In the profile, the nuisance parameters are re-estimated separately for each target value. The construction gives a one-dimensional function even when the original model has several parameters.

For a parameter vector written as \(\theta=(\psi,\lambda)\), where \(\lambda\) denotes nuisance parameters, the definition is \(L_p(\psi)=\sup_{\lambda}L(\psi,\lambda)\), with the supremum restricted to values allowed by the model. A constrained maximizer may be denoted \(\hat{\lambda}(\psi)\); this notation emphasizes that its value depends on the fixed target. When the maximizing nuisance value exists, \(\ell_p(\psi)=\ell(\psi,\hat{\lambda}(\psi))\).

Two Properties of the Profile

The profile retains the likelihood scale and compares the best fit available at each target value. The following basic property explains how its maximum relates to the unrestricted likelihood maximum.

Proof. Every feasible \(\theta\) belongs to the set indexed by \(\psi=g(\theta)\). Thus \(L(\theta)\leq L_p(g(\theta))\), so \(\sup_\theta L(\theta)\leq\sup_\psi L_p(\psi)\). Conversely, for each \(\psi\), the set defining \(L_p(\psi)\) is a subset of the full parameter space, so \(L_p(\psi)\leq\sup_\theta L(\theta)\). Taking the supremum over \(\psi\) gives the reverse inequality. If \(\hat{\theta}\) is a global maximizer, then \(L_p(g(\hat{\theta}))\geq L(\hat{\theta})\), and it cannot exceed the global maximum. Therefore equality holds. \(\square\)

This argument uses suprema and does not require a maximizer to exist at every target value. In applications, however, it matters whether the nuisance-parameter maximum is actually attained. In the binomial regression below, the intercept score equation gives a useful existence and uniqueness condition.

Profiling the Intercept in Binomial Regression

Consider independent grouped binomial observations \(Y_i\sim\operatorname{Binomial}(m_i,p_i)\), with fixed covariates \(x_i\), positive group sizes \(m_i\), and logistic mean model \( \operatorname{logit}(p_i)=\alpha+\beta x_i \). Here \(\beta\) is the scalar target and \(\alpha\) is the nuisance intercept. For a fixed value \(b\) of the slope, the likelihood as a function of the intercept is

The binomial coefficients do not depend on either parameter, but they are part of the likelihood and should be retained when reporting its numerical value. The log-likelihood, apart from those constants, is \( \sum_i\{y_i(\alpha+b x_i)-m_i\log(1+\exp(\alpha+b x_i))\} \). Differentiating with respect to \(\alpha\) gives the intercept score equation

Proof. The derivative of the score with respect to \(\alpha\) is

for every finite \(\alpha\), because \(m_i>0\) and \(0<p_i(\alpha,b)<1\). Hence the score is strictly decreasing. As \(\alpha\to-\infty\), every \(p_i(\alpha,b)\to0\), so the score tends to \(S>0\). As \(\alpha\to+\infty\), every \(p_i(\alpha,b)\to1\), so the score tends to \(S-M<0\). The score is continuous, so it has a zero by the intermediate value theorem; strict decrease makes that zero unique. The log-likelihood increases before this zero and decreases after it, so the zero is the unique finite maximizer. \(\square\)

If \(S=0\) or \(S=M\), the score need not have a finite zero: the likelihood can instead approach its supremum as the intercept tends to \(-\infty\) or \(+\infty\). This boundary case is one reason to distinguish a supremum in the definition from an attained maximum.

Worked Binomial Calculations

With a binary covariate \(x=0,1\), write \(p_0\) and \(p_1\) for the probabilities in the two groups. At a fixed slope \(b\), the logistic model imposes \( \operatorname{logit}(p_1)-\operatorname{logit}(p_0)=b \). When the two groups have equal size, profiling the intercept also requires the fitted probabilities to sum to the observed total successes divided by that common size. These two equations determine the fitted probabilities at the fixed slope.

Reading and Using the Profile

For each proposed slope \(b\), the calculation has the same logic: impose that slope, find the intercept maximizing the likelihood under that constraint, and evaluate the likelihood at the resulting fit. The curve \(b\mapsto L_p(b)\), or its log version \(b\mapsto\ell_p(b)\), records the best attainable fit at every slope value. Its global maximum agrees with the unrestricted maximum when that maximum exists, as shown above.

In computation, the nuisance fit is often obtained by solving the intercept score equation numerically. For a simple two-group model, the equations may be solved directly as in the worked calculation. For larger designs, the same fixed-target principle applies, but the nuisance optimization may require a numerical GLM fit. A profile value is meaningful only if the optimization respects the target constraint and the model's feasible parameter space.

Common Mistakes

• Holding the nuisance estimate fixed. Evaluating \(L(\psi,\hat{\lambda})\) with one fixed \(\hat{\lambda}\) is not generally the profile. For each \(\psi\), maximize over nuisance values anew.
• Confusing a target value with a fitted estimate. In the slope example, \(b\) is fixed during the constrained fit; \(\hat{\alpha}(b)\) is then found from the intercept score equation.
• Dropping likelihood constants without saying so. Binomial coefficients do not affect the maximizing parameters, but omitting them changes the numerical likelihood. Log-likelihoods with the constants omitted are not the full log-likelihood values reported above.
• Assuming every conditional maximum is attained. With all failures or all successes, the intercept supremum may occur only as the intercept diverges. State whether the profile uses a maximum or a supremum.
• Using an unweighted average of group proportions. At zero slope, the common binomial probability is the pooled success fraction, weighted by group sizes.