Why Profile the Likelihood?
A confidence interval for a genetic parameter should reflect how well the observed data support different values of that parameter. A Wald interval summarizes uncertainty locally, using an estimate and its standard error. This can work well when the likelihood is approximately symmetric and the estimate is comfortably inside the parameter space. But genetic parameters often have constraints, and likelihoods can be asymmetric or relatively flat on one side of an estimate. In those settings, a likelihood-ratio approach can describe the range of parameter values more directly.
The key challenge is that a model may contain several parameters while the interval concerns only one. The remaining parameters are nuisance parameters: they matter for the likelihood, but they are not the target of this particular interval. Profile likelihood handles them by finding, separately for each fixed target value, the nuisance-parameter values that make the data as likely as possible.
From a Likelihood-Ratio Test to an Interval
Let \((\hat{\theta},\hat{\eta})\) be a maximum-likelihood estimate (MLE) of all model parameters. At a proposed value \(\theta_0\), compare the best possible fit when \(\theta=\theta_0\) with the unrestricted best fit. The profile likelihood-ratio statistic is
A small value means that fixing \(\theta\) at \(\theta_0\) does not reduce the maximized log-likelihood much. A large value indicates that the best fit under that constraint is substantially worse than the unrestricted fit. For a scalar target parameter, under standard regularity conditions and when the null value is not on a problematic boundary, \(\Lambda(\theta_0)\) is approximately distributed as a chi-square random variable with one degree of freedom when \(\theta_0\) is the true value.
Inverting this likelihood-ratio test gives an approximate \(100(1-\alpha)\%\) confidence set: retain the values of \(\theta\) whose statistic does not exceed the \(1-\alpha\) quantile of the chi-square distribution with one degree of freedom. For an approximate 95% interval, that cutoff is about 3.841. The set is often an interval, but the definition is the set of retained values; if the likelihood is irregular or has multiple peaks, it need not be one connected interval.
The confidence level describes the approximate long-run coverage of the procedure under the model: across repeated data sets generated under the assumptions, intervals constructed this way contain the true parameter about \(100(1-\alpha)\%\) of the time. It does not mean that, after observing these data, there is a 95% probability that the fixed parameter lies in this particular interval.
Worked Example: Profiling a Nuisance Parameter in Genotype Categories
Suppose, hypothetically, 20 unrelated individuals are assigned to one of three mutually exclusive genotype categories, labelled A, B, and C. Use a categorical sampling model with probabilities \(\theta\), \((1-\theta)\eta\), and \((1-\theta)(1-\eta)\), respectively. The target is \(\theta\), the probability of category A; \(\eta\) is a nuisance parameter that splits the probability remaining after category A between B and C. Assume individuals are sampled independently and the category probabilities follow this model. The hypothetical counts are A = 12, B = 5, and C = 3.
Ignoring the multinomial coefficient, which is constant in the parameters, the likelihood is
At a fixed \(\theta\), the terms involving \(\eta\) are proportional to \(\eta^5(1-\eta)^3\). Maximizing this binomial-form likelihood gives \(\hat{\eta}(\theta)=5/(5+3)=0.625\), for every fixed \(\theta\) in the interior of its parameter space. Substituting that maximizing value profiles out the nuisance parameter:
The unrestricted MLEs are \(\hat{\theta}=12/20=0.600\) and \(\hat{\eta}=5/8=0.625\). The constant nuisance-parameter factor cancels in the likelihood ratio. Thus the 95% endpoint equation is
Solving this equation on either side of \(0.600\) gives endpoints approximately \(0.38295\) and \(0.79288\), or \((0.383,0.793)\) to three decimal places. Substituting either endpoint into the displayed equation gives a statistic of approximately 3.841. Under the stated independent categorical model and the usual interior regularity conditions, this is an approximate 95% profile-likelihood interval for the category-A probability.
Why the Nuisance Parameter Must Be Re-Optimized
Profiling is not the same as plugging the unrestricted nuisance estimate into the likelihood for every candidate target value. The unrestricted estimate is optimal at \(\hat{\theta}\); it need not be optimal when \(\theta\) is fixed elsewhere. The profile likelihood allows the nuisance parameters to adjust to each candidate target value, so the comparison asks how well that target value can fit the data under the best permitted nuisance settings.
The construction can be expressed as a sequence of constrained optimizations:
Maximize \(\ell(\theta,\eta)\) over all allowed values to obtain \((\hat{\theta},\hat{\eta})\) and the maximum log-likelihood.
For each proposed \(\theta\), maximize \(\ell(\theta,\eta)\) over the nuisance parameters subject to that fixed value.
Compute twice the difference between the unrestricted maximum and the constrained maximum.
Keep candidate values whose likelihood-ratio statistic is no larger than the chosen chi-square quantile, subject to the approximation being appropriate.
Worked Example: Comparing Profile-Likelihood and Wald Intervals
Use the same hypothetical genotype-category data and model. Because category A versus categories B and C together is a binomial count, the MLE of \(\theta\) is \(12/20=0.600\). The plug-in standard error for this binomial probability is
The approximate 95% Wald interval is the estimate plus or minus 1.96 standard errors:
This Wald calculation is reasonable as a comparison here because the estimate is interior and the expected counts under the fitted binomial model are 12 in category A and 8 in the combined other categories, rather than being near zero. It is not identical to the profile interval \((0.383,0.793)\). The Wald interval is symmetric around 0.600; the profile interval is asymmetric, with more distance from the estimate to its lower endpoint than to its upper endpoint. The two intervals are broadly similar in this example, but they use different approximations and need not agree in other data sets.
Coverage Rationale and Conditions
The coverage rationale for the profile interval comes from test inversion. For each candidate \(\theta_0\), the likelihood-ratio statistic provides an approximate test of \(H_0:\theta=\theta_0\). Values not rejected by that test are retained, producing the confidence set. When the likelihood-ratio approximation is accurate across the relevant parameter values, this inversion gives approximately the stated coverage.
The usual chi-square calibration relies on regularity conditions. In broad terms, the model must be identifiable, the likelihood must be sufficiently smooth near the true parameter, the true parameter must be an interior point of its allowed space, and the relevant information must be finite and nonzero. The nuisance parameters also need to be estimable under the constrained and unrestricted fits. When these conditions fail, the ordinary \(\chi^2_1\) cutoff may not give the intended coverage. In particular, boundary estimates, sparse data, weak identification, or multiple likelihood peaks call for caution; a profile curve alone does not guarantee nominal coverage.
Worked Example: A Boundary Estimate and Its Warning
Suppose, hypothetically, that 10 independent carriers of a variant are assessed for a binary phenotype under a Bernoulli model with penetrance \(\theta\). Assume the phenotype status is correctly recorded and the model treats all 10 observations as independent with the same penetrance. None of the 10 carriers has the phenotype. The likelihood is \(L(\theta)=(1-\theta)^{10}\), and its MLE is the boundary value \(\hat{\theta}=0\).
If one mechanically uses the usual 95% likelihood-ratio cutoff of 3.841, the statistic at a candidate \(\theta\) is
The upper endpoint solves \(-20\log(1-\theta)=3.841\), so
The mechanical likelihood-ratio set is therefore \([0,0.1747]\). However, this should not be presented as a generally calibrated 95% interval: the MLE is on the boundary, violating the ordinary interior-parameter condition behind the \(\chi^2_1\) approximation. Also, the usual Wald standard error \(\sqrt{\hat{\theta}(1-\hat{\theta})/n}\) is zero here, producing a degenerate interval at zero and failing to represent plausible positive penetrance values. The example shows why neither method should be applied without checking the model and the location of the estimate.
Interpreting and Reporting a Profile Interval
A profile-likelihood interval is an approximate likelihood-based confidence interval for a specified parameter under a specified model. It is not a measure of the probability that a genetic effect is real, and it does not establish a causal effect. Its interpretation concerns parameter values compatible with the data according to the likelihood and the chosen calibration.
A clear report should state the target parameter and its scale, the model used, the confidence level and likelihood-ratio cutoff, and how nuisance parameters were handled. For example: “Under the stated categorical model, the approximate 95% profile-likelihood interval for the probability of category A was 0.383 to 0.793; the nuisance split parameter was maximized separately at each fixed target value.” When the interval is close to a parameter boundary or the data are sparse, describe the standard chi-square calibration as potentially unreliable rather than implying exact coverage.
Common Mistakes
- Keeping nuisance estimates fixed. This computes a constrained likelihood with an arbitrary nuisance setting, not the profile likelihood. Re-maximize the nuisance parameters at every candidate target value.
- Using the wrong difference. The likelihood-ratio statistic is twice the unrestricted minus constrained log-likelihood. It is nonnegative because the unrestricted fit includes all constrained possibilities.
- Confusing the confidence level with a posterior probability. Approximate 95% coverage is a repeated-sampling property of the procedure, not a 95% probability statement about the fixed parameter after seeing the data.
- Assuming the interval must be symmetric. The profile interval follows the shape of the likelihood and may be asymmetric. Do not replace its endpoints with a symmetric estimate-plus-or-minus calculation.
- Ignoring boundaries and sparse data. The usual chi-square cutoff depends on regularity conditions. Check whether the estimate is interior and whether the model is identifiable before treating the nominal confidence level as dependable.
Check Your Understanding
Answer each question using the definitions and conditions in this tutorial.
- In a likelihood \(L(\theta,\eta)\), what operation defines the profile likelihood for \(\theta\), and why is \(\eta\) re-estimated for each fixed \(\theta\)?
- For a scalar target parameter, what is the usual approximate 95% likelihood-ratio cutoff, and what quantity is compared with it?
- Why can a profile-likelihood interval be asymmetric even when it is centered near the MLE?
- In the genotype-category example, what is the nuisance-parameter estimate at a fixed target value, and which two categories determine it?
- Why should the ordinary chi-square calibration be treated cautiously when the MLE is on the boundary of the parameter space?