Profile Likelihood For Restricted Comparisons

Why Profile the Likelihood?

A likelihood-ratio comparison asks whether the data are substantially less likely under a restricted model than under a more general model. The restriction often concerns only part of the parameter vector. Other parameters, called nuisance parameters, still need to be estimated. For a fair comparison, they must be optimized separately under the restriction and under the unrestricted model.

For example, testing a specified mean in a normal model with unknown variance does not mean the variance is known. The variance should be chosen to give the greatest likelihood at the specified mean. That maximization is the essential step in profile likelihood. It is aligned with the SOA Exam ASTAM (Advanced Short-Term Actuarial Mathematics) syllabus.

Definition: Let the parameter be written as $(\psi,\eta)$, where $\psi$ is the parameter of interest and $\eta$ is a nuisance parameter. The profile likelihood for $\psi$ is $$ L_p(\psi)=\sup_{\eta} L(\psi,\eta), $$ where the supremum is over the allowed values of $\eta$ for that fixed $\psi$. The profile log-likelihood is $\ell_p(\psi)=\log L_p(\psi)$, when the likelihood is positive.

Suppose the null hypothesis restricts the parameter to a subset $\Theta_0$ of the unrestricted parameter space $\Theta$. The generalized likelihood-ratio statistic compares the best likelihood in each space. It is important that “best” means maximized over all free parameters in that space, including nuisance parameters.

Definition: The likelihood ratio and likelihood-ratio statistic for a nested comparison are $$ \Lambda=\frac{\sup_{\theta\in\Theta_0}L(\theta)} {\sup_{\theta\in\Theta}L(\theta)},\qquad T=-2\log\Lambda =2\left\{\sup_{\theta\in\Theta}\ell(\theta) -\sup_{\theta\in\Theta_0}\ell(\theta)\right\}. $$ When the unrestricted supremum is positive, $0\leq\Lambda\leq1$. If $\Lambda=0$, define $T=+\infty$. A strictly positive ratio requires additional conditions ensuring the null supremum is positive.

How the Restricted Maximum Is Found

If the null hypothesis fixes $\psi=\psi_0$, the restricted maximum is found by profiling at $\psi_0$: maximize $L(\psi_0,\eta)$ over the nuisance parameter $\eta$. The unrestricted maximum instead optimizes over both $\psi$ and $\eta$. More generally, a restriction may link parameters, such as $\lambda_1=\lambda_2$. Then optimize jointly over the parameter values satisfying that relationship.

Write the parameter spaces.
Identify the unrestricted space and the subset allowed by the null hypothesis.

Maximize under the null.
Enforce the restriction, then optimize every remaining free parameter, including nuisance parameters.

Maximize without the restriction.
Find the unrestricted maximum using the same observations and likelihood model.

Form and check the statistic.
Compute $T=2(\ell_{\mathrm{unrestricted}}-\ell_{\mathrm{null}})$. The unrestricted maximum cannot be smaller, so $T\geq0$.

Deriving the Profile-Likelihood Comparison

The formula follows directly from the definition of the likelihood ratio. Write $L_0$ for the likelihood maximized over the restricted space and $L_1$ for the likelihood maximized over the unrestricted space. Then $\Lambda=L_0/L_1$. Taking logarithms converts this ratio into a difference of maximized log-likelihoods:

-2\log\Lambda =-2\log\left(\frac{L_0}{L_1}\right) =2(\log L_1-\log L_0) =2(\ell_1-\ell_0).

This also explains why nuisance parameters are profiled under both hypotheses: $L_0$ and $L_1$ are each the largest likelihood attainable in their respective parameter spaces. Holding a nuisance parameter at its unrestricted estimate while imposing the null generally does not produce $L_0$, and therefore does not produce the likelihood-ratio statistic.

Worked Example: Testing a Normal Mean with Unknown Variance

Suppose four independent observations $X_1,\ldots,X_4$ are from a normal distribution $N(\mu,\sigma^2)$, with both $\mu$ and $\sigma^2>0$ unknown. The observed values are $1,2,3,4$. Test $H_0:\mu=2$ against the unrestricted alternative, profiling over the variance. Calculate $T$ to four decimal places.

Method and assumptions. The observations are independent and identically distributed normal variables, and the variance is unknown under both hypotheses. For a fixed mean $\mu$, let $\mathrm{RSS}(\mu)=\sum_{j=1}^4(x_j-\mu)^2$. The likelihood is proportional to $(\sigma^2)^{-4/2}\exp\{-\mathrm{RSS}(\mu)/(2\sigma^2)\}$. Maximizing over $\sigma^2$ gives $\widehat{\sigma}^2(\mu)=\mathrm{RSS}(\mu)/4$.

Unrestricted fit. The sample mean is $\bar{x}=(1+2+3+4)/4=2.5$. The unrestricted residual sum of squares is $\mathrm{RSS}(\bar{x})=(1-2.5)^2+(2-2.5)^2+(3-2.5)^2+(4-2.5)^2=2.25+0.25+0.25+2.25=5$. Thus $\widehat{\sigma}^2=5/4=1.25$.

Restricted fit. Under $H_0$, $\mu=2$, but the variance is still estimated. The restricted residual sum of squares is $\mathrm{RSS}(2)=(1-2)^2+(2-2)^2+(3-2)^2+(4-2)^2=1+0+1+4=6$. Thus the null-profile estimate is $\widehat{\sigma}_0^2=6/4=1.5$.

Statistic. Profiling the variance in the normal likelihood gives a maximized log-likelihood difference equal to $(4/2)\log(6/5)$. Therefore $T=4\log(6/5)=0.729286227\ldots$, which rounds to 0.7293 to four decimal places. The statistic is nonnegative, as required, because the unrestricted residual sum of squares is smaller.

Common mistake avoided. Treating the variance as known would use a different likelihood comparison. The calculation above re-estimates it at each mean, including at the null value.

Profiling Common Parameters Across Groups

A common restriction in actuarial data analysis says that multiple groups share a parameter. Under the null, the shared parameter is estimated from all groups together. Under the alternative, each group has its own parameter. The nuisance estimates are not carried over from one fit to the other; the likelihood is maximized anew in each parameter space.

Worked Example: Testing Equal Exponential Rates

Two independent groups have exponential loss observations with rates $\lambda_1$ and $\lambda_2$. Group 1 contains $n_1=3$ observations with total $S_1=6$; group 2 contains $n_2=3$ observations with total $S_2=12$. Test $H_0:\lambda_1=\lambda_2$ against separate positive rates. Calculate $T$ to four decimal places.

Model and likelihood. For independent exponential observations with density $f(x\mid\lambda)=\lambda e^{-\lambda x}$, $x>0,\lambda>0$, the likelihood for a group with sample size $n$ and total $S$ is proportional to $\lambda^n e^{-\lambda S}$. Its log-likelihood is $n\log\lambda-\lambda S$, maximized at $\widehat{\lambda}=n/S$.

Unrestricted fit. The separate rate estimates are $\widehat{\lambda}_1=3/6=0.5$ and $\widehat{\lambda}_2=3/12=0.25$. The maximized log-likelihood, omitting constants common to both fits, is $3\log(0.5)-0.5(6)+3\log(0.25)-0.25(12)=3\log(0.5)+3\log(0.25)-6$.

Restricted fit. Under a common rate $\lambda$, the combined sample size is $6$ and the combined total is $18$. Hence $\widehat{\lambda}_0=6/18=1/3$, and the restricted log-likelihood is $6\log(1/3)-(1/3)(18)=6\log(1/3)-6$.

Statistic. The linear terms $-6$ cancel. Thus $T=2\{3\log(0.5)+3\log(0.25)-6\log(1/3)\}=6\log(1.5)+6\log(0.75)=6\log(1.125)=0.7066982\ldots$. To four decimal places, $T=0.7067$.

Common mistake avoided. The null estimate is the pooled rate $6/18$, not the arithmetic average of the two separate rate estimates. It is obtained by maximizing the common-rate likelihood.

Worked Example: Testing a Shared Binomial Probability

Two independent portfolios each have 10 policies. In the first portfolio, 7 policies have a claim; in the second, 3 have a claim. Model each count as binomial with a common number of trials of 10. Test a common claim probability against separate probabilities. Calculate $T$ to four decimal places.

Model and likelihood. Under the unrestricted model, the probabilities are $p_1,p_2$; under the null, $p_1=p_2=p$. For a group with $x$ successes in $n$ trials, the log-likelihood, apart from the binomial coefficient, is $x\log p+(n-x)\log(1-p)$. Its maximum occurs at $x/n$.

Unrestricted fit. The estimates are $\widehat p_1=7/10=0.7$ and $\widehat p_2=3/10=0.3$. The maximized log-likelihood, omitting constants that cancel, is $7\log(0.7)+3\log(0.3)+3\log(0.3)+7\log(0.7)=14\log(0.7)+6\log(0.3)$.

Restricted fit. Under a shared probability, there are $7+3=10$ successes in $20$ trials, so $\widehat p_0=10/20=0.5$. The restricted log-likelihood is $10\log(0.5)+10\log(0.5)=20\log(0.5)$.

Statistic. Subtracting the restricted log-likelihood and multiplying by 2 gives $T=2\{14\log(0.7)+6\log(0.3)-20\log(0.5)\}=28\log(1.4)+12\log(0.6)$. Numerically this is $3.291315139\ldots$, or $T=3.2913$ to four decimal places.

Common mistake avoided. The expression already includes the factor of 2 in the coefficients 28 and 12. Multiplying the displayed expression by 2 again would double the statistic incorrectly.

Calibration and Interpretation

The statistic measures the loss in maximized log-likelihood caused by imposing the restriction. A value of zero means the restricted model attains the unrestricted maximum. Larger values indicate a greater likelihood penalty for the restriction, but the statistic alone does not give a p-value; calibration requires a reference distribution.

Under the regularity conditions for Wilks’ theorem— including an identifiable model, an interior true parameter, and a smooth restriction—the null distribution of $T$ is asymptotically chi-square with degrees of freedom equal to the number of independent restrictions. For example, requiring two otherwise free rates to be equal imposes one restriction. This approximation is not automatic: boundary parameters, small samples, or violated regularity conditions can change the appropriate calibration.

Comparison feature	Restricted fit	Unrestricted fit
Parameter space	Only values satisfying the null restriction	All allowed values in the larger model
Nuisance parameters	Maximized subject to the null	Maximized without the null restriction
Role in $T$	Supplies $\ell_0$, the restricted maximum	Supplies $\ell_1$, the unrestricted maximum

Common Errors to Check

Fixing nuisance parameters at unrestricted estimates. This generally does not maximize the likelihood under the null. Refit nuisance parameters with the restriction imposed.
Using different data likelihoods. Both maxima must be based on the same observed data and compatible likelihood definitions, including any terms that differ between the fits.
Reversing the ratio. The null maximum belongs in the numerator, so $\Lambda\leq1$ when the unrestricted supremum is positive. Consequently $T\geq0$.
Dropping or repeating the factor of 2. Either calculate $2(\ell_1-\ell_0)$, or simplify the expression carefully so the factor is included exactly once.
Assuming a chi-square result without checking conditions. The chi-square calibration is an asymptotic regular-model result, not a definition of the statistic.

Key takeaway: A profile-likelihood comparison is built from two optimized fits. Impose the null restriction and maximize over every remaining parameter; then maximize over the unrestricted model. Twice the difference in their maximized log-likelihoods is the likelihood-ratio statistic.

Check Your Understanding

For each problem, identify the restricted and unrestricted fits before calculating or interpreting the comparison.

Independent observations $1,2,3$ are modeled as $N(\mu,\sigma^2)$, with unknown variance. Test $H_0:\mu=1.5$. Use the profiled normal likelihood and give $T$ to four decimal places. (Answer: $\mathrm{RSS}_1=2,\ \mathrm{RSS}_0=2.75,\ T=3\log(1.375)\approx0.9554$.)
Two independent exponential groups have $n_1=n_2=2$ and observed totals $S_1=4,S_2=8$. Test equal rates against separate rates. Find the pooled null estimate and $T$ to four decimal places. (Answer: $\widehat{\lambda}_0=4/12=1/3,\ T=4\log(1.125)\approx0.4700$.)
Two binomial groups have $(n_1,x_1)=(8,6)$ and $(n_2,x_2)=(12,6)$. Test a shared probability against separate probabilities. Give the estimates and $T$ to four decimal places. (Answer: $\widehat p_0=12/20=0.6,\ \widehat p_1=0.75,\ \widehat p_2=0.5,\ T=2[6\log(1.25)+2\log(0.625)+6\log(5/6)+6\log(1.25)]\approx1.2876$.)
Suppose the restricted and unrestricted maximized log-likelihoods are $-42.7$ and $-40.2$. Calculate $T$ and state which fit has the larger likelihood. (Answer: $T=2[-40.2-(-42.7)]=5.0$; the unrestricted fit has the larger likelihood.)
A nested comparison has an unrestricted parameter vector with four free components. The null imposes two independent smooth equalities, and the true parameter is interior in a regular identifiable model. State the usual asymptotic null calibration and its degrees of freedom. (Answer: a chi-square distribution with $2$ degrees of freedom, subject to the stated regularity conditions.)

What You'll Learn