Profile Likelihood Estimation of the Power Index

Why Estimate the Power by Profiling?

In a Tweedie model, the power index $p$ determines how the variance changes with the mean: $\operatorname{Var}(Y_i)=\phi\mu_i^p$. For the compound Poisson–Gamma case, $1<p<2$, the response can have both an atom at zero and a continuous distribution for positive values. The power therefore affects more than the spread of positive observations; it also affects the probability of observing zero.

Estimating $p$ requires comparing likelihoods across candidate powers. A fair comparison must allow the other model parameters to adjust at each candidate value. If the regression coefficients or dispersion are held at estimates obtained for a different power, the resulting comparison is not the profile likelihood. The central operation is repeated fitting: fix $p$, maximize over the nuisance parameters, record the maximized log-likelihood, and then maximize those recorded values over $p$.

Definition and Likelihood Contributions

Suppose $Y_1,\ldots,Y_n$ are independent responses from a Tweedie model with $1<p<2$. Let the mean for observation $i$ be $\mu_i(\beta)$, specified by a chosen mean model and link, and let the common dispersion be $\phi>0$. Write the nuisance parameter vector as $\psi=(\beta,\phi)$. For fixed $p$, the likelihood is the product of the individual probability masses or densities.

Definition: The profile log-likelihood for the power index is $$ \ell_{\mathrm{prof}}(p)=\sup_{\psi\in\Psi(p)}\ell(\psi,p), \qquad \ell(\psi,p)=\sum_{i=1}^n \log f(y_i;\mu_i(\beta),\phi,p), $$ where $\Psi(p)$ is the allowed nuisance-parameter space at that power. A profile-likelihood estimate is any value $\hat p$ maximizing $\ell_{\mathrm{prof}}(p)$ over the allowed power interval. When the supremum is attained, the corresponding nuisance estimates are $\hat\psi(\hat p)$.

For $1<p<2$, the zero probability has the closed form

P(Y_i=0)=\exp\!\left\{-\frac{\mu_i^{\,2-p}}{\phi(2-p)}\right\}.

Thus an observed zero contributes $-\mu_i^{\,2-p}/\{\phi(2-p)\}$ to the log-likelihood. An observed positive value contributes the log of the positive-outcome density $f(y_i;\mu_i,\phi,p)$. That density can be evaluated by the exact series or approximated by a saddlepoint density; either choice must be used consistently when constructing the profile. With an approximation, the result is an approximate profile likelihood, and its accuracy depends on the density approximation.

At every trial $p$, the fitting step maximizes the sum of these contributions over both $\beta$ and $\phi$. The dispersion is a nuisance parameter in this calculation, not a quantity that should be carried over unchanged from a fit at another power.

How the Profile Is Constructed

Choose a candidate power.
Select $p$ inside the admissible interval, here $1<p<2$.

Evaluate the likelihood.
For the observed data, calculate each zero-probability or positive-density contribution at the candidate $p$.

Refit nuisance parameters.
Maximize over the regression coefficients and dispersion, starting anew or using a suitable warm start from a nearby power.

Record and maximize.
Store the maximized log-likelihood, repeat over a grid of powers, and refine the search near the largest recorded value.

Worked Example: Locating a Profile Maximum

Suppose six nonnegative outcomes are $0,0,0.8,1.3,2.0,4.1$, and the mean model is intercept-only with $\log(\mu_i)=\beta_0$. An exact-density fitting routine is run at each listed power, re-estimating both $\beta_0$ and $\phi$. The following values are illustrative output from that procedure.

Candidate $p$	Fitted $\hat\beta_0(p)$	Fitted $\hat\phi(p)$	$\ell_{\mathrm{prof}}(p)$
1.20	0.94	0.81	-12.84
1.40	0.96	0.76	-12.31
1.60	0.99	0.71	-12.08
1.80	1.03	0.66	-12.25

Among these four grid points, $p=1.60$ has the largest profile log-likelihood, since $-12.08$ is greater than $-12.25$, $-12.31$, and $-12.84$. This identifies the best of the displayed candidates, not necessarily the continuous maximum. A finer search around 1.60 is needed before reporting a final estimate. Notice also that the fitted intercept and dispersion change with $p$: the tabulated profile compares each power after allowing those parameters to adapt.

Two Properties of Profiling

Profiling has a direct optimization interpretation, and its local curvature shows how uncertainty in the nuisance estimates affects information about $p$.

Result: If the joint log-likelihood has a maximum at $(\hat\psi,\hat p)$, then $\hat p$ maximizes the profile log-likelihood, and $\hat\psi$ maximizes the likelihood over nuisance parameters at $\hat p$.

Proof. By definition, $\ell_{\mathrm{prof}}(p)$ is the supremum of $\ell(\psi,p)$ over $\psi$. At the joint maximum, $\ell(\psi,p)\leq \ell(\hat\psi,\hat p)$ for every allowed pair $(\psi,p)$. In particular, taking the supremum over $\psi$ at any fixed $p$ gives $\ell_{\mathrm{prof}}(p)\leq\ell(\hat\psi,\hat p)$. At $p=\hat p$, the supremum is at least $\ell(\hat\psi,\hat p)$, and cannot be larger because $(\hat\psi,\hat p)$ is a joint maximum. Hence $\ell_{\mathrm{prof}}(\hat p)=\ell(\hat\psi,\hat p)$, which is at least $\ell_{\mathrm{prof}}(p)$ for every $p$. $\square$

For a local curvature formula, assume the log-likelihood is twice continuously differentiable near an interior solution, and that for each nearby $p$ the nuisance maximizer $\hat\psi(p)$ is an interior, differentiable solution of $\ell_\psi(\hat\psi(p),p)=0$. Assume also that the nuisance Hessian $\ell_{\psi\psi}$ is nonsingular there. Subscripts denote vectors or matrices of partial derivatives.

Result: Under these conditions, with $q(p)=\ell(\hat\psi(p),p)$, $$ q''(p)=\ell_{pp}-\ell_{p\psi}\ell_{\psi\psi}^{-1}\ell_{\psi p}, $$ where all derivatives on the right are evaluated at $(\hat\psi(p),p)$.

Derivation. Differentiate the nuisance first-order condition with respect to $p$. The chain rule gives

\ell_{\psi\psi}\hat\psi'(p)+\ell_{\psi p}=0, \qquad \hat\psi'(p)=-\ell_{\psi\psi}^{-1}\ell_{\psi p}.

Next, differentiate $q(p)=\ell(\hat\psi(p),p)$. Since $\ell_\psi=0$ at the nuisance maximum, $q'(p)=\ell_p$. Differentiating again and substituting the expression for $\hat\psi'(p)$ yields

q''(p)=\ell_{pp}+\ell_{p\psi}\hat\psi'(p) =\ell_{pp}-\ell_{p\psi}\ell_{\psi\psi}^{-1}\ell_{\psi p}.

This formula shows why the profile curvature is not generally the curvature obtained by varying $p$ while holding the nuisance parameters fixed. The nuisance parameters move as $p$ changes, and that movement changes the curvature. The result is local: if the maximizing nuisance solution jumps between competing modes, the profile need not be smooth at the jump.

Worked Example: Why Repeated Fitting Matters

At a candidate $p=1.5$, suppose optimization gives $\hat\beta_0=1.00$, $\hat\phi=0.70$, and log-likelihood $-8.20$. At $p=1.6$, refitting gives $\hat\beta_0=1.04$, $\hat\phi=0.66$, and log-likelihood $-8.10$. These are hypothetical likelihood-evaluation outputs for the same intercept-only model and data.

The profile comparison favors $p=1.6$, because $-8.10>-8.20$. If instead the parameters fitted at $p=1.5$ were held fixed at $p=1.6$, suppose the log-likelihood were $-8.35$. That fixed-parameter comparison would suggest $-8.35<-8.20$, apparently favoring $p=1.5$. It answers a different question: how the likelihood changes with $p$ at one fixed nuisance setting. Profiling answers how well each power can fit after its nuisance parameters have been optimized.

Uncertainty from the Profile Shape

A sharply peaked profile indicates that relatively small changes in $p$ produce substantial likelihood loss; a broad profile indicates weaker information about the power. Near an interior, well-behaved maximum, the observed profile curvature gives the local approximation

\widehat{\operatorname{SE}}(\hat p)\ \approx\ \left\{-\ell_{\mathrm{prof}}''(\hat p)\right\}^{-1/2},

provided the second derivative is negative and the profile is adequately approximated by a quadratic near its maximum. This standard-error approximation is local and can be unreliable for an asymmetric or flat profile.

A likelihood-ratio interval uses the profile without requiring symmetry. Under the usual regularity conditions for likelihood inference—an interior true power, an identifiable and differentiable model, a nonsingular information matrix, and a sufficiently large sample—the likelihood-ratio statistic for one scalar parameter is asymptotically chi-squared with one degree of freedom. An approximate $100(1-\alpha)\%$ profile interval is therefore the set of powers satisfying

2\{\ell_{\mathrm{prof}}(\hat p)-\ell_{\mathrm{prof}}(p)\} \leq \chi^2_{1,\,1-\alpha}.

This is an asymptotic interval, not an exact finite-sample guarantee. It also relies on a reliable likelihood calculation. If the estimate is near an allowed endpoint, if the profile is irregular, or if the density approximation is poor, the usual chi-squared calibration may be inaccurate and should be treated cautiously.

Worked Example: A Profile-Likelihood Interval

Suppose the maximized profile is $-18.40$ at $\hat p=1.55$, and near its maximum it is well approximated by

\ell_{\mathrm{prof}}(p)=-18.40-\frac{(p-1.55)^2}{2(0.12)^2}.

For an approximate 95% interval, use $\chi^2_{1,0.95}=3.84$. The likelihood-ratio condition becomes

2\{\ell_{\mathrm{prof}}(1.55)-\ell_{\mathrm{prof}}(p)\} =\frac{(p-1.55)^2}{0.12^2}\leq 3.84.

Taking square roots gives $|p-1.55|\leq 0.12\sqrt{3.84}$. Since $\sqrt{3.84}=1.96$, the half-width is $0.12(1.96)=0.2352$, and the interval is approximately $[1.3148,1.7852]$. This calculation uses the stated quadratic approximation; an actual analysis should find the interval endpoints from the evaluated profile and check that both endpoints lie inside the admissible power range.

Common Pitfalls and Practical Checks

Holding dispersion fixed across powers. This compares likelihoods at a constrained nuisance value, not the profile likelihood. Re-estimate dispersion at every candidate $p$.
Choosing the largest grid value and stopping. A grid maximum is only the best grid point. Refine the search around it and inspect the profile over the full allowed interval.
Treating an approximate density as exact. A saddlepoint-based profile is an approximation. Record the density method, and avoid reporting more precision than that calculation supports.
Reading uncertainty from the point estimate alone. The maximum does not describe how much the likelihood changes nearby. Inspect the profile and, when conditions are appropriate, use a likelihood-ratio interval.
Ignoring endpoint behavior. A profile maximum near a boundary can make standard interior likelihood approximations unreliable. Report the boundary constraint and interpret the usual interval calibration cautiously.

The power estimate is useful because it lets the data inform the variance–mean relationship rather than forcing a prespecified power. But it is meaningful only relative to the chosen mean model, the density evaluation method, and the nuisance refitting procedure. A profile graph is a practical diagnostic: it reveals the maximizing region, flatness, asymmetry, and possible irregularities that a single reported estimate conceals.

Check Your Understanding

Use the definitions and results above to answer these questions.

Why must the regression coefficients and dispersion be re-estimated at each candidate power to obtain a profile likelihood?
For an observed zero in the compound Poisson–Gamma case, what likelihood contribution is used?
What does the joint-maximum profiling result say about the relationship between a joint estimate and the profile maximum?
Why is the second derivative of the profile not generally equal to $\ell_{pp}$ with nuisance parameters held fixed?
State two conditions that make the usual profile likelihood-ratio interval questionable.