The Invariance Property of Maximum Likelihood Estimators

Why Transforming an MLE Works

A statistical model may be expressed using a parameter that is not the quantity of direct interest. For example, a model might use a success probability $p$, while the question asks about a function of it, such as $p(1-p)$. The likelihood principle gives a direct way to estimate that function: first find a maximum likelihood estimator of the original parameter, then apply the function to that estimator.

This rule is called the invariance property of maximum likelihood estimators. Its key justification is not that a transformed estimator automatically has desirable properties such as unbiasedness. Rather, it follows from how likelihood maximization works: no parameter value can have a likelihood greater than the global maximum, and the transformed estimate corresponds to at least one parameter value attaining that maximum.

Definition: Let $X$ have a model indexed by parameter $\theta$ in parameter space $\Theta$, and let $L(\theta;x)$ be the likelihood for observed data $x$. An MLE $\hat{\theta}$ is a value in $\Theta$ satisfying $L(\hat{\theta};x)\geq L(\theta;x)$ for every $\theta\in\Theta$. For a target parameter $\eta=g(\theta)$, the transformed estimate is $g(\hat{\theta})$.

The Invariance Property

For a transformed parameter, different original parameter values can lead to the same target value. To account for this, define the likelihood for a possible value $\eta$ by considering every original parameter value that maps to $\eta$. This set is called the fiber over $\eta$.

Definition: For $\eta$ in the image $\Psi=g(\Theta)$, define its fiber by $\Theta_\eta=\{\theta\in\Theta:g(\theta)=\eta\}$. The likelihood for the transformed parameter is $$ L_g(\eta;x)=\sup_{\theta\in\Theta_\eta}L(\theta;x). $$ If the supremum is attained, this is the largest likelihood among all original parameter values corresponding to $\eta$.

Theorem: If $\hat{\theta}$ is a global MLE in $\Theta$, then $g(\hat{\theta})$ is an MLE of $\eta=g(\theta)$, using the likelihood $L_g$ for the transformed parameter. The function $g$ need not be one-to-one.

The result applies to an entire parameter vector as well as to a single parameter. If $\theta$ includes nuisance parameters, the likelihood for $\eta$ maximizes over all combinations of the parameter and nuisance parameter that produce that value of $\eta$. A global MLE of the full vector maps to an MLE of the target parameter under this definition.

Why the Result Holds

The proof is a direct comparison of likelihood values. Let $\hat{\eta}=g(\hat{\theta})$. Since $\hat{\theta}$ is an MLE, its likelihood is at least as large as the likelihood at every point in $\Theta$. In particular, every point in the fiber for any $\eta$ has likelihood no greater than $L(\hat{\theta};x)$. On the other hand, the fiber for $\hat{\eta}$ contains $\hat{\theta}$ itself.

L_g(\eta;x) =\sup_{\theta\in\Theta_\eta}L(\theta;x) \leq L(\hat{\theta};x) \quad\text{for every }\eta\in\Psi,

L_g(\hat{\eta};x) =\sup_{\theta\in\Theta_{\hat{\eta}}}L(\theta;x) \geq L(\hat{\theta};x).

Together, these inequalities show that $L_g(\hat{\eta};x)$ is at least as large as $L_g(\eta;x)$ for every possible $\eta$. Thus $\hat{\eta}=g(\hat{\theta})$ maximizes the transformed likelihood. If the original likelihood has multiple global maximizers, each one maps to a transformed value that is an MLE; distinct transformed values can therefore both be tied as MLEs.

Find the original parameter space.
Record which values of $\theta$ are allowed, including any boundaries or constraints.

Find a global MLE.
Do not rely only on a score equation if it gives a stationary point; check that the candidate is a maximum over the full parameter space.

Apply the function.
Calculate $g(\hat{\theta})$. This is an MLE of the transformed parameter under the theorem’s conditions.

Check the target’s range and interpretation.
The estimate must be an attainable value of $g(\theta)$, and it estimates that transformed parameter—not a different quantity such as its expectation.

Worked Applications

The examples use different transformations to show how the result works for a one-to-one map and for a many-to-one map. In each case, the data model and its assumptions are stated before the likelihood is used.

Worked Example: Estimating a Mean Waiting Time From a Geometric Model

Assume four independent observations $X_1,\ldots,X_4$ are geometric waiting times, measured as positive integer numbers of trials up to and including the first success. Each trial has the same success probability $p$, with $0<p\leq1$. The observed waiting times are $2,1,3,2$. Estimate the mean waiting time $\mu=1/p$ by maximum likelihood.

Method. Find the MLE of $p$, then apply the function $g(p)=1/p$. For this geometric convention, $P(X_j=x)=p(1-p)^{x-1}$ for $x=1,2,\ldots$. The observations sum to $2+1+3+2=8$, so the likelihood is $$ L(p)=\prod_{j=1}^4p(1-p)^{X_j-1} =p^4(1-p)^{(2-1)+(1-1)+(3-1)+(2-1)} =p^4(1-p)^4. $$

For $0<p<1$, the log-likelihood is $\ell(p)=4\log p+4\log(1-p)$. Its derivative is $\ell'(p)=4/p-4/(1-p)$. Setting this equal to zero gives $4(1-p)=4p$, hence $\hat p=1/2$. Also, $\ell''(p)=-4/p^2-4/(1-p)^2<0$, so the log-likelihood is strictly concave; the likelihood is zero at the boundary $p=1$, and the candidate is the global maximum.

The mean waiting time is $\mu=1/p$, so invariance gives $\hat\mu=1/\hat p=1/(1/2)=2$ trials. As a check, writing the likelihood in terms of $\mu\geq1$ gives $L(\mu)=\mu^{-4}(1-1/\mu)^4$. Its log-likelihood derivative is $4(2-\mu)/[\mu(\mu-1)]$ for $\mu>1$, which is positive for $1<\mu<2$ and negative for $\mu>2$. Thus its maximum is indeed at $\mu=2$. The common mistake this avoids is estimating the mean by an unrelated transformation of the data rather than transforming the MLE of $p$.

Worked Example: Estimating a Two-Year Event Probability From a Poisson Rate

Suppose independent counts $Y_j$ in observation windows of lengths $t_j$ years follow $Y_j\sim\operatorname{Poisson}(\lambda t_j)$, with a common constant rate $\lambda>0$ per year. The observed counts are $1,2,0,3$, in windows of lengths $1,2,1,2$ years, respectively. Estimate the probability of at least one event in a two-year period.

Method. Estimate the rate, then transform it to the requested probability. The total count is $1+2+0+3=6$, and the total exposure is $1+2+1+2=6$ years. Ignoring factors that do not depend on $\lambda$, the likelihood is $L(\lambda)\propto\lambda^6e^{-6\lambda}$. The log-likelihood, up to a constant, is $\ell(\lambda)=6\log\lambda-6\lambda$. Therefore, $\ell'(\lambda)=6/\lambda-6=0$ gives $\hat\lambda=1$ event per year, and $\ell''(\lambda)=-6/\lambda^2<0$ confirms a maximum.

Under the stated Poisson model, the number of events in two years is Poisson with mean $2\lambda$. The probability of at least one event is $\eta=1-e^{-2\lambda}$. Applying the invariance property gives $$ \hat\eta=1-e^{-2\hat\lambda}=1-e^{-2}=0.8646647\ldots\approx0.8647. $$

The estimate is a probability, so it is in the valid range from zero to one. The map $1-e^{-2\lambda}$ is increasing for $\lambda>0$, which also means the transformed likelihood reaches its maximum at the image of $\hat\lambda$. The result is rounded to four decimal places. This is a probability estimate under the specified constant-rate Poisson model, not a claim that the observed sample itself had that proportion of two-year periods with an event.

Worked Example: A Many-to-One Transformation of a Binomial Probability

Assume $X\sim\operatorname{Binomial}(8,p)$, where the eight trials are independent and have a common success probability $0<p<1$. The observed count is $X=6$. Estimate the transformed parameter $q=p(1-p)$.

Method. First maximize the binomial likelihood. For the observed count, $L(p)=\binom{8}{6}p^6(1-p)^2$. The log-likelihood derivative is $6/p-2/(1-p)$. Equating this to zero gives $6(1-p)=2p$, so $\hat p=6/8=3/4$. The second derivative is $-6/p^2-2/(1-p)^2<0$, and the likelihood tends to zero at both boundaries, confirming the global maximum.

The target function $q=p(1-p)$ is not one-to-one: its range is $0<q\leq1/4$, and a value below $1/4$ generally corresponds to two possible values of $p$. In this sample, $$ \hat q=\hat p(1-\hat p) =\frac{3}{4}\left(1-\frac{3}{4}\right) =\frac{3}{16}=0.1875. $$

For this value, the equation $p(1-p)=3/16$ has roots $p=1/4$ and $p=3/4$. The likelihood at the MLE is $\binom{8}{6}(3/4)^6(1/4)^2=5103/16384\approx0.3117$; at the other point in the same fiber it is $\binom{8}{6}(1/4)^6(3/4)^2=63/16384\approx0.0038$. The fiber likelihood at $\hat q$ is therefore $5103/16384$, the global maximum. Since no other fiber can contain a likelihood value larger than the global maximum, $\hat q=0.1875$ is an MLE. The common mistake this avoids is assuming that a many-to-one transformation invalidates invariance; it does not, provided the transformed likelihood maximizes over the full fiber.

A Common Exam Trap: Adding a Jacobian

A transformation of a parameter is not the same operation as a transformation of a random variable’s probability density. A Jacobian is used when changing variables in a density, because the probability assigned to an interval must be adjusted for the change in scale. Invariance of an MLE instead follows by maximizing likelihood over parameter values. One does not multiply a likelihood by the derivative of a parameter transformation.

For the Poisson example, the transformation was $\eta=1-e^{-2\lambda}$, so its inverse is $\lambda=-\tfrac12\log(1-\eta)$. An incorrect calculation would multiply the likelihood, expressed in terms of $\eta$, by $\left|d\lambda/d\eta\right|=1/[2(1-\eta)]$. In terms of $\lambda$, this extra factor is $e^{2\lambda}/2$, so the incorrect objective is proportional to $\lambda^6e^{-6\lambda}e^{2\lambda}/2=\lambda^6e^{-4\lambda}/2$. Its maximum occurs at $\lambda=6/4=1.5$, which would give the incorrect estimate $1-e^{-3}\approx0.9502$. The correct likelihood has no Jacobian factor; its maximum is at $\hat\lambda=1$, giving $1-e^{-2}\approx0.8647$.

Scope and Practical Checks

The theorem depends on having a global MLE in the original parameter space. A stationary point that is not a maximum cannot be made into an MLE merely by transforming it. If the likelihood has a supremum that is not attained, there may be no MLE to transform. In constrained models, include boundary values when determining the original maximum, and make sure the transformed estimate belongs to the image of the allowed parameter space.

Invariance concerns the maximum of the likelihood. It does not state that $g(\hat{\theta})$ is unbiased, has minimum variance, or has any particular sampling distribution. Those are separate properties requiring separate analysis. The property is aligned with the SOA VEE Mathematical Statistics syllabus because it turns a likelihood maximization result into an estimate of a requested parameter function without requiring a fresh maximization in every case.

Key takeaway: If $\hat{\theta}$ globally maximizes the likelihood, then $g(\hat{\theta})$ maximizes the likelihood for $\eta=g(\theta)$, where that likelihood is maximized over all original parameter values mapping to each $\eta$. No one-to-one condition and no Jacobian factor are required.

Check Your Understanding

For each problem, identify the original MLE and apply the transformation; check that the reported estimate is in the target parameter’s allowable range.

Assume $X\sim\operatorname{Binomial}(10,p)$ and observe $X=7$. Estimate $p^2$ by maximum likelihood. (Answer: $\hat p=0.7$, so $\widehat{p^2}=0.49$.)
Four independent geometric waiting times, on the positive-integer convention with success probability $p$, are $1,4,2,3$. Estimate the mean waiting time $1/p$. (Answer: $\hat p=4/10=0.4$, so the mean estimate is $2.5$ trials.)
Independent Poisson counts are observed over windows of 1, 2, and 1 years, with counts $2,0,1$, respectively. Estimate the probability of at least one event in four months, assuming a constant rate per year. (Answer: total count $3$, total exposure $4$, $\hat\lambda=0.75$; the estimate is $1-e^{-0.75/3}=1-e^{-0.25}\approx0.2212$.)
Assume $X\sim\operatorname{Binomial}(5,p)$, observe $X=2$, and define $q=p(1-p)$. Give the MLE of $q$ and the two values of $p$ in its fiber. (Answer: $\hat p=0.4$, $\hat q=0.24$; the fiber has $p=0.4$ and $p=0.6$.)
Counts follow a Poisson model with constant rate $\lambda$ per year. If the total count is 5 over 2 years, estimate $\lambda/(1+\lambda)$. (Answer: $\hat\lambda=5/2=2.5$, so the estimate is $2.5/3.5=5/7\approx0.7143$.)

What You'll Learn