Cramér-Rao Bounds for Functions of Parameters

Why a Function of the Parameter Changes the Bound

An estimator does not always target the model parameter itself. A decision-maker may instead need a transformed quantity, such as the square of a probability, the logarithm of a rate, or an exponential function of a mean. The Cramér-Rao bound handles these targets by accounting for how quickly the target changes as the parameter changes.

The key adjustment is the derivative of the target function. A target that changes rapidly with the parameter generally requires more estimator variance than a target that changes slowly, for a fixed amount of information in the data. This tutorial is aligned with the SOA VEE Mathematical Statistics syllabus.

Definition: Let $X$ have a distribution indexed by a scalar parameter $\theta$, and let $\tau(\theta)$ be a differentiable scalar function. An estimator $T$ is unbiased for $\tau(\theta)$ if $E_\theta[T]=\tau(\theta)$ for every $\theta$ in the parameter space. Under the regularity conditions for the one-parameter Cramér-Rao bound, if the Fisher information in the data is $I(\theta)$, then $$ \operatorname{Var}_\theta(T)\geq \frac{[\tau'(\theta)]^2}{I(\theta)}. $$

Here $I(\theta)$ is the Fisher information in the full data, not necessarily in one observation. For $n$ independent observations with identical distributions and information $I_1(\theta)$ per observation, additivity gives $I(\theta)=nI_1(\theta)$. The result applies to an unbiased estimator of the stated function; it is not automatically a bound for a biased estimator.

Conditions and a Short Derivation

The bound requires a regular one-parameter model. In particular, the parameter lies in an open interval, the support of the observations does not depend on the parameter, and the likelihood can be differentiated under the integral or sum used to calculate expectations. The score has mean zero and finite, positive variance, and the estimator has finite variance. The target function must be differentiable at the parameter value under consideration. If these conditions fail, the usual bound need not apply.

Write the data likelihood as $L(\theta;X)$ and the score as $U(\theta)=\partial \log L(\theta;X)/\partial\theta$. Under the regularity conditions, $E_\theta[U(\theta)]=0$ and $E_\theta[U(\theta)^2]=I(\theta)$. Differentiating the unbiasedness identity gives the relationship needed for the bound:

E_\theta[T]=\tau(\theta) \quad\Longrightarrow\quad \frac{d}{d\theta}E_\theta[T] =E_\theta[TU(\theta)] =\tau'(\theta).

Because the score has mean zero, $E_\theta[TU(\theta)]=\operatorname{Cov}_\theta(T,U(\theta))$. Applying the Cauchy-Schwarz inequality to this covariance yields

[\tau'(\theta)]^2 =\operatorname{Cov}_\theta(T,U(\theta))^2 \leq \operatorname{Var}_\theta(T)\operatorname{Var}_\theta(U(\theta)) =\operatorname{Var}_\theta(T)I(\theta).

Since $I(\theta)>0$, division by $I(\theta)$ gives the stated bound. The derivative is squared: its sign does not affect the lower bound. The derivation also shows why the result depends on the target function, not just on the data model.

Worked Applications

Worked Example: A Squared Probability in a Bernoulli Model

Suppose $X_1,\ldots,X_4$ are independent Bernoulli observations with success probability $p$, where $0<p<1$. Find the Cramér-Rao lower bound at $p=0.5$ for the variance of any unbiased estimator of $\tau(p)=p^2$. Verify that an unbiased estimator exists and compare its variance at $p=0.5$ with the bound.

Method and assumptions. The Bernoulli model has parameter-independent support $\{0,1\}$, and its Fisher information for one observation is $1/[p(1-p)]$. Independence makes the information for four observations $I(p)=4/[p(1-p)]$. The target is differentiable, with $\tau'(p)=2p$.

Calculate the bound. At $p=0.5$, the information is $I(0.5)=4/(0.5\cdot0.5)=16$, and the target derivative is $\tau'(0.5)=2(0.5)=1$. Therefore,

\operatorname{Var}(T)\geq \frac{1^2}{16}=\frac{1}{16}=0.0625.

Check an unbiased estimator. Define $T=\binom{4}{2}^{-1}\sum_{i<j}X_iX_j$, the average of the six pairwise products. Independence gives $E[X_iX_j]=p^2$ for each pair, so $E[T]=p^2$. At $p=0.5$, the number of successes $K=\sum_iX_i$ has probabilities $P(K=k)=\binom{4}{k}/16$, and $T=\binom{K}{2}/6$. Thus $T$ takes values $0,0,1/6,1/2,1$ for $K=0,1,2,3,4$, respectively. Its expectation is $6(1/16)(1/6)+4(1/16)(1/2)+1(1/16)=1/4$. Also, $E[T^2]=6(1/16)(1/36)+4(1/16)(1/4)+1(1/16)=13/96$. Hence $\operatorname{Var}(T)=13/96-(1/4)^2=7/96$, which exceeds $1/16=6/96$, as required.

Common mistake avoided. Using the information for only one observation would incorrectly give a bound four times too large. The data contain four independent observations, so the information in the denominator is $16$, not $4$.

Worked Example: The Logarithm of an Exponential Rate

Five independent waiting times have exponential distributions with rate $\lambda>0$, so each has density $f(x;\lambda)=\lambda e^{-\lambda x}$ for $x>0$. Find the variance lower bound for any unbiased estimator of $\tau(\lambda)=\log\lambda$.

Method and assumptions. The support $x>0$ does not depend on $\lambda$, and the regularity conditions hold. For one observation, the log likelihood is $\log\lambda-\lambda x$. Its score is $1/\lambda-x$, and its Fisher information is $1/\lambda^2$. By independence, the information from five observations is $I(\lambda)=5/\lambda^2$.

The target derivative is $\tau'(\lambda)=1/\lambda$. Substituting both quantities into the function-of-parameter bound gives

\operatorname{Var}(T)\geq \frac{(1/\lambda)^2}{5/\lambda^2} =\frac{1/\lambda^2}{5/\lambda^2} =\frac{1}{5}.

The rate cancels, so the lower bound is $0.2$, in squared units of the logarithmic target, for every $\lambda>0$. This bound does not assert that an unbiased estimator exists or that it attains the bound; it states the minimum variance required of any such estimator if one exists.

Common mistake avoided. A tempting but incorrect answer is $\lambda^2/5$, which reverses the role of the derivative or the information. The derivative contributes $1/\lambda^2$ after squaring, while information is $5/\lambda^2$. Dividing these quantities gives $1/5$, not $\lambda^2/5$.

Worked Example: An Exponential Target of a Normal Mean

Let $X_1,\ldots,X_8$ be independent normal observations with distribution $N(\mu,4)$, where the variance $4$ is known and $\mu$ is unrestricted. Find the lower bound at $\mu=0$ for an unbiased estimator of $\tau(\mu)=e^\mu$.

Method and assumptions. The observations have parameter-independent real support. For one observation, the Fisher information for the mean of $N(\mu,\sigma^2)$ with known variance is $1/\sigma^2$, here $1/4$. Therefore the information in eight observations is $I(\mu)=8/4=2$.

The target derivative is $\tau'(\mu)=e^\mu$, so $\tau'(0)=1$. The lower bound at the specified parameter value is

\operatorname{Var}_0(T)\geq \frac{[\tau'(0)]^2}{I(0)} =\frac{1^2}{2} =0.5.

Thus any unbiased estimator of $e^\mu$, if one exists under this sampling model, must have variance at least $0.5$ at $\mu=0$. The variance $4$ is known and belongs in the information calculation; substituting a standard deviation of $2$ as though it were the variance would give the wrong information.

Common mistake avoided. Applying the bound for estimating $\mu$ directly would give $1/I=0.5$ here only because $\tau'(0)=1$. At a different value of $\mu$, the derivative factor $e^{2\mu}$ changes the bound. It cannot generally be omitted.

Interpreting and Using the Bound

The bound is a pointwise statement: at each parameter value, it limits the variance of unbiased estimators of the chosen target. It is a lower bound, not a guarantee that an estimator with that variance exists. If an estimator attains the bound under the model, it is efficient for that target in the Cramér-Rao sense. Equality requires the score and the centered estimator to be linearly related with probability one; the bound alone does not establish that condition.

A zero derivative at a particular parameter value makes this lower bound zero. That does not prove that an estimator has zero variance. It means the bound is uninformative at that point. A derivative close to zero can similarly produce a small bound, while a large derivative can increase it. Always evaluate the derivative at the parameter value of interest.

A reliable calculation separates three ingredients: the target $\tau(\theta)$, its derivative $\tau'(\theta)$, and the Fisher information in the full sample. A common general error is to use $\tau'(\theta)/I(\theta)$ instead of $[\tau'(\theta)]^2/I(\theta)$. Another is to substitute the information for one observation when the data consist of an independent sample. The following sequence helps prevent both errors.

Identify the target.
Write the function $\tau(\theta)$ that the estimator must estimate unbiasedly.

Differentiate the target.
Calculate $\tau'(\theta)$ and keep the derivative separate until it is squared in the bound.

Find full-sample information.
Use the model’s Fisher information and, for independent observations, add the individual information contributions.

Substitute and interpret.
Compute $[\tau'(\theta)]^2/I(\theta)$, then state that it is a lower bound for the variance of an unbiased estimator.

This method is especially useful when the requested estimand is a transformation rather than the model parameter itself. It keeps the target’s sensitivity and the sample’s information distinct, which is essential for a correct interpretation of the bound.

Check Your Understanding

For each problem, state the target derivative and use the Fisher information for the full sample before calculating the bound.

Three independent exponential waiting times have rate $\lambda>0$. Find the lower bound for an unbiased estimator of $\log\lambda$. (Answer: $1/3$)
Six independent observations have distribution $N(\mu,9)$, with known variance $9$. At $\mu=2$, find the lower bound for an unbiased estimator of $\mu^2$. (Answer: $24$)
Ten independent Bernoulli observations have success probability $p$. At $p=0.2$, find the lower bound for an unbiased estimator of $p(1-p)$. (Answer: $0.00576$)
Six independent Poisson observations have mean $\lambda>0$. At $\lambda=9$, find the lower bound for an unbiased estimator of $\sqrt{\lambda}$. (Answer: $1/24$)
Four independent observations have distribution $N(\mu,4)$, with known variance $4$. At $\mu=0$, find the lower bound for an unbiased estimator of $e^{2\mu}$. (Answer: $4$)

What You'll Learn