Regularity Conditions for the Cramér-Rao Bound

Why Regularity Conditions Matter

The Cramér–Rao argument uses derivatives of probabilities and expectations. Those derivatives cannot always be moved inside an integral or an infinite sum. In particular, a density may have a support that changes with the parameter, creating a boundary contribution that a pointwise derivative of the density does not capture. The regularity conditions are safeguards against these failures.

It is useful to separate two issues. Conditions on the probability model can justify the score-mean identity. The full argument for an estimator also needs conditions allowing its expectation to be differentiated, as well as finite variances. Satisfying the first set does not automatically satisfy the second.

Definition: In a one-parameter model with parameter $\theta$, write the probability mass function or density as $f_\theta(x)$. Where $f_\theta(x)>0$ and the derivative exists, the score is $$ U_\theta(x)=\frac{\partial}{\partial\theta}\log f_\theta(x) =\frac{\partial f_\theta(x)/\partial\theta}{f_\theta(x)}. $$ The Fisher information in one observation is $I(\theta)=E_\theta[U_\theta(X)^2]$, when this expectation exists. For a sample, the score is the derivative of the log of the joint probability mass function or density.

Model-Side Conditions

The following is a standard sufficient framework for justifying the score-mean identity. It is a useful framework, not a claim that every model where an identity happens to hold must satisfy every condition listed here.

Open parameter range: the parameter lies in an open interval, and the conditions hold in a neighborhood of the value being considered. This avoids treating an endpoint as if it were an ordinary interior point.
Common support: the support of $f_\theta$, as a function of $x$, does not depend on $\theta$. A common dominating measure is used: counting measure for a discrete model or Lebesgue measure for a continuous density.
Differentiability: for almost every $x$ in the common support, $f_\theta(x)$ is differentiable in $\theta$.
Interchange for total probability: differentiation can be interchanged with integration or summation of $f_\theta$. For example, a suitable local integrable bound on $\left|\partial f_\theta(x)/\partial\theta\right|$ can justify differentiation under the integral. For a discrete model, a corresponding bound or convergence result must justify differentiating the infinite sum.
Finite information: $E_\theta[U_\theta(X)^2]$ exists and is finite. This also gives a finite second moment for the score.

Under these model-side conditions, total probability is one for every $\theta$, and differentiating that identity gives

1=\int f_\theta(x)\,dx \quad\Longrightarrow\quad 0=\int \frac{\partial f_\theta(x)}{\partial\theta}\,dx =\int U_\theta(x)f_\theta(x)\,dx =E_\theta[U_\theta(X)].

For a discrete model, the integrals are replaced by sums. The important step is the justified interchange of differentiation with integration or summation. Merely differentiating the density at each fixed $x$ does not establish the identity.

The Additional Condition for an Estimator

Let $T=T(X)$ be an estimator, and suppose its expectation is $\tau(\theta)$. To relate the derivative of that expectation to the score, one must also justify differentiating the expectation itself. A suitable condition is that $T(x)f_\theta(x)$ can be differentiated under the integral or sum locally in $\theta$. One sufficient way to establish this is an integrable local bound on $\left|T(x)\partial f_\theta(x)/\partial\theta\right|$, together with the usual differentiability assumptions. The exact conditions can depend on the model and estimator.

With a common support and a justified interchange, the resulting identity is

\frac{d}{d\theta}E_\theta[T(X)] =\int T(x)\frac{\partial f_\theta(x)}{\partial\theta}\,dx =E_\theta[T(X)U_\theta(X)].

For the usual Cramér–Rao argument, the estimator must also have finite variance, and the score must have finite variance. These requirements ensure the expectations and covariance calculations used in that argument are well defined. If $T$ is unbiased for $\theta$, its expectation is $\tau(\theta)=\theta$, so the left side is $1$. This estimator-side differentiation is an additional requirement; it does not follow just from differentiating total probability.

Key distinction: Model-side regularity can establish $E_\theta[U_\theta(X)]=0$. The relation involving $E_\theta[T(X)U_\theta(X)]$ additionally requires justified differentiation of $E_\theta[T(X)]$. Finite score information alone does not supply that estimator-side justification.

Worked Example: Checking the Conditions in a Bernoulli Model

Let $X\sim\operatorname{Bernoulli}(p)$, with $0<p<1$, so $P_p(X=1)=p$ and $P_p(X=0)=1-p$. There is no interest-rate or mortality assumption in this probability model. Check the score-mean identity and the estimator identity for $T=X$.

Step 1: Check the model conditions. The support is the fixed set $\{0,1\}$, the parameter range is open, and both probabilities are differentiable in $p$. All sums have only two terms, so differentiation and summation can be interchanged directly. The score has finite second moment for every $0<p<1$.

Step 2: Calculate the score. For $x\in\{0,1\}$,

U_p(x)=\frac{x}{p}-\frac{1-x}{1-p} =\frac{x-p}{p(1-p)}.

Thus $U_p(1)=1/p$ and $U_p(0)=-1/(1-p)$. Its expectation is

E_p[U_p(X)] =p\left(\frac{1}{p}\right) +(1-p)\left(-\frac{1}{1-p}\right) =1-1=0.

Step 3: Check finite information. Since the score takes two finite values,

I(p)=E_p[U_p(X)^2] =p\frac{1}{p^2}+(1-p)\frac{1}{(1-p)^2} =\frac{1}{p(1-p)}.

Step 4: Check the estimator-side identity. Here $T=X$ is bounded, so its expectation is a finite sum that can be differentiated term by term. Since $E_p[X]=p$, its derivative is $1$. Directly,

E_p[XU_p(X)] =p\left(\frac{1}{p}\right)+(1-p)(0)=1.

The two calculations agree. The fixed finite support makes both interchanges straightforward, and $T$ has finite variance. The common mistake to avoid is to treat the score-mean identity alone as enough to justify the estimator identity; here both are justified, but by checking the estimator expectation separately.

Worked Example: An Infinite Discrete Support With a Valid Interchange

Suppose $X_1,\ldots,X_n$ are independent with $X_j\sim\operatorname{Poisson}(\lambda)$, where $\lambda>0$, and take $T=\bar X$. There is no interest-rate or mortality assumption. Check the support, score, information, and estimator-side identity for this model.

Step 1: Check the support and differentiability. The joint support is all $n$-tuples of nonnegative integers, independent of $\lambda$, and $\lambda$ lies in an open interval. The joint probability is differentiable in $\lambda$. Unlike the Bernoulli example, the support is infinite, so the interchange for the sum needs justification. For $\lambda$ in a compact neighborhood inside $(0,\infty)$, Poisson probabilities and their derivatives have summable tails; multiplying by $\bar X$ still gives a summable bound. Thus the total-probability and $T$-weighted sums can be differentiated there. The Poisson distribution also has finite moments of all orders for finite $\lambda$.

Step 2: Calculate the joint score. With $S=\sum_{j=1}^n X_j$, the joint log probability, up to the displayed terms, is

\log f_\lambda(x_1,\ldots,x_n) =-n\lambda+S\log\lambda-\sum_{j=1}^n\log(x_j!).

Differentiating gives $U_\lambda=S/\lambda-n$. Since $E_\lambda[S]=n\lambda$,

E_\lambda[U_\lambda] =\frac{n\lambda}{\lambda}-n=0.

Step 3: Verify finite information. Independence gives $\operatorname{Var}_\lambda(S)=n\lambda$. Therefore,

I(\lambda)=E_\lambda[U_\lambda^2] =\operatorname{Var}_\lambda\left(\frac{S}{\lambda}-n\right) =\frac{n\lambda}{\lambda^2} =\frac{n}{\lambda}.

Step 4: Check the estimator identity. Since $E_\lambda[\bar X]=\lambda$, its derivative is $1$. To verify the score expression, use $E[S^2]=n\lambda+(n\lambda)^2$:

E_\lambda[\bar XU_\lambda] =\frac{1}{n}E_\lambda\left[S\left(\frac{S}{\lambda}-n\right)\right] =\frac{1}{n}\left(\frac{n\lambda+(n\lambda)^2}{\lambda}-n(n\lambda)\right) =1.

The differentiability of the expectation is justified here as well as the differentiability of total probability. A common mistake is to assume that an infinite sum may always be differentiated term by term; the summability or domination condition must still be checked.

Worked Example: Why a Moving Endpoint Breaks the Usual Score Identity

Let $X\sim\operatorname{Uniform}(0,\theta)$, where $\theta>0$, with density $f_\theta(x)=1/\theta$ for $0<x<\theta$ and zero otherwise. Consider $T=2X$, which is unbiased for $\theta$. There is no interest-rate or mortality assumption. Identify the failure and check the resulting score calculations.

Step 1: Locate the support problem. The support $(0,\theta)$ depends on $\theta$. At a fixed interior value $0<x<\theta$, the pointwise log-density derivative is $U_\theta(x)=-1/\theta$. But this derivative describes only the change in density at points that remain in the interior; it does not account for the moving upper endpoint.

Step 2: Test the score-mean identity. Taking the expectation of that interior score gives

E_\theta[U_\theta(X)]=-\frac{1}{\theta}\ne 0.

The failure can be seen by differentiating total probability correctly. The density is $1/\theta$ on an interval of length $\theta$, so its integral is one. The moving-endpoint contribution matters: the interior integral of $\partial f_\theta/\partial\theta=-1/\theta^2$ is $-1/\theta$, while the upper endpoint contributes $f_\theta(\theta^-)=1/\theta$. Together they give zero.

Step 3: Compare the estimator derivative with the score expression. The mean of $X$ is $\theta/2$, so $E_\theta[T]=\theta$ and its derivative is $1$. In contrast, the pointwise score calculation gives

E_\theta[TU_\theta] =E_\theta\left[2X\left(-\frac{1}{\theta}\right)\right] =-\frac{2}{\theta}\frac{\theta}{2} =-1.

The discrepancy occurs because differentiating the expectation also produces a boundary contribution. Indeed, the interior integral of $T(x)\partial f_\theta(x)/\partial\theta$ is $-1$, while the endpoint contribution $T(\theta)f_\theta(\theta^-)=2$ makes the full derivative equal to $1$.

Step 4: Check the misleading formal calculation. The interior score has second moment $E_\theta[U_\theta^2]=1/\theta^2$, while $\operatorname{Var}_\theta(T)=4\operatorname{Var}(X)=\theta^2/3$. Applying a regular-model bound using that score information would give a lower bound of $\theta^2$, which exceeds this estimator’s variance. This is not a contradiction: the regularity conditions needed for that argument fail. The common mistake is to differentiate the density only at fixed interior $x$ and ignore the parameter-dependent boundary.

A Practical Regularity Check

Before using a score identity or a Cramér–Rao calculation, write down the model and check each link in the argument. In particular, distinguish differentiating total probability from differentiating an estimator’s expectation. A model may pass the first check while a proposed estimator fails the second, for example because $T$ is not integrable enough for the weighted derivative to be interchanged.

Fix the model and parameter point.
State $f_\theta$, the parameter range, the support, and whether the observation is discrete or continuous.

Check model-side differentiation.
Verify differentiability and a valid interchange for the total probability sum or integral. A support that moves with $\theta$ is an immediate warning.

Check information.
Confirm that the score is defined almost surely and that its second moment is finite.

Check the estimator separately.
For the intended $T$, justify differentiating $E_\theta[T]$, and verify that $T$ has finite variance when the argument requires it.

This checklist is aligned with the SOA VEE Mathematical Statistics syllabus. Its main purpose is not to replace calculation, but to show when the calculation has a valid mathematical basis. In regular fixed-support models, the needed identities often follow from straightforward differentiation. When support depends on the parameter, boundary terms may change the result, so the ordinary argument cannot simply be reused.

Check Your Understanding

For each problem, decide which regularity checks apply and carry out the requested calculation.

Let $X\sim\operatorname{Bernoulli}(p)$, $0<p<1$. Find the score at $x=0$ and $x=1$, then give its expectation. (Answer: $U_p(0)=-1/(1-p)$, $U_p(1)=1/p$, and $E_p[U_p(X)]=0$.)
Let $X_1,\ldots,X_4$ be independent $\operatorname{Poisson}(\lambda)$ variables with $\lambda=2$. Find the joint Fisher information and state whether the support depends on $\lambda$. (Answer: $I(2)=4/2=2$; the joint support is the fixed set of nonnegative-integer 4-tuples.)
For $X\sim\operatorname{Uniform}(0,\theta)$, $\theta>0$, use the interior score to find $E_\theta[U_\theta(X)]$. Does the score-mean identity follow from the usual common-support argument? (Answer: $E_\theta[U_\theta(X)]=-1/\theta$; no, because the upper endpoint depends on $\theta$, and its boundary contribution cannot be omitted.)
Let $X$ have density $f_\lambda(x)=\lambda e^{-\lambda x}$ for $x>0$, with $\lambda>0$, and take $T=X^2$. Find $E_\lambda[T]$, its derivative, and $E_\lambda[TU_\lambda(X)]$. (Answer: $E_\lambda[T]=2/\lambda^2$, its derivative is $-4/\lambda^3$, and $E_\lambda[TU_\lambda(X)]=-4/\lambda^3$; the support is fixed and the required moments are finite.)
Suppose a model satisfies the model-side conditions and has finite Fisher information, but an estimator $T$ has infinite variance. Is that alone enough to use the usual finite-variance Cramér–Rao argument for $T$? (Answer: No; finite variance of the estimator is also required, and differentiating $E_\theta[T]$ must be justified separately.)

What You'll Learn