When The Mean Parameter Can Be Inverted

Why Invert the Mean Map?

A canonical parameter describes a distribution through its exponential-family representation, while the mean often gives the parameter a more direct interpretation. To move between these descriptions, we need to know whether the mean identifies exactly one canonical parameter and which mean values are possible. The answer comes from the log-partition function: under regularity conditions, its first derivative is strictly increasing.

There is one important qualification. In a general exponential family, the derivative gives the expectation of the sufficient statistic, not necessarily the expectation of the response itself. In the common GLM families treated below, the sufficient statistic is the response, so the two coincide.

Definition: Consider a one-parameter exponential family with density or mass function, relative to a fixed measure, $$ f_\theta(y)=h(y)\exp\{\theta T(y)-A(\theta)\}, $$ where $A(\theta)=\log\int h(y)e^{\theta T(y)}\,d\nu(y)$ is finite on an open interval $\Theta$. The mean map for the sufficient statistic is $$ \mu(\theta)=E_\theta[T(Y)]=A'(\theta). $$ Its admissible mean range is the image $\{\mu(\theta):\theta\in\Theta\}$, the means attained at finite natural parameters.

We use the interior of the natural-parameter space, so that every $\theta\in\Theta$ has a neighborhood in which the normalizing integral is finite. Assume also that $T(Y)$ is not almost surely constant under the family. These conditions rule out degenerate cases and allow the log-partition derivatives to describe genuine variation in the statistic.

Why the Mean Map Is Strictly Increasing

The key identity is that the second derivative of the log-partition function is the variance of the sufficient statistic. The regularity condition on $\Theta$ justifies differentiating the normalizing integral. In particular, around any $\theta\in\Theta$, choose $\epsilon>0$ such that $\theta-\epsilon,\theta+\epsilon\in\Theta$. For each fixed nonnegative integer $k$, the factor $|T(y)|^k e^{\theta T(y)}$ is bounded by a constant times $e^{(\theta-\epsilon)T(y)}+e^{(\theta+\epsilon)T(y)}$; the latter terms are integrable against $h\,d\nu$. This supplies a local integrable bound for the derivatives needed below.

Theorem: For the family above, $A'(\theta)=E_\theta[T(Y)]$ and $A''(\theta)=\operatorname{Var}_\theta(T(Y))$. If $T(Y)$ is not almost surely constant, then $A''(\theta)>0$ for every $\theta\in\Theta$. Consequently, $\theta\mapsto A'(\theta)$ is strictly increasing and has a unique inverse on its image.

Proof. Write $Z(\theta)=\int h(y)e^{\theta T(y)}\,d\nu(y)$, so that $A(\theta)=\log Z(\theta)$. Differentiation under the integral, justified by the local bound above, gives

Z'(\theta)=\int T(y)h(y)e^{\theta T(y)}\,d\nu(y), \qquad Z''(\theta)=\int T(y)^2h(y)e^{\theta T(y)}\,d\nu(y).

Since $f_\theta(y)=h(y)e^{\theta T(y)}/Z(\theta)$, these identities yield $Z'(\theta)/Z(\theta)=E_\theta[T(Y)]$ and $Z''(\theta)/Z(\theta)=E_\theta[T(Y)^2]$. Differentiating $A=\log Z$ therefore gives

A'(\theta)=\frac{Z'(\theta)}{Z(\theta)}=E_\theta[T(Y)], \qquad A''(\theta)=\frac{Z''(\theta)}{Z(\theta)} -\left(\frac{Z'(\theta)}{Z(\theta)}\right)^2 =\operatorname{Var}_\theta(T(Y)).

A random variable has variance zero exactly when it is almost surely constant. Thus the nondegeneracy assumption implies $A''(\theta)>0$. For $\theta_1<\theta_2$, the mean value theorem gives some $c\in(\theta_1,\theta_2)$ such that

A'(\theta_2)-A'(\theta_1)=A''(c)(\theta_2-\theta_1)>0.

So $A'$ is strictly increasing. A strictly increasing function takes each value in its image exactly once, which proves that the inverse exists there. $\square$

Because $A'$ is continuous and strictly increasing on an open interval, its image is also an interval; its endpoints are determined by the limits of $A'(\theta)$ at the ends of $\Theta$. The image need not equal the entire real line. A mean at an endpoint limit may be approached as $\theta$ tends to the edge of its domain, without being attained at any finite $\theta$.

Corollary: If $A''(\theta)>0$ and $\mu=A'(\theta)$, the inverse map $\theta=\theta(\mu)$ is differentiable on the admissible mean range and $$ \frac{d\theta}{d\mu}=\frac{1}{A''(\theta(\mu))}. $$

To see this, fix $\theta_0\in\Theta$ and set $\mu_0=A'(\theta_0)$. Strict monotonicity and continuity imply continuity of the inverse near $\mu_0$. For $\mu\ne\mu_0$, the difference quotient for the inverse is the reciprocal of the difference quotient for $A'$:

\frac{\theta(\mu)-\theta_0}{\mu-\mu_0} = \left[ \frac{A'(\theta(\mu))-A'(\theta_0)} {\theta(\mu)-\theta_0} \right]^{-1}.

As $\mu\to\mu_0$, continuity of the inverse gives $\theta(\mu)\to\theta_0$. The bracketed term therefore tends to $A''(\theta_0)>0$, proving the derivative formula.

Worked Inversions

Worked Example: Bernoulli Mean and Log-Odds

Let $Y$ be Bernoulli with success probability $p$. Its canonical representation has $T(y)=y$, natural parameter $\theta=\log\{p/(1-p)\}$, and log-partition function $A(\theta)=\log(1+e^\theta)$.

Differentiate the log-partition function:

A'(\theta)=\frac{e^\theta}{1+e^\theta}.

Thus $\mu=E[Y]=e^\theta/(1+e^\theta)$. Since $\theta$ ranges over all real numbers, this expression is strictly increasing and takes values strictly between 0 and 1. To solve for $\theta$, multiply by the denominator and rearrange:

\mu(1+e^\theta)=e^\theta, \qquad \mu=e^\theta(1-\mu), \qquad \theta=\log\left(\frac{\mu}{1-\mu}\right),\quad 0<\mu<1.

For example, if $\mu=0.8$, then $\theta=\log(0.8/0.2)=\log 4$. The values $\mu=0$ and $\mu=1$ are not attained at finite $\theta$; they are limits as $\theta\to-\infty$ and $\theta\to+\infty$, respectively.

Worked Example: Poisson Mean and Log-Rate

For a Poisson response with mean $\lambda$, the canonical parameter is $\theta=\log\lambda$, and $A(\theta)=e^\theta$. Hence

\mu=A'(\theta)=e^\theta.

The natural parameter ranges over $\mathbb{R}$, so the mean ranges over $(0,\infty)$. Inverting gives $\theta=\log\mu$. For a mean of 3, the unique natural parameter is $\theta=\log 3$, and substituting confirms $A'(\log 3)=e^{\log 3}=3$. Zero is a limiting mean as $\theta\to-\infty$, not the mean at a finite Poisson natural parameter.

Worked Example: Fixed-Shape Gamma Mean

Let $Y$ have a Gamma distribution with fixed shape $\alpha>0$ and rate $-\theta$, where $\theta<0$. Its density can be written

f_\theta(y)=\frac{y^{\alpha-1}}{\Gamma(\alpha)} \exp\{\theta y+\alpha\log(-\theta)\},\qquad y>0.

In the form $h(y)\exp\{\theta y-A(\theta)\}$, this means $A(\theta)=-\alpha\log(-\theta)$. Differentiating on the domain $\theta<0$ gives

A'(\theta)=-\frac{\alpha}{\theta}, \qquad A''(\theta)=\frac{\alpha}{\theta^2}>0.

As $\theta$ ranges over $(-\infty,0)$, the mean $\mu=-\alpha/\theta$ ranges over $(0,\infty)$. Solving for the parameter gives $\theta=-\alpha/\mu$. For $\alpha=2$ and $\mu=4$, the parameter is $\theta=-1/2$; substitution gives $-2/(-1/2)=4$. The mean is uniquely recovered, and no finite $\theta<0$ gives mean zero.

Admissible Mean Ranges in Common Families

The same procedure—differentiate the log-partition function, then find the image over the natural-parameter domain—gives the following ranges. The listed means refer to the response $Y$, with the family’s fixed quantities held fixed.

Family	Natural parameter and domain	Mean map	Admissible mean range
Bernoulli	$\theta\in\mathbb{R}$	$\mu=e^\theta/(1+e^\theta)$	$(0,1)$
Binomial with fixed size $m$	$\theta\in\mathbb{R}$	$\mu=m e^\theta/(1+e^\theta)$	$(0,m)$
Poisson	$\theta\in\mathbb{R}$	$\mu=e^\theta$	$(0,\infty)$
Normal with fixed variance $\sigma^2>0$	$\theta\in\mathbb{R}$	$\mu=\sigma^2\theta$	$\mathbb{R}$
Gamma with fixed shape $\alpha>0$	$\theta<0$	$\mu=-\alpha/\theta$	$(0,\infty)$
Inverse Gaussian in its mean-dispersion representation	$\theta<0$	$\mu=1/\sqrt{-2\theta}$	$(0,\infty)$

For the binomial row, differentiating $A(\theta)=m\log(1+e^\theta)$ gives $m e^\theta/(1+e^\theta)$, so the response mean is strictly between 0 and $m$. For the normal row, $A(\theta)=\sigma^2\theta^2/2$, so $A'(\theta)=\sigma^2\theta$, which covers every real mean. In the inverse Gaussian mean-dispersion representation, the cumulant function is $\kappa(\theta)=-\sqrt{-2\theta}$; differentiating gives $\kappa'(\theta)=1/\sqrt{-2\theta}$. As $\theta$ ranges over negative values, this derivative takes every positive value.

These are ranges for finite natural parameters. For example, a binomial variable can equal 0 or $m$, but a finite canonical parameter does not give either endpoint as its expectation. The endpoint means arise as limits of the family. By contrast, the normal mean map has no finite endpoints to exclude.

Common Mistake: Confusing the Parameter Domain With the Mean Range

A frequent error is to assume that because the natural parameter ranges over all real numbers, the mean must also range over all real numbers. The Bernoulli example disproves this: $\theta\in\mathbb{R}$, but $A'(\theta)\in(0,1)$. The mean range is the image of the derivative, not the natural-parameter domain itself. Another error is to include limiting boundary means as though they came from finite parameter values. For Bernoulli, solving $\mu=e^\theta/(1+e^\theta)$ requires $\theta=\log\{\mu/(1-\mu)\}$, which is not finite at $\mu=0$ or $1$.

Strict monotonicity also depends on nondegeneracy. If $T(Y)$ is almost surely constant, then its variance is zero, $A''(\theta)=0$, and the mean map cannot distinguish natural parameters. For a nondegenerate regular family, positivity of the variance guarantees unique inversion; the actual admissible range must still be found from the specific derivative and parameter domain.

Key takeaway: For a regular, nondegenerate one-parameter exponential family, $A''(\theta)=\operatorname{Var}_\theta(T(Y))>0$. Therefore $A'(\theta)$ is one-to-one, and the natural parameter can be recovered uniquely for every mean in the image of $A'$—but not necessarily for means outside that image or at unattained boundary limits.

Check Your Understanding

Use the derivative of the log-partition function and its natural-parameter domain to answer each question.

In a regular exponential family, why does positive variance of $T(Y)$ imply that the mean map is one-to-one?
For a Bernoulli response, what natural parameter corresponds to mean $0.25$? Are the means 0 and 1 attained at finite natural parameters?
A Poisson mean is 5. Find its natural parameter and state the full admissible mean range.
For a fixed-shape Gamma family with shape $\alpha=3$, find the natural parameter corresponding to mean 6.
Explain why the normal family with fixed positive variance has an admissible mean range different from the Bernoulli family, even though both have natural parameters ranging over all real numbers.

Family	Natural parameter and domain	Mean map	Admissible mean range
Bernoulli	\(\theta\in\mathbb{R}\)	\(\mu=e^\theta/(1+e^\theta)\)	\((0,1)\)
Binomial with fixed size \(m\)	\(\theta\in\mathbb{R}\)	\(\mu=m e^\theta/(1+e^\theta)\)	\((0,m)\)
Poisson	\(\theta\in\mathbb{R}\)	\(\mu=e^\theta\)	\((0,\infty)\)
Normal with fixed variance \(\sigma^2>0\)	\(\theta\in\mathbb{R}\)	\(\mu=\sigma^2\theta\)	\(\mathbb{R}\)
Gamma with fixed shape \(\alpha>0\)	\(\theta<0\)	\(\mu=-\alpha/\theta\)	\((0,\infty)\)
Inverse Gaussian in its mean-dispersion representation	\(\theta<0\)	\(\mu=1/\sqrt{-2\theta}\)	\((0,\infty)\)