Gamma Densities with Fixed Shape

From a Gamma Density to Canonical Form

A gamma model can be indexed by its shape and rate. When the shape is fixed and only the rate changes, the density has a particularly direct exponential-family representation: the response itself is the sufficient statistic, and the rate becomes the negative of the natural parameter. The minus sign matters because a gamma density must decay as the response grows.

Throughout, let the fixed shape be $\alpha>0$, let the varying rate be $\lambda>0$, and let the response be $Y>0$. Using the rate parameterization, the gamma density is

f(y;\alpha,\lambda) = \frac{\lambda^\alpha}{\Gamma(\alpha)} y^{\alpha-1}e^{-\lambda y}, \qquad y>0,

where $\Gamma(\alpha)=\int_0^\infty t^{\alpha-1}e^{-t}\,dt$. Outside $y>0$, the density is zero. The shape $\alpha$ is held fixed in this tutorial; only $\lambda$ varies.

Definition: A density is in canonical one-parameter exponential-family form if it can be written as $f(y;\eta)=h(y)\exp\{\eta T(y)-A(\eta)\}$, with a parameter-free base measure $h(y)$, statistic $T(y)$, natural parameter $\eta$, and log-partition function $A(\eta)$. For the gamma family with fixed shape $\alpha>0$, one such representation is $$ T(y)=y,\qquad \eta=-\lambda<0,\qquad h(y)=\frac{y^{\alpha-1}}{\Gamma(\alpha)}\mathbf{1}_{\{y>0\}}, \qquad A(\eta)=-\alpha\log(-\eta). $$

The indicator in $h(y)$ records the common support $y>0$. The power $y^{\alpha-1}$ is part of the base measure because $\alpha$ is fixed; it does not vary with $\eta$. The natural parameter is negative because $\eta=-\lambda$, while every valid rate satisfies $\lambda>0$.

Deriving the Log-Partition Function

The log-partition function is determined by the requirement that the density integrate to one. With the base measure above and $T(y)=y$, the normalizing integral is

\int_0^\infty h(y)e^{\eta y}\,dy = \frac{1}{\Gamma(\alpha)} \int_0^\infty y^{\alpha-1}e^{\eta y}\,dy.

For $\eta<0$, set $t=(-\eta)y$. Then $y=t/(-\eta)$ and $dy=dt/(-\eta)$. Applying the definition of the gamma function gives

\frac{1}{\Gamma(\alpha)} \int_0^\infty y^{\alpha-1}e^{\eta y}\,dy = \frac{1}{\Gamma(\alpha)(-\eta)^\alpha} \int_0^\infty t^{\alpha-1}e^{-t}\,dt = (-\eta)^{-\alpha}.

In canonical form, the normalizing factor is $e^{A(\eta)}$. Therefore $e^{A(\eta)}=(-\eta)^{-\alpha}$, so $A(\eta)=-\alpha\log(-\eta)$ for $\eta<0$. Equivalently, substituting $\eta=-\lambda$ yields

h(y)e^{\eta y-A(\eta)} = \frac{y^{\alpha-1}}{\Gamma(\alpha)} \exp\!\left\{-\lambda y+\alpha\log\lambda\right\} = \frac{\lambda^\alpha}{\Gamma(\alpha)} y^{\alpha-1}e^{-\lambda y}.

This recovers the original gamma density, including its normalizing constant. In this convention the factor $1/\Gamma(\alpha)$ is included in the base measure, so it does not appear in $A(\eta)$. A different allocation of parameter-free constants can change the displayed base measure and log-partition function together, but not the density.

Theorem: For fixed $\alpha>0$, the natural-parameter space for this representation is exactly $\{\eta:\eta<0\}$, and its log-partition function is $A(\eta)=-\alpha\log(-\eta)$.

Proof. When $\eta<0$, the substitution above proves that the normalizing integral is finite and equals $(-\eta)^{-\alpha}$. At $\eta=0$, the integral is proportional to $\int_0^\infty y^{\alpha-1}\,dy$, which diverges because $\alpha>0$. If $\eta>0$, the integrand $y^{\alpha-1}e^{\eta y}$ does not have an integrable tail; in particular, its exponential factor grows without bound. Thus the integral is finite exactly for $\eta<0$. Taking its logarithm gives $A(\eta)=-\alpha\log(-\eta)$. $\square$

Worked Representations

Worked Example: Shape 2 and Rate 3

Let $\alpha=2$ and $\lambda=3$. Since $\Gamma(2)=1$, the gamma density is

f(y;2,3)=9y e^{-3y},\qquad y>0.

The canonical quantities are $\eta=-3$, $T(y)=y$, $h(y)=y$ for $y>0$, and

A(-3)=-2\log(3).

Reconstructing the density,

h(y)e^{\eta y-A(\eta)} = y\exp\{-3y+2\log(3)\} = 9y e^{-3y}.

The coefficient $9$ comes from $e^{-A(-3)}$; the power of $y$ remains in the base measure.

Worked Example: Shape One-Half and Rate 4

For $\alpha=\tfrac12$, $\lambda=4$, and $\Gamma(\tfrac12)=\sqrt{\pi}$, the density is

f(y;\tfrac12,4) = \frac{2}{\sqrt{\pi}}y^{-1/2}e^{-4y}, \qquad y>0.

Here $\eta=-4$, $h(y)=y^{-1/2}/\sqrt{\pi}$, and

A(-4)=-\frac12\log(4)=-\log(2).

Thus

h(y)e^{\eta y-A(\eta)} = \frac{y^{-1/2}}{\sqrt{\pi}} e^{-4y+\log(2)} = \frac{2}{\sqrt{\pi}}y^{-1/2}e^{-4y}.

The base measure has a negative power of $y$, but this causes no problem: the density is integrable near zero because its exponent is $-\tfrac12>-1$.

Worked Example: Shape 3 and Rate One-Half

Let $\alpha=3$, $\lambda=\tfrac12$. Since $\Gamma(3)=2$, the density becomes

f(y;3,\tfrac12) = \frac{y^2}{16}e^{-y/2}, \qquad y>0.

The base measure is $h(y)=y^2/2$, the natural parameter is $\eta=-\tfrac12$, and the log-partition value is

A(-\tfrac12) = -3\log(\tfrac12) = 3\log(2).

Consequently,

h(y)e^{\eta y-A(\eta)} = \frac{y^2}{2}e^{-y/2-3\log(2)} = \frac{y^2}{2}e^{-y/2}\frac{1}{8} = \frac{y^2}{16}e^{-y/2}.

This example also checks the normalizing constant directly: the exponential factor contributed $1/8$, and the base measure contributed $1/2$.

What the Fixed-Shape Assumption Does

Fixing $\alpha$ is not a cosmetic choice. It makes $y^{\alpha-1}/\Gamma(\alpha)$ a parameter-free base measure while the rate changes through $\eta=-\lambda$ and $A(\eta)$. If the shape changes as well, this displayed base measure changes with the parameter, so it is no longer the same one-parameter representation. This tutorial's canonical form describes varying rate at one fixed shape, not simultaneous variation of shape and rate.

The natural parameter also has an interpretable sign constraint. Increasing $\lambda$ makes $\eta=-\lambda$ more negative, increasing the strength of the exponential decay $e^{\eta y}$. A nonnegative $\eta$ cannot normalize this power-law carrier over the positive half-line, as the normalizing-integral calculation showed.

Theorem: For fixed $\alpha>0$, the gamma densities indexed by $\lambda>0$ form a regular one-parameter exponential family in the representation above: their support is common, their natural-parameter space is open, and the statistic $T(y)=y$ is not constant on the support.

Proof. Every density in the family is supported on the same set $(0,\infty)$, independently of $\lambda$. The natural parameter ranges over $(-\infty,0)$, which is an open subset of the real line. Finally, $T(y)=y$ takes different values on subsets of positive measure within $(0,\infty)$; for example, it is at most $1$ on $(0,1)$ and greater than $2$ on $(2,3)$. Thus it is not constant almost everywhere under any of these gamma densities, which are positive throughout their support. These are the stated regularity properties. $\square$

This result is useful as a structural check: the parameter changes through a single scalar natural parameter over an open domain, while the support and carrier remain fixed. It does not require treating the shape as an additional parameter.

Common Mistake: Using the Rate as the Natural Parameter

A frequent error is to write the exponent as $+\lambda y$, set $\eta=\lambda>0$, and then choose a log-partition function to normalize the density. That sign does not reproduce the gamma kernel: $e^{+\lambda y}$ grows rather than decays as $y$ increases, and its integral over $(0,\infty)$ is infinite. The correction is to retain the density's factor $e^{-\lambda y}$, set $\eta=-\lambda$, and use the domain $\eta<0$.

Another mistake is to put $y^{\alpha-1}$ into the log-partition function. It depends on the observation $y$, whereas $A(\eta)$ may depend only on the natural parameter. With fixed shape, this power belongs in the base measure; the log-partition function is the parameter-only term $-\alpha\log(-\eta)$.

Key takeaway: At fixed shape $\alpha>0$, a gamma response with rate $\lambda>0$ has statistic $T(y)=y$, natural parameter $\eta=-\lambda$, power-law base measure $y^{\alpha-1}/\Gamma(\alpha)$ on $y>0$, and log-partition function $A(\eta)=-\alpha\log(-\eta)$ on $\eta<0$.

Check Your Understanding

Use the fixed-shape gamma representation to answer the following.

For shape $4$ and rate $2$, identify $\eta$, $T(y)$, $h(y)$, and $A(\eta)$.
Why must the natural parameter be negative in this representation?
For shape $\tfrac32$, write the base measure and log-partition function.
What part of the stated representation would cease to be parameter-free if the shape were allowed to vary?
Show that the log-partition function for shape $2$ at $\eta=-5$ is $-2\log(5)$.