Gamma and Beta Families in Canonical Coordinates

Why Canonical Coordinates Matter for These Families

A familiar distribution can look different when written in canonical coordinates. The useful step is to identify which functions of the observation multiply the parameters in the log density. For gamma and beta distributions, the shape parameters multiply logarithms of the observation or its complement; the rate parameter of a gamma distribution multiplies the observation itself.

The choice of which parameters are fixed matters. A gamma family with fixed shape and varying rate has one canonical statistic, while allowing shape to vary introduces a second. A beta family with both shape parameters varying has two canonical statistics, one for each endpoint of its support. In each case, integrability determines the natural parameter space, and the log-partition function supplies the normalizing constant.

Canonical form: A density \(f(x;\eta)\), relative to a specified carrier measure, is in canonical form if it can be written \(f(x;\eta)=h(x)\exp\{\eta^{\mathsf T}T(x)-A(\eta)\}\). Here \(T(x)\) is the canonical statistic, \(\eta\) is the natural parameter, \(h(x)\) is the carrier density, and \(A(\eta)\) is the log-partition function. Its value is determined by requiring the density to integrate to one.

Gamma Distributions: Fixed and Varying Shape

Use the shape-rate convention: for shape \(\alpha>0\) and rate \(\beta>0\), a gamma random variable has density \(f(x)=\beta^\alpha x^{\alpha-1}e^{-\beta x}/\Gamma(\alpha)\) for \(x>0\), and density zero otherwise. The carrier measure below is Lebesgue measure. The gamma function is \(\Gamma(a)=\int_0^\infty x^{a-1}e^{-x}\,dx\), defined for \(a>0\).

Taking logarithms of the gamma density gives \(\alpha\log\beta-\log\Gamma(\alpha)+(\alpha-1)\log x-\beta x\). This immediately shows how the canonical statistics depend on which parameters are allowed to vary.

Fixed shape, varying rate: Fix \(\alpha>0\). Set \(T(x)=x\), \(\eta=-\beta\), and \(h(x)=x^{\alpha-1}\) for \(x>0\). The natural parameter space is \(\eta<0\), and \(A(\eta)=\log\Gamma(\alpha)-\alpha\log(-\eta)\).

Varying shape and rate: Set \(T(x)=(\log x,x)\), \(\eta_1=\alpha-1\), and \(\eta_2=-\beta\), with \(h(x)=1\) on \(x>0\). The natural parameter space is \(\eta_1>-1,\ \eta_2<0\), and \(A(\eta_1,\eta_2)=\log\Gamma(\eta_1+1)-(\eta_1+1)\log(-\eta_2)\).

The fixed-shape representation puts the factor \(x^{\alpha-1}\) in the carrier. The varying-shape representation cannot do so, because that factor depends on a parameter; instead, \(\log x\) enters the canonical statistic. These are different representations of gamma subfamilies, not different probability laws.

Theorem: For the varying-shape gamma representation with carrier density \(h(x)=1\) on \(x>0\), the normalizing integral is finite exactly when \(\eta_1>-1\) and \(\eta_2<0\). On this natural parameter space it equals \(\Gamma(\eta_1+1)(-\eta_2)^{-(\eta_1+1)}\), so its logarithm is the log-partition function stated above.

Proof. Write the normalizing integral as \(\int_0^\infty x^{\eta_1}e^{\eta_2x}\,dx\). Near zero the exponential factor is bounded above and below by positive constants, so integrability there holds exactly when \(\eta_1>-1\). When this condition holds, if \(\eta_2\geq 0\), the integrand does not decay at infinity and the integral diverges. If \(\eta_2<0\), substitute \(u=(-\eta_2)x\). Then \(x=u/(-\eta_2)\) and \(dx=du/(-\eta_2)\), giving

\int_0^\infty x^{\eta_1}e^{\eta_2x}\,dx =(-\eta_2)^{-(\eta_1+1)} \int_0^\infty u^{\eta_1}e^{-u}\,du =(-\eta_2)^{-(\eta_1+1)}\Gamma(\eta_1+1).

Taking the logarithm of this integral gives \(A(\eta_1,\eta_2)\). This proves both the natural parameter restrictions and the stated normalizer. \(\square\)

Worked Example: Gamma With Fixed Shape

Let \(X\) have a gamma distribution with fixed shape \(\alpha=2\) and rate \(\beta=3\). Its density is \(f(x)=3^2x^{2-1}e^{-3x}/\Gamma(2)\) for \(x>0\). Since \(\Gamma(2)=1\), this is \(9xe^{-3x}\).

For the fixed-shape canonical representation, \(T(x)=x\), \(\eta=-3\), and \(h(x)=x\). The log-partition function is \(A(-3)=\log\Gamma(2)-2\log(3)=-2\log 3\). Thus the canonical expression is \(x\exp\{-3x- A(-3)\}=x\exp\{-3x+2\log 3\}=9xe^{-3x}\), as required. The natural parameter condition \(\eta<0\) is satisfied.

Worked Example: Gamma With Both Parameters Varying

Take shape \(\alpha=3\) and rate \(\beta=2\). The canonical parameters are \(\eta_1=\alpha-1=2\) and \(\eta_2=-\beta=-2\). The log-partition function is \(A(2,-2)=\log\Gamma(3)-3\log 2=\log 2-3\log 2=-2\log 2\), because \(\Gamma(3)=2\).

The canonical density is \(\exp\{2\log x-2x-A(2,-2)\}=x^2e^{-2x}\exp(2\log 2)=4x^2e^{-2x}\) for \(x>0\). The usual gamma density gives \(2^3x^{3-1}e^{-2x}/\Gamma(3)=8x^2e^{-2x}/2=4x^2e^{-2x}\), which agrees. Both natural parameter inequalities hold: \(2>-1\) and \(-2<0\).

Beta Distributions: Two Endpoint Statistics

For shape parameters \(a>0\) and \(b>0\), the beta density is \(f(x)=x^{a-1}(1-x)^{b-1}/B(a,b)\) on \(0<x<1\), and zero elsewhere. The beta function is \(B(a,b)=\int_0^1 x^{a-1}(1-x)^{b-1}\,dx\). When both shapes vary, take carrier density \(h(x)=1\) on \((0,1)\), canonical statistic \(T(x)=(\log x,\log(1-x))\), and natural parameters \(\eta_1=a-1,\ \eta_2=b-1\). The candidate log-partition function is \(\log B(\eta_1+1,\eta_2+1)\).

Theorem: For the beta canonical representation with carrier density \(1\) on \((0,1)\), the natural parameter space is exactly \(\eta_1>-1,\ \eta_2>-1\). On this space the normalizing integral is \(B(\eta_1+1,\eta_2+1)\), and \(A(\eta_1,\eta_2)=\log B(\eta_1+1,\eta_2+1)\).

Proof. The normalizing integral is \(\int_0^1 x^{\eta_1}(1-x)^{\eta_2}\,dx\). Near zero, \((1-x)^{\eta_2}\) is bounded above and below by positive constants, so integrability there holds exactly when \(\eta_1>-1\). Near one, \(x^{\eta_1}\) is bounded above and below by positive constants, so integrability there holds exactly when \(\eta_2>-1\). These two conditions are therefore necessary and sufficient.

For completeness, the beta-gamma identity follows by multiplying the two gamma integrals, for \(p,q>0\), and changing variables from \(u,v>0\) to \(r=u+v\), \(t=u/(u+v)\). The inverse is \(u=rt,\ v=r(1-t)\), with absolute Jacobian \(r\). Thus

\Gamma(p)\Gamma(q) =\int_0^\infty\int_0^\infty u^{p-1}v^{q-1}e^{-(u+v)}\,du\,dv =\int_0^\infty r^{p+q-1}e^{-r}\,dr \int_0^1 t^{p-1}(1-t)^{q-1}\,dt =\Gamma(p+q)B(p,q).

All integrands are nonnegative, so the change of variables and separation of the integrals are justified by Tonelli's theorem. This identity also gives \(B(p,q)=\Gamma(p)\Gamma(q)/\Gamma(p+q)\). Setting \(p=\eta_1+1\) and \(q=\eta_2+1\) identifies the normalizer and hence its logarithm \(A\). \(\square\)

Worked Example: Beta With Both Shapes Varying

Let \(X\) have a beta distribution with \(a=2\) and \(b=3\). Its canonical parameters are \(\eta_1=1\) and \(\eta_2=2\). The normalizer is \(B(2,3)=\Gamma(2)\Gamma(3)/\Gamma(5)=1\cdot2/24=1/12\), so \(A(1,2)=\log(1/12)=-\log 12\).

The canonical expression is \(\exp\{\log x+2\log(1-x)-A(1,2)\}=12x(1-x)^2\). The standard beta density gives the same coefficient, \(1/B(2,3)=12\), and the same powers. Its natural parameters satisfy \(1>-1\) and \(2>-1\).

Fixing a Beta Shape Changes the Representation

If \(a>0\) is fixed and only \(b\) varies, use \(T(x)=\log(1-x)\), \(\eta=b-1>-1\), and \(h(x)=x^{a-1}\) on \((0,1)\). Then \(A(\eta)=\log B(a,\eta+1)\). If instead \(b\) is fixed and \(a\) varies, use \(T(x)=\log x\), \(\eta=a-1>-1\), and \(h(x)=(1-x)^{b-1}\); the log-partition function is \(A(\eta)=\log B(\eta+1,b)\).

Worked Example: Beta With One Shape Fixed

Fix \(a=2\) and take \(b=3\). In the fixed-\(a\) representation, \(h(x)=x\), \(T(x)=\log(1-x)\), and \(\eta=b-1=2\). The log-partition function is \(A(2)=\log B(2,3)=-\log 12\). Therefore \(h(x)\exp\{\eta T(x)-A(\eta)\}=x\exp\{2\log(1-x)+\log 12\}=12x(1-x)^2\).

This recovers the same beta density as the two-statistic representation, but the fixed shape is now part of the carrier rather than a canonical parameter. The one-dimensional natural parameter space is \(\eta>-1\): near \(x=1\), the varying factor behaves as \((1-x)^\eta\), which is integrable exactly under that condition; near zero the fixed carrier factor \(x\) is integrable.

Reading and Checking the Canonical Forms

The partition functions are not optional constants: they are exactly the logarithms of the integrals that normalize the canonical kernels. For the full gamma family, the kernel is \(x^{\eta_1}e^{\eta_2x}\); for the full beta family, it is \(x^{\eta_1}(1-x)^{\eta_2}\). The natural parameter restrictions arise from whether those kernels are integrable at the boundaries of their supports.

A reliable check is to substitute the canonical parameters back into \(h(x)\exp\{\eta^{\mathsf T}T(x)-A(\eta)\}\) and simplify. This checks both the signs and the normalizing constant. In particular, the gamma rate is positive in conventional coordinates but its canonical coordinate is negative, \(\eta_2=-\beta\).

Common mistake: It is incorrect to write the varying-shape gamma family with only \(T(x)=x\) and put \(x^{\alpha-1}\) into the carrier while claiming that the carrier is parameter-independent. When \(\alpha\) varies, \(x^{\alpha-1}\) depends on the parameter. The correction is to include \(\log x\) as a canonical statistic, with coefficient \(\eta_1=\alpha-1\). The same reasoning puts \(\log x\) and \(\log(1-x)\) into the full beta statistic when both shapes vary.

Key takeaway: For gamma and beta families, derive canonical coordinates by collecting parameter coefficients in the log density, then derive the natural parameter space from endpoint integrability. Fixing a shape parameter moves its associated power into the carrier and removes its logarithm from the canonical statistic.

Check Your Understanding

Use the canonical forms and endpoint conditions developed above.

For a gamma distribution with fixed shape \(\alpha=4\), what are the canonical statistic, natural parameter, and natural parameter space when the rate varies?
For a gamma density with both shape and rate varying, explain why \(\eta_1\leq -1\) is excluded.
Write the canonical parameters and log-partition function for a beta distribution with shapes \(a=3\) and \(b=2\).
If the second beta shape is fixed, which factor belongs in the carrier density, and what statistic corresponds to the varying shape?
Why does a gamma rate \(\beta>0\) correspond to a negative canonical parameter?