Base Measures and Parameter-Free Carrier Terms

Why Separate the Carrier Term?

An exponential-family representation separates the part of a probability model that changes with the parameter from the part that does not. This distinction is useful when forming likelihoods: the parameter-free factors still contribute to a probability or density, but they do not change how the likelihood varies with the parameter. A binomial coefficient, a normal-density constant, and an indicator of a fixed support can all belong to this parameter-free part.

The separation is meaningful only after specifying a reference measure. For discrete outcomes, the usual reference measure is counting measure; for a continuous outcome, it is usually Lebesgue measure. The same carrier function is therefore best understood relative to that measure, rather than as a factor that exists independently of the probability model.

Definition: Let $\nu$ be a reference measure on the response space. A one-parameter exponential-family density or mass function has the form $$ f_\theta(y)=h(y)\exp\{\theta T(y)-A(\theta)\} $$ with respect to $\nu$, where $T(y)$ is the statistic in the kernel, $h(y)\geq 0$ is a parameter-free carrier function, and $A(\theta)$ is the log-partition function. The base measure is the measure $\mu$ defined by $\mu(B)=\int_B h(y)\,\nu(dy)$. Thus the density with respect to $\mu$ is $\exp\{\theta T(y)-A(\theta)\}$.

For this representation to define a probability model, assume $T$ and $h$ are measurable, the measure $\mu$ is well-defined, and $$ 0<\int \exp\{\theta T(y)\}\,\mu(dy)<\infty. $$ Then normalization requires $$ A(\theta)=\log\int \exp\{\theta T(y)\}\,\mu(dy). $$ The carrier must not depend on $\theta$. It may depend on the observed value $y$, and it may be zero outside the support. A factor that depends on the parameter belongs in the parameter-dependent part, even if it looks like a constant with respect to $y$.

Worked Examples of Carrier Terms

In each example, the goal is to identify the parameter-free factor without losing any part of the original probability or density.

Worked Example: A Binomial Coefficient

Let $Y\sim\operatorname{Binomial}(5,p)$, with $0<p<1$. For $y\in\{0,1,2,3,4,5\}$, its probability mass function is $$ P(Y=y)=\binom{5}{y}p^y(1-p)^{5-y}. $$ Use counting measure on $\{0,1,2,3,4,5\}$, so the mass function is a density relative to that measure.

Define the natural parameter $\theta=\log\{p/(1-p)\}$. Solving for $p$ gives $p=e^\theta/(1+e^\theta)$ and $1-p=1/(1+e^\theta)$. Substituting, $$ P(Y=y)=\binom{5}{y}\exp\{y\theta-5\log(1+e^\theta)\}. $$ Thus $T(y)=y$, $A(\theta)=5\log(1+e^\theta)$, and $h(y)=\binom{5}{y}$ on the stated support. The binomial coefficient depends on the observed count and the fixed trial count, but not on $p$ or $\theta$, so it belongs in the carrier.

For a numerical check, take $p=0.3$ and $y=2$. Directly, $$ P(Y=2)=\binom{5}{2}(0.3)^2(0.7)^3=10(0.09)(0.343)=0.3087. $$ Here $\theta=\log(3/7)$, and the exponential-family expression gives $$ 10\exp\{2\log(3/7)-5\log(10/7)\} =10(3/7)^2(7/10)^5 =0.3087. $$ The coefficient is not discarded: it remains in $h(y)$, so the probability is unchanged.

Worked Example: The Normal-Density Constant

Suppose $Y\sim N(\mu,4)$, where the variance $4$ is fixed and $\mu$ varies. The density with respect to Lebesgue measure is $$ f_\mu(y)=\frac{1}{\sqrt{8\pi}}\exp\left\{-\frac{(y-\mu)^2}{8}\right\}. $$ Expanding the square gives $$ -\frac{(y-\mu)^2}{8}=-\frac{y^2}{8}+\frac{\mu y}{4}-\frac{\mu^2}{8}. $$ Set $\theta=\mu/4$, so $\mu=4\theta$. Then $\mu y/4=\theta y$ and $\mu^2/8=2\theta^2$. Therefore $$ f_\mu(y)=\underbrace{\frac{1}{\sqrt{8\pi}}\exp(-y^2/8)}_{h(y)} \exp\{\theta y-2\theta^2\}. $$ The carrier is $h(y)=(8\pi)^{-1/2}\exp(-y^2/8)$, while $T(y)=y$ and $A(\theta)=2\theta^2$. In particular, the normalizing constant $1/\sqrt{8\pi}$ is part of the carrier because the variance is fixed and it does not depend on $\mu$.

At $\mu=2$ and $y=1.5$, $\theta=0.5$. The carrier value and kernel are $$ h(1.5)=\frac{1}{\sqrt{8\pi}}\exp(-2.25/8)\approx 0.150568, \qquad \exp\{(0.5)(1.5)-2(0.5)^2\}=\exp(0.25)\approx 1.284025. $$ Their product is $0.150568\times1.284025\approx0.1933$. Direct evaluation of the original density gives $$ \frac{1}{\sqrt{8\pi}}\exp\left\{-\frac{(1.5-2)^2}{8}\right\} =\frac{1}{\sqrt{8\pi}}\exp(-0.03125) \approx 0.1933, $$ as required.

Worked Example: A Fixed Discrete Support

Let $Y$ have a geometric distribution on $\{0,1,2,\ldots\}$, with $$ P(Y=y)=p(1-p)^y,\qquad 0<p<1. $$ Set $\theta=\log(1-p)$, which ranges over $\theta<0$. Then $p=1-e^\theta$, and $$ P(Y=y)=(1-e^\theta)e^{\theta y} =\mathbf{1}_{\{0,1,2,\ldots\}}(y)\exp\{\theta y-A(\theta)\}, $$ where $A(\theta)=-\log(1-e^\theta)$. The carrier is the indicator of the nonnegative integers, relative to counting measure on the integers. It is parameter-free because the support does not change with $p$.

For example, if $p=0.25$ and $y=2$, then $\theta=\log(0.75)$ and $A(\theta)=-\log(0.25)=\log 4$. The representation gives $$ \exp\{2\log(0.75)-\log 4\} =(0.75)^2(0.25)=0.140625, $$ which agrees with $p(1-p)^2=0.25(0.75)^2=0.140625$. The indicator contributes one at $y=2$; outside the support it contributes zero.

What the Carrier Does in a Likelihood

A carrier term is not an optional decoration. It is part of the density or mass function, and it is needed to preserve the correct probabilities and density values. Its special property is that it does not vary with the parameter. The following result makes the consequence precise for independent observations.

Result: Suppose $Y_1,\ldots,Y_n$ are independent and have the common density $f_\theta(y)=h(y)\exp\{\theta T(y)-A(\theta)\}$ with respect to a common reference measure. For observed values with $h(Y_i)>0$, the log-likelihood is $$ \ell(\theta)=\sum_{i=1}^n\log h(Y_i)+\theta\sum_{i=1}^nT(Y_i)-nA(\theta). $$ Consequently, the carrier terms cancel from likelihood ratios between parameter values. If $A$ is differentiable, they also make no contribution to the score.

To see this, independence gives the product likelihood $$ L(\theta)=\prod_{i=1}^n h(Y_i)\exp\{\theta T(Y_i)-A(\theta)\}. $$ Taking its logarithm and distributing the sum gives the displayed expression for $\ell(\theta)$. For two parameter values $\theta_1,\theta_0$, subtracting their log-likelihoods cancels $\sum_i\log h(Y_i)$, leaving $$ \ell(\theta_1)-\ell(\theta_0) =(\theta_1-\theta_0)\sum_{i=1}^n T(Y_i) -n\{A(\theta_1)-A(\theta_0)\}. $$ Exponentiating gives the likelihood ratio. If $A$ is differentiable, differentiating the finite expression for $\ell$ gives $$ \frac{d\ell(\theta)}{d\theta}=\sum_{i=1}^n T(Y_i)-nA'(\theta), $$ because the observed carrier terms are constant in $\theta$. These cancellations require a common parameter-free carrier and support; they do not justify dropping factors that depend on the parameter. $\square$

Carrier Choices and Their Limits

The split between $h$ and $A$ is not always unique. For any fixed constant $c>0$, replacing $h(y)$ by $c\,h(y)$ and $A(\theta)$ by $A(\theta)+\log c$ leaves the density unchanged: $$ c\,h(y)\exp\{\theta T(y)-A(\theta)-\log c\} =h(y)\exp\{\theta T(y)-A(\theta)\}. $$ A convention for the carrier is therefore useful when comparing representations. The natural practical choice is to place all factors that depend on $y$ but not on $\theta$ in $h(y)$, and then determine $A(\theta)$ so the distribution integrates or sums to one.

The separation also relies on parameter-free support. In the geometric example, the indicator is fixed as $\theta$ changes. If the support itself changes with the parameter, its indicator is parameter-dependent and cannot simply be treated as an ordinary carrier term that cancels from likelihood ratios. Similarly, a factor that appears constant as a function of $y$ may still depend on $\theta$; it must then be retained in the parameter-dependent part.

Common Mistake

Wrong reasoning: “The binomial coefficient and normal-density constant do not involve the parameter, so I can remove them from the probability model.”
Correction: A factor can be omitted when maximizing a likelihood over the parameter only if it is constant in that parameter and the support is fixed. It must still be included in the probability mass function or density. In the binomial example, omitting $\binom{5}{y}$ changes the probability assigned to each count; in the normal example, omitting $1/\sqrt{8\pi}$ changes the density and its integral.

The key distinction is between not affecting parameter comparisons and not being part of the model. A carrier has the first property, not the second.

Key takeaway: Relative to a chosen reference measure, put every parameter-free factor—including coefficients, density constants, and fixed-support indicators—in the carrier $h(y)$. Keep parameter-dependent factors in the exponential kernel and choose $A(\theta)$ to ensure normalization.

Check Your Understanding

For each question, identify which factors depend on the parameter and explain your reasoning.

In the binomial example, which parts of the mass function belong to $h(y)$, $T(y)$, and $A(\theta)$?
Why does the normalizing constant in the normal example belong to the carrier when the variance is fixed?
What role does the support indicator play in the geometric example, and why is it parameter-free?
For independent observations from a common exponential-family model, which carrier terms disappear from a likelihood ratio, and under what condition?
Suppose a factor is constant in $y$ but depends on $\theta$. Should it be included in the carrier? Explain.