Carrier Measures and Equivalent Representations

Why the Carrier Matters

A probability law is not a density by itself: it is a measure that assigns probabilities to events. A density is a way of expressing that law relative to a chosen reference measure, called a carrier. The same law can therefore have different densities—and different reference factors—in different representations. Keeping track of the carrier is essential when comparing exponential-family forms.

In the canonical form established in the earlier material on exponential families, a law can be written, relative to a carrier measure $\mu$, as an exponential factor involving a statistic and a parameter, multiplied by a reference factor $h$. The carrier and $h$ work together: changing one generally changes the other. The probability law is unchanged only when the new density and new carrier give the same probabilities to every measurable event.

Definition: Let $(\mathcal X,\mathcal A)$ be a measurable space and let $\mu$ be a measure on it. A probability measure $P_\theta$ has density $f_\theta$ with respect to $\mu$ if $P_\theta(B)=\int_B f_\theta\,d\mu$ for every $B\in\mathcal A$. In a common-carrier exponential-family representation, $$ \frac{dP_\theta}{d\mu}(x) =h(x)\exp\{\eta(\theta)^{T}T(x)-A(\theta)\}, $$ where the displayed density is nonnegative and integrates to one for each parameter value under consideration. The carrier $\mu$ and the parameter-independent reference factor $h$ are part of the representation, not separate properties of the probability law.

Counting measure on a countable sample space assigns mass one to each point, so a probability mass function is a density relative to counting measure. Lebesgue measure on the real line is the familiar reference for many continuous densities. Other choices are possible. A weighted counting measure, for instance, assigns a specified positive weight to each point; a law’s density relative to that measure is its probability at the point divided by the weight.

Changing Between Equivalent Carriers

Two measures $\mu$ and $\nu$ are equivalent if they have the same null sets: $\mu(B)=0$ exactly when $\nu(B)=0$. If equivalent carriers are sigma-finite, the Radon–Nikodym theorem gives a derivative $d\mu/d\nu$. Under equivalence this derivative is positive and finite $\nu$-almost everywhere. Consequently, a density can be converted from one carrier to the other by multiplying by this derivative.

Theorem: Suppose $\mu$ and $\nu$ are equivalent sigma-finite measures on $(\mathcal X,\mathcal A)$, and $P_\theta$ has density $f_\theta$ with respect to $\mu$. Then $P_\theta$ has density $$ \frac{dP_\theta}{d\nu}(x) =f_\theta(x)\frac{d\mu}{d\nu}(x) $$ with respect to $\nu$, up to $\nu$-null sets. In particular, if $f_\theta=h\exp\{\eta(\theta)^TT-A(\theta)\}$, the new reference factor is $h_\nu=h(d\mu/d\nu)$, with the same parameter-dependent exponential factor.

Proof. Write $r=d\mu/d\nu$, which exists because both measures are sigma-finite. For any $B\in\mathcal A$, the change-of-measure identity for nonnegative measurable functions gives $$ P_\theta(B)=\int_B f_\theta\,d\mu =\int_B f_\theta r\,d\nu. $$ Thus $f_\theta r$ is a density of $P_\theta$ with respect to $\nu$. Densities with respect to the same measure are unique almost everywhere, so this is the stated density. Since $r$ is determined by the two carriers and not by $\theta$, multiplying $h$ by $r$ gives a parameter-independent reference factor. $\square$

The sigma-finiteness condition matters. For example, if both $\mu$ and $\nu$ are sigma-finite, then $\rho=\mu+\nu$ is sigma-finite: take countable finite-measure covers for $\mu$ and $\nu$, and their pairwise intersections form a countable cover on which $\rho$ is finite. Without the hypotheses needed for a Radon–Nikodym derivative, that argument cannot simply be invoked. In particular, adding two arbitrary measures does not guarantee a sigma-finite measure.

Worked Changes of Carrier

Worked Example: Bernoulli Under Weighted Counting Measure

Let $X\sim\operatorname{Bernoulli}(p)$, where $0<p<1$, on $\mathcal X=\{0,1\}$. With counting measure $\mu$, the density is $p^x(1-p)^{1-x}$. Set $\eta=\log\{p/(1-p)\}$; then $p=e^\eta/(1+e^\eta)$, and the density has canonical form $$ \frac{dP_p}{d\mu}(x) =\exp\{\eta x-\log(1+e^\eta)\}. $$

Now define a weighted counting measure $\nu$ by $\nu(\{0\})=2$ and $\nu(\{1\})=3$. Since $\mu(\{x\})=1$, the derivative $d\mu/d\nu$ equals $1/2$ at zero and $1/3$ at one. The new density is therefore $$ \frac{dP_p}{d\nu}(x) =\frac{1}{w_x}\exp\{\eta x-\log(1+e^\eta)\}, \qquad w_0=2,\quad w_1=3. $$

The resulting probabilities are unchanged: at zero, the density times the carrier mass is $\frac12(1+e^\eta)^{-1}\cdot2=(1+e^\eta)^{-1}=1-p$. At one it is $\frac13 e^\eta(1+e^\eta)^{-1}\cdot3=p$. Thus $h_\nu(0)=1/2$ and $h_\nu(1)=1/3$; neither value depends on $p$.

Worked Example: Poisson Under a Probability Carrier

Let $X\sim\operatorname{Poisson}(\lambda)$, with $\lambda>0$, on the nonnegative integers. Under counting measure $\mu$, its density is $e^{-\lambda}\lambda^k/k!$. Write $\eta=\log\lambda$, so its canonical form is $\exp\{\eta k-e^\eta\}/k!$.

Choose the probability measure $\nu$ on the nonnegative integers with $\nu(\{k\})=2^{-(k+1)}$. Because $\mu(\{k\})=1$, $d\mu/d\nu(k)=2^{k+1}$. Hence $$ \frac{dP_\lambda}{d\nu}(k) =\frac{2^{k+1}}{k!}\exp\{\eta k-e^\eta\}. $$

To verify the law, multiply this density by $\nu(\{k\})$: the probability at $k$ is $2^{k+1}e^{-\lambda}\lambda^k/k!\cdot2^{-(k+1)}=e^{-\lambda}\lambda^k/k!$, as required. The density also integrates to one under $\nu$, because $$ \sum_{k=0}^\infty \frac{2^{k+1}}{k!}e^{-\lambda}\lambda^k\,2^{-(k+1)} =e^{-\lambda}\sum_{k=0}^\infty\frac{\lambda^k}{k!} =e^{-\lambda}e^\lambda=1. $$ The changed reference factor $2^{k+1}/k!$ is parameter-independent, even though it differs from the original $1/k!$.

Worked Example: Exponential Law Under a Weighted Lebesgue Carrier

Let $X$ have an exponential distribution with rate $\lambda>0$, on $(0,\infty)$. Relative to Lebesgue measure $\mu$, its density is $\lambda e^{-\lambda x}$. With $\eta=-\lambda$ and $A(\theta)=-\log\lambda$, this is $\exp\{\eta x-A(\theta)\}$.

Take $\nu(dx)=e^{-x}\,dx$ on $(0,\infty)$. This is a finite measure, since $\int_0^\infty e^{-x}\,dx=1$, and is equivalent to Lebesgue measure there. Its derivative satisfies $d\mu/d\nu=e^x$. The density relative to $\nu$ becomes $$ \frac{dP_\lambda}{d\nu}(x) =\lambda e^{-(\lambda-1)x} =\exp\{(\eta+1)x-A(\theta)\}. $$

For a direct check, integrate this density against $\nu$: $$ \int_0^\infty \lambda e^{-(\lambda-1)x}e^{-x}\,dx =\lambda\int_0^\infty e^{-\lambda x}\,dx =\lambda\cdot\frac{1}{\lambda}=1. $$ The factor $e^x$ moved into the density changes its expression, but the product of density and carrier still gives the original probability law.

Absorbing the Reference Factor into the Carrier

When $h$ is nonnegative, measurable, and finite $\mu$-almost everywhere, one can define a measure $\lambda$ by $\lambda(B)=\int_B h\,d\mu$. The exponential-family representation then uses a density with respect to $\lambda$, rather than a separate $h$ factor. This is a reorganization of the representation: it does not change the probability law, and it is valid when the displayed density is interpreted on the part of the space where $h$ is positive.

Result: If $P_\theta$ has density $h(x)\exp\{\eta(\theta)^TT(x)-A(\theta)\}$ with respect to $\mu$, and $\lambda(B)=\int_B h\,d\mu$, then $$ P_\theta(B) =\int_B \exp\{\eta(\theta)^TT(x)-A(\theta)\}\,d\lambda(x) $$ for every measurable $B$.

Proof. By the definition of $\lambda$, for every nonnegative measurable $g$, $\int g\,d\lambda=\int gh\,d\mu$. Apply this identity on $B$ with $g(x)=\exp\{\eta(\theta)^TT(x)-A(\theta)\}$. The result is $$ \int_B \exp\{\eta(\theta)^TT-A(\theta)\}\,d\lambda =\int_B h\exp\{\eta(\theta)^TT-A(\theta)\}\,d\mu =P_\theta(B). $$ Thus the probability assigned to each event is unchanged. $\square$

Worked Example: A Mixed Law Needs a Mixed Carrier

Suppose a random variable has probability $q$ at zero and, with probability $1-q$, has an exponential distribution with rate $\lambda$, where $0<q<1$ and $\lambda>0$. A suitable carrier on $[0,\infty)$ is $\mu=\delta_0+\operatorname{Leb}_{(0,\infty)}$, the sum of unit mass at zero and Lebesgue measure on the positive half-line.

Relative to this carrier, a density is $q$ at zero and $(1-q)\lambda e^{-\lambda x}$ for $x>0$. Its integral is $$ q+\int_0^\infty(1-q)\lambda e^{-\lambda x}\,dx =q+(1-q)=1. $$ Lebesgue measure alone cannot serve as a carrier for this law: the singleton $\{0\}$ has Lebesgue measure zero but has probability $q>0$. No Lebesgue density can assign positive probability to that singleton. The carrier must account for both the atom and the continuous part.

Common Mistake and Practical Checks

A common mistake is to treat $h(x)$ as an intrinsic property of the distribution and conclude that changing it changes the law. The Bernoulli and Poisson examples show why that conclusion is wrong: the new density must be integrated against the new carrier. In general, it is the product of the density and carrier—not the density alone—that determines event probabilities.

A related mistake is to change carriers without checking their relationship. The simple derivative formula above requires equivalent sigma-finite carriers. When a law has an atom and a proposed carrier gives that atom measure zero, the law is not absolutely continuous with respect to that carrier, so the formula cannot produce a density there. Before rewriting a family, check that the new carrier dominates each law in the family. If a single common carrier is intended, also check that the resulting reference factor can be chosen independently of the parameter.

State the carrier.
Specify the measure with respect to which the density is being written, including its support and any point masses.

Check the change-of-measure conditions.
For the equivalent-carrier formula, verify equivalence and sigma-finiteness before using $d\mu/d\nu$.

Transform and verify.
Multiply the old density by $d\mu/d\nu$, then check that integrating the new density over any event against $\nu$ recovers its original probability.

Key takeaway: A carrier determines the scale against which a density is measured. Under an appropriate change of carrier, the density and its reference factor change together while the probability law remains fixed.

Check Your Understanding

Use the carrier and density together in each answer.

For a probability mass function written relative to counting measure, what happens to its density if each point is assigned a positive carrier weight $w_k$?
State the hypotheses that justify multiplying a density relative to $\mu$ by $d\mu/d\nu$ to obtain a density relative to $\nu$.
In the weighted-counting Bernoulli example, verify the probability at $x=0$ by multiplying the density by $\nu(\{0\})$.
Why can a law with positive probability at zero not have a density with respect to Lebesgue measure alone?
Explain how $h$ can be incorporated into a new measure, and state which probability identity verifies that this leaves the law unchanged.