Expectations from Joint Laws

From a Random Vector to an Integral

A random vector can have several coordinates whose values are related. To find the expectation of a function of those coordinates, it is often more useful to work with the vector’s joint law than to work directly on the original probability space. The joint law records the probabilities assigned to measurable sets of possible coordinate values; integrating against it gives a precise way to compute the expectation.

There is an important qualification. A joint law is a measure on a product space, but it need not be a product measure. Fubini’s theorem gives iterated integrals when the measure and integrand satisfy its hypotheses—for example, when the joint law has a density with respect to a product of reference measures. The joint law alone does not justify treating the coordinates as if they were independent.

Joint Laws and Expectations

Let \((\Omega,\mathcal F,P)\) be a probability space, let \((S,\mathcal S)\) and \((T,\mathcal T)\) be measurable spaces, and let \(Z=(X,Y):\Omega\to S\times T\) be a measurable random vector. The product space is equipped with the product sigma-algebra \(\mathcal S\otimes\mathcal T\).

Definition: The joint law of \(Z\) is the probability measure \(\nu\) on \((S\times T,\mathcal S\otimes\mathcal T)\) defined, for every measurable \(A\subseteq S\times T\), by

\nu(A)=P\bigl(Z^{-1}(A)\bigr).

Equivalently, \(\nu=P\circ Z^{-1}\).

For a measurable function \(g:S\times T\to[0,\infty]\), its expectation as a function of the random vector is the possibly infinite quantity \(E[g(X,Y)]\). For a real-valued \(g\), this expectation is finite when \(E[|g(X,Y)|]<\infty\); in that case \(g(X,Y)\) is integrable.

Theorem: For every measurable \(g:S\times T\to[0,\infty]\),

E[g(X,Y)]=\int_{S\times T}g(x,y)\,\nu(d(x,y)).

The identity also holds for real-valued measurable \(g\) when \(E[|g(X,Y)|]<\infty\).

Proof. First take \(g=\mathbf 1_A\), where \(A\) is measurable in \(S\times T\). By the definition of the joint law,

E[\mathbf 1_A(X,Y)] =P(Z^{-1}(A)) =\nu(A) =\int_{S\times T}\mathbf 1_A\,d\nu.

Linearity gives the identity for nonnegative simple functions. For a general nonnegative measurable \(g\), choose nonnegative simple functions \(g_n\) increasing pointwise to \(g\). The monotone convergence theorem gives convergence of the expectations and of the integrals under \(\nu\), so the identity follows. For an integrable real-valued \(g\), apply the nonnegative result to its positive and negative parts. Both integrals are finite because \(E[|g(X,Y)|]<\infty\), and subtracting them proves the claim. \(\square\)

This result is sometimes called the integration formula for a pushforward measure. It allows us to replace an expectation on \(\Omega\) with an integral over the values of the random vector. The integrand remains the function \(g\); the measure changes from \(P\) to the joint law \(\nu\).

When the Integral Can Be Iterated

Suppose the joint law has a density \(p\) with respect to a product measure \(\mu\otimes\lambda\), where \(\mu\) on \(S\) and \(\lambda\) on \(T\) are sigma-finite. This means that, for measurable \(A\subseteq S\times T\),

\nu(A)=\int_A p(x,y)\,(\mu\otimes\lambda)(d(x,y)).

Since \(\nu\) is a probability measure, \(p\geq 0\) and \(\int p\,d(\mu\otimes\lambda)=1\). The expectation formula then becomes an integral of \(g(x,y)p(x,y)\) against the product reference measure.

Theorem: Under the density assumptions above, if \(g\geq 0\) is measurable, then

E[g(X,Y)] =\int_S\left(\int_T g(x,y)p(x,y)\,\lambda(dy)\right)\mu(dx) =\int_T\left(\int_S g(x,y)p(x,y)\,\mu(dx)\right)\lambda(dy).

The equalities also hold for real-valued measurable \(g\) if \(\int_{S\times T}|g|p\,d(\mu\otimes\lambda)<\infty\).

Proof. The joint-law formula and the density assumption give \(E[g(X,Y)]=\int_{S\times T}gp\,d(\mu\otimes\lambda)\). If \(g\geq0\), then \(gp\geq0\), so Tonelli’s theorem gives both iterated integrals, including when their common value is infinite. If \(g\) is real-valued and \(\int |g|p\,d(\mu\otimes\lambda)<\infty\), the integrand \(gp\) is absolutely integrable. Fubini’s theorem therefore gives both iterated integrals and shows they are finite. The stated integrability condition is exactly \(E[|g(X,Y)|]<\infty\), by the joint-law formula applied to \(|g|\). \(\square\)

For a finite or countable joint distribution, the integral against \(\nu\) is a weighted sum over the possible pairs. Iterating a sum and an integral may also be justified by Tonelli for nonnegative terms or by Fubini for absolutely summable terms. The same principle applies: check the measure and integrability conditions before changing the order.

Worked Examples

Worked Example: A Finite Joint Distribution

Suppose \((X,Y)\) takes values in \(\{0,1\}^2\), with joint probabilities \(P((X,Y)=(0,0))=0.2\), \(P((X,Y)=(0,1))=0.3\), \(P((X,Y)=(1,0))=0.1\), and \(P((X,Y)=(1,1))=0.4\). These probabilities are nonnegative and sum to \(1\), as required. Compute \(E[XY]\) and \(E[X+2Y]\).

For \(XY\), the product is zero at the first three possible pairs and one at \((1,1)\). Thus

$$ E[XY]=0(0.2)+0(0.3)+0(0.1)+1(0.4)=0.4. $$

For \(X+2Y\), the values at \((0,0),(0,1),(1,0),(1,1)\) are respectively \(0,2,1,3\). Therefore

$$ E[X+2Y]=0(0.2)+2(0.3)+1(0.1)+3(0.4)=1.7. $$

Each computation uses the joint probabilities assigned to pairs, not separate calculations based only on the coordinate values.

Worked Example: Integrating a Joint Density

On the unit square, let the joint density with respect to two-dimensional Lebesgue measure be \(p(x,y)=1+(2x-1)(2y-1)\) for \(0\leq x,y\leq1\), and zero elsewhere. Because both factors \(2x-1\) and \(2y-1\) lie in \([-1,1]\), \(p\geq0\). Also, \(\int_0^1(2x-1)\,dx=0\), so integrating \(p\) over the square gives \(1\). Compute \(E[XY]\).

The density theorem and Fubini apply: \(xy p(x,y)\) is bounded on a set of finite area, hence integrable. Integrate first in \(y\):

\begin{aligned} E[XY] &=\int_0^1\int_0^1 xy\bigl[1+(2x-1)(2y-1)\bigr]\,dy\,dx\\ &=\int_0^1 x\left[\frac12+\frac{2x-1}{6}\right]dx\\ &=\frac14+\frac16\left(\frac23-\frac12\right) =\frac14+\frac1{36} =\frac5{18}. \end{aligned}

Here the inner integral uses \(\int_0^1y\,dy=1/2\) and \(\int_0^1 y(2y-1)\,dy=2/3-1/2=1/6\). The iterated calculation is valid because the joint law has a density and the integrand is absolutely integrable.

Worked Example: An Expectation from Countably Many Atoms

Let \(N\) take values in \(\mathbb N\) with \(P(N=n)=2^{-n}\), and define \(X=N\), \(Y=0\). The probabilities sum to \(1\), since \(\sum_{n=1}^{\infty}2^{-n}=1\). For \(g(x,y)=x+y\), the joint-law formula gives

E[g(X,Y)]=\sum_{n=1}^{\infty}n\,2^{-n}.

To evaluate the sum without assuming its value, write \(n=\sum_{k=1}^{n}1\). All terms are nonnegative, so Tonelli’s theorem permits exchanging the two sums:

\begin{aligned} \sum_{n=1}^{\infty}n\,2^{-n} &=\sum_{n=1}^{\infty}\sum_{k=1}^{n}2^{-n} =\sum_{k=1}^{\infty}\sum_{n=k}^{\infty}2^{-n}\\ &=\sum_{k=1}^{\infty}2^{1-k} =2. \end{aligned}

Thus \(E[X+Y]=2\). The use of Tonelli is justified by nonnegativity; no prior assumption that this expectation is finite was needed.

A Common Pitfall: Confusing Joint and Product Laws

The joint law describes the probability of pairs. Its marginal laws describe the separate coordinates, but knowing those two marginal laws does not generally determine the joint law. In particular, one cannot replace an integral against \(\nu\) by an iterated integral against two marginal laws unless additional assumptions justify doing so. The density example used a product reference measure for integration; that fact alone did not make its joint law a product measure.

For any proposed computation, first identify the measure being integrated against. If the joint law is discrete, sum using its joint masses. If a density is given, integrate the product of the function and that density. Then check nonnegativity for Tonelli or absolute integrability for Fubini before exchanging the order of integration.

Check Your Understanding

Use the definitions and conditions in this tutorial to answer the following questions.

How is the joint law of a measurable random vector defined from the probability measure on its original space?
State the integration identity that expresses \(E[g(X,Y)]\) using the joint law, and give its integrability condition for real-valued \(g\).
What condition permits applying Tonelli to an iterated integral involving a density?
In the density example, why was exchanging the order of integration valid?
Why can a joint law not generally be replaced by an iterated integral against the two marginal laws?

What You'll Learn