Joint Laws of Mixed Independent Variables

Why Mixed Joint Laws Need More Than a Density

When two random variables are independent, their joint law is determined by their marginal laws. If both variables have probability mass functions, multiplying the masses gives the joint probabilities; if both are absolutely continuous, multiplying the densities gives a joint density. Mixed marginals require a more careful description: their joint law may place probability at points, along lines, and across two-dimensional regions. These parts are all accounted for by a product of probability measures, but they cannot in general be represented by one ordinary density with respect to area.

The distinction matters whenever a variable can take an exact value with positive probability as well as values from a continuous distribution. For instance, a measurement might be exactly zero for some observations and continuously distributed when it is positive. Independence still gives a simple rule, but the rule must be applied to each component of the marginal laws.

Product Laws and Mixed Marginals

Definition: The joint law of a pair \((X,Y)\) is the probability measure \(P_{(X,Y)}\) on \(\mathbb{R}^2\) defined by \(P_{(X,Y)}(A)=P((X,Y)\in A)\) for Borel sets \(A\subseteq\mathbb{R}^2\). The variables \(X\) and \(Y\) are independent if \(P(X\in B,Y\in C)=P(X\in B)P(Y\in C)\) for every pair of Borel sets \(B,C\subseteq\mathbb{R}\). Equivalently, their joint law is the product measure \(P_X\otimes P_Y\).

A discrete law is a countable collection of point masses. An absolutely continuous law has a density with respect to Lebesgue measure. For this tutorial, a mixed law means a law made up of point masses and an absolutely continuous component. If the atoms are \(a_i\), with probabilities \(p_i\), and the continuous component has density \(g\), then the law can be written

P_X(B)=\sum_i p_i\,\mathbf{1}_{\{a_i\in B\}}+\int_B g(x)\,dx, \qquad \sum_i p_i+\int_{\mathbb{R}}g(x)\,dx=1.

Here \(B\) is any Borel subset of \(\mathbb{R}\), and \(\mathbf{1}_{\{a_i\in B\}}\) equals 1 when \(a_i\in B\) and 0 otherwise. The density \(g\) need not integrate to 1 by itself: its integral is the probability assigned to the absolutely continuous component. The all-discrete and all-absolutely-continuous cases are special cases of this representation.

Independence says to multiply the entire marginal laws, not just their densities. In a mixed case, expanding that product gives components of different dimensions: atom-by-atom terms are point masses; atom-by-density terms are supported on vertical or horizontal lines; density-by-density terms have a density with respect to area.

Expanding the Product Measure

Theorem: Suppose \(X\) and \(Y\) are independent. Suppose their marginal laws have the mixed forms \(P_X=\sum_i p_i\delta_{a_i}+g(x)\,dx\) and \(P_Y=\sum_j q_j\delta_{b_j}+h(y)\,dy\), where \(\delta_a\) is a unit point mass at \(a\), the weights are nonnegative, and the mass and density terms together each have total probability 1. Then \[ P_{(X,Y)} = \sum_{i,j}p_iq_j\,\delta_{(a_i,b_j)} + \sum_i p_i\,\delta_{a_i}\otimes h(y)\,dy + \sum_j q_j\,g(x)\,dx\otimes\delta_{b_j} + g(x)h(y)\,dx\,dy. \] The sums may be finite or countable. Each term describes a component of the joint probability measure; the last term is the part that has a density with respect to two-dimensional Lebesgue measure.

Proof. By independence, \(P_{(X,Y)}=P_X\otimes P_Y\). Substitute the two marginal decompositions. Distributing the product gives four terms: the product of the two atomic parts, the \(X\)-atomic part times the \(Y\)-density part, the \(X\)-density part times the \(Y\)-atomic part, and the product of the two density parts. These are precisely the four displayed terms. For countably many atoms, the component measures are nonnegative; their sum defines the product measure, and countable additivity justifies the countable sums. Their total mass is the product of the total marginal masses, namely \(1\cdot1=1\). Therefore the expansion accounts for the entire joint law. \(\square\)

For example, the first mixed term assigns probability to points with \(X=a_i\) while \(Y\) ranges continuously. It is not a two-dimensional density: it is supported on the vertical line \(x=a_i\). Similarly, the next term is supported on horizontal lines. A joint density with respect to area exists for the whole joint law when both marginals are absolutely continuous; when atoms are present, the full law generally has singular components that such a density cannot represent.

Worked Example: A Bernoulli Variable and a Mixed Variable

Let \(X\sim\operatorname{Bernoulli}(0.3)\). Independently, let \(Y\) equal 0 with probability \(0.4\), and otherwise have a uniform distribution on \((0,2)\). Find the point masses and the continuous part of the joint law.

The marginal probabilities for \(X\) are \(P(X=0)=0.7\) and \(P(X=1)=0.3\). The atom of \(Y\) at 0 has probability 0.4. Its continuous component has density \(0.6/2=0.3\) on \((0,2)\), since that component has total probability 0.6.

Independence gives two point masses: \(P(X=0,Y=0)=0.7(0.4)=0.28\) and \(P(X=1,Y=0)=0.3(0.4)=0.12\). For each fixed value of \(X\), the continuous component of \(Y\) contributes probability along that vertical line. For any Borel set \(B\subseteq\mathbb{R}\),

\begin{aligned} P(X=0,Y\in B) &=0.7\left(0.4\,\mathbf{1}_{\{0\in B\}}+ 0.3\int_{B\cap(0,2)}dy\right),\\ P(X=1,Y\in B) &=0.3\left(0.4\,\mathbf{1}_{\{0\in B\}}+ 0.3\int_{B\cap(0,2)}dy\right). \end{aligned}

The continuous probabilities across both vertical lines total \(0.7(0.6)+0.3(0.6)=0.42+0.18=0.60\). Including the point masses gives \(0.28+0.12+0.60=1\), as required. There is no density with respect to area for this joint law: \(X\) is discrete, so the pair is concentrated on the two vertical lines \(x=0\) and \(x=1\).

Worked Example: Two Mixed Marginals Produce Four Components

Let \(X\) equal \(-1\) with probability 0.25, and with the remaining probability 0.75 have an exponential distribution with rate 1 on \(x>0\). Independently, let \(Y\) equal 0 with probability 0.4, and with probability 0.6 be uniform on \((0,1)\). Describe all components of the joint law and check their total probability.

The \(X\) density component is \(0.75e^{-x}\) for \(x>0\), and the \(Y\) density component is \(0.6\) on \((0,1)\). Multiplying component weights gives a point mass at \((-1,0)\) of size \(0.25(0.4)=0.10\).

When \(X=-1\) and \(Y\) is in its continuous component, the probability on the vertical line \(x=-1\) has density \(0.25(0.6)=0.15\) with respect to \(dy\), for \(0<y<1\). When \(Y=0\) and \(X\) is in its continuous component, the probability on the horizontal line \(y=0\) has density \(0.75(0.4)e^{-x}=0.30e^{-x}\), for \(x>0\).

When both variables are in their continuous components, the joint density with respect to area is \(0.75e^{-x}\cdot0.6=0.45e^{-x}\) on \(x>0,\ 0<y<1\). The four component probabilities are \(0.10\), \(0.25(0.6)=0.15\), \(0.75(0.4)=0.30\), and \(0.75(0.6)=0.45\). Their sum is \(0.10+0.15+0.30+0.45=1\). Thus the point, two line-supported, and area-supported components together form a probability law.

Rectangle Probabilities and Marginal Checks

Result: If \(X\) and \(Y\) are independent with marginal distribution functions \(F_X\) and \(F_Y\), then their joint distribution function satisfies \(F_{X,Y}(x,y)=F_X(x)F_Y(y)\) for all real \(x,y\). This identity applies whether the marginals are discrete, absolutely continuous, or mixed.

Proof. By definition, \(F_{X,Y}(x,y)=P(X\le x,Y\le y)\). The events \(\{X\le x\}\) and \(\{Y\le y\}\) are independent because \(X\) and \(Y\) are independent. Hence \(P(X\le x,Y\le y)=P(X\le x)P(Y\le y)=F_X(x)F_Y(y)\). No density is needed for this argument. \(\square\)

The same factorization applies to any rectangle \(B\times C\) with Borel sides: its probability is \(P_X(B)P_Y(C)\). Rectangles are particularly useful for checking a proposed joint law. Summing or integrating a correctly constructed product law over all values of one coordinate must recover the other coordinate's marginal law.

Worked Example: A Discrete-Continuous Rectangle Probability

Let \(X\) have the mixed law \(P(X=0)=0.2\), with its remaining probability 0.8 uniformly distributed on \((0,1)\). Independently, let \(Z\) take values 1, 2, and 3 with probabilities 0.2, 0.5, and 0.3. Find \(P(X>0.5,\ Z=2)\), and identify the joint components.

The event \(X>0.5\) excludes the atom at zero. In the uniform component, the interval \((0.5,1)\) has length 0.5, so \(P(X>0.5)=0.8(0.5)=0.4\). Independence therefore gives \(P(X>0.5,Z=2)=0.4(0.5)=0.20\).

For each \(z\in\{1,2,3\}\), the joint law has an atom at \((0,z)\) of size \(0.2P(Z=z)\). It also has a continuous component along the horizontal line \(Z=z\), with density \(0.8P(Z=z)\) over \(0<x<1\). For \(z=2\), the atom has mass \(0.2(0.5)=0.10\), and the line density is \(0.8(0.5)=0.40\). Integrating that density over \((0.5,1)\) gives \(0.40(0.5)=0.20\), confirming the event calculation.

Common Mistakes and Why the Distinction Matters

Multiplying only the continuous densities. This omits any mass at atoms. Under independence, multiply every component of one marginal by every component of the other.
Calling line-supported probabilities a joint density. A density with respect to area cannot represent positive probability concentrated on a line or a point. Describe those components separately as measures.
Assuming atomless means absolutely continuous. A marginal can have no atoms and still have no density, as with a singular-continuous law such as the Cantor distribution. If both marginals are absolutely continuous, independence gives the product density. Merely having atomless marginals does not justify that conclusion.
Using the product rule without independence. The identity \(P(X\in B,Y\in C)=P(X\in B)P(Y\in C)\) is guaranteed by independence, not by the marginal distributions alone.

The product-measure viewpoint is useful because it handles all the strata without forcing them into a single density formula. It also clarifies exactly which assumptions a later calculation needs: probabilities of rectangles factor under independence, while a product density requires both marginals to be absolutely continuous. If a marginal has additional singular-continuous mass, it must also be included as a component of its probability measure; the atom-plus-density expansion alone is not exhaustive in that case.

Check Your Understanding

Use the product-law approach to answer these questions.

If \(X\) has an atom at \(a\) and a continuous component, while \(Y\) is absolutely continuous and independent of \(X\), where is the atom-by-density component of the joint law supported?
Suppose independent \(X\) and \(Y\) each have an atom of probability 0.3 and an absolutely continuous component of probability 0.7. What are the probabilities of the atom-by-atom component and the density-by-density component?
Why does the factorization of the joint distribution function hold even if one marginal has atoms?
Does an atomless marginal necessarily have a density with respect to Lebesgue measure? Explain what this means for a product-density claim.
For two independent mixed variables, name the four types of terms obtained by expanding their product laws.