Why Mixed Laws Need More Than a Density Check
A distribution can place positive probability at particular values and spread its remaining probability continuously over an interval. For such a law, neither a probability mass function alone nor a density alone describes all the probability. Checking independence therefore requires attention to both point events and sets that contain continuous probability.
The relevant comparison has four parts: both variables at atoms, the first at an atom and the second in a continuous set, the reverse combination, and both variables in continuous sets. A dependence hidden in any one of these parts can invalidate independence.
The set \(S_X\) itself need not have probability one: the probability there is the non-atomic remainder of the law. In the criterion below, “continuous sets” means Borel subsets of this atom-free part. This terminology includes the case where the remainder is absolutely continuous with respect to length; the general criterion also works for atomless remainders that are not described by a density.
The Four-Part Independence Criterion
Let \(X\) and \(Y\) be real-valued random variables, with atom sets \(D_X\) and \(D_Y\). Use \(p_x=P(X=x)\) and \(q_y=P(Y=y)\) for their atom probabilities. Independence requires that probabilities factor for every pair of Borel sets. For mixed laws, that requirement can be separated into the following four types of comparisons.
- For every \(x\in D_X\) and \(y\in D_Y\), \(P(X=x,Y=y)=p_xq_y\).
- For every \(x\in D_X\) and every Borel set \(B\subseteq S_Y\), \(P(X=x,Y\in B)=p_xP(Y\in B)\).
- For every \(y\in D_Y\) and every Borel set \(A\subseteq S_X\), \(P(X\in A,Y=y)=P(X\in A)q_y\).
- For every Borel set \(A\subseteq S_X\) and every Borel set \(B\subseteq S_Y\), \(P(X\in A,Y\in B)=P(X\in A)P(Y\in B)\).
The quantification over sets matters. Comparing just the probability of each atom, or just the probability of one chosen interval, does not generally verify a condition that must hold for all relevant sets.
Conversely, assume the four conditions. Take arbitrary Borel sets \(C,D\subseteq\mathbb{R}\). Decompose them into their atomic and atom-free parts: \(C=(C\cap D_X)\cup(C\cap S_X)\) and \(D=(D\cap D_Y)\cup(D\cap S_Y)\). These are disjoint unions. Countable additivity lets us sum the joint probabilities over the four resulting combinations.
For the atomic–atomic combination, condition 1 gives \(\sum_{x\in C\cap D_X}\sum_{y\in D\cap D_Y}p_xq_y =(\sum_{x\in C\cap D_X}p_x)(\sum_{y\in D\cap D_Y}q_y)\). The sums are well-defined, since the atom sets are countable and all terms are nonnegative. Condition 2 gives the product of the atomic probability in \(C\) and the atom-free probability in \(D\); summing it over \(x\in C\cap D_X\) yields that product. Condition 3 gives the corresponding product for the atom-free part of \(C\) and atomic part of \(D\). Condition 4 gives the product for the two atom-free parts.
Adding these four products gives \(P(X\in C,Y\in D)=P(X\in C)P(Y\in D)\), by distributivity and the disjoint decompositions. This holds for every pair of Borel sets \(C,D\), which is the definition of independence. \(\square\)
Working Through the Four Combinations
Worked Example: Probabilities Under Independence
Let \(X\) equal \(0\) with probability \(0.4\), and otherwise have the uniform distribution on \((0,1)\). Let \(Y\) equal \(2\) with probability \(0.3\), and otherwise have the uniform distribution on \((0,2)\). Suppose \(X\) and \(Y\) are independent. Calculate one probability from each of the four combinations.
The continuous component of \(X\) has total probability \(0.6\), and that of \(Y\) has total probability \(0.7\). First, for the atom–atom combination, \(P(X=0,Y=2)=0.4(0.3)=0.12\).
For an atom–continuous set, the interval \([0.5,1]\) has length \(0.5\), so its conditional probability under the uniform component of \(Y\) is \(0.5/2=0.25\). Therefore, \(P(X=0,Y\in[0.5,1])=0.4(0.7)(0.25)=0.07\).
For a continuous–atom set, \([0.2,0.6]\) has length \(0.4\), so its conditional probability under the uniform component of \(X\) is \(0.4/1=0.4\). Thus, \(P(X\in[0.2,0.6],Y=2)=0.6(0.4)(0.3)=0.072\).
Finally, the continuous-component probabilities are \(0.6(0.4)=0.24\) for \(X\in[0.2,0.6]\) and \(0.7(0.5)=0.35\) for \(Y\in[0,1]\). Independence gives \(P(X\in[0.2,0.6],Y\in[0,1])=0.24(0.35)=0.084\).
Each calculation uses both the probability of entering the continuous component and the conditional probability within that component. Omitting either factor would confuse a component’s conditional distribution with its total probability in the mixture.
Worked Example: An Atom–Continuous Mismatch
Suppose the joint law assigns probabilities \(0.4, 0.1, 0.1,\) and \(0.4\) to the following four cases: \((X,Y)=(0,0)\); \(X=0\) with \(Y\) uniform on \((1,2)\); \(Y=0\) with \(X\) uniform on \((1,2)\); and \(X,Y\) independently uniform on \((1,2)\). Calculate the marginals and test independence.
The probability that \(X=0\) is \(0.4+0.1=0.5\). The remaining probability \(0.5\) is continuous and uniform on \((1,2)\): the component with \(Y=0\) contributes \(0.1\) times a uniform \(X\), and the component where both are continuous contributes \(0.4\) times a uniform \(X\). By the same calculation, \(P(Y=0)=0.5\), and the continuous remainder of \(Y\) is also uniform on \((1,2)\).
If \(X\) and \(Y\) were independent, their atom–atom probability would be \(0.5(0.5)=0.25\). The actual probability \(P(X=0,Y=0)=0.4\) already disproves independence.
The atom–continuous check also fails. Conditional on the component \(X=0\) with continuous \(Y\), the probability that \(Y\in(1,1.5)\) is \(0.5\). Hence \(P(X=0,Y\in(1,1.5))=0.1(0.5)=0.05\). Under independence, the probability would instead be \(P(X=0)P(Y\in(1,1.5))=0.5(0.5)(0.5)=0.125\).
The marginals are identical in shape, but their joint law does not factor. Checking only the marginal distributions cannot establish independence.
When Densities Do—and Do Not—Finish the Check
If the part of the joint law on \(S_X\times S_Y\) has a density \(g(x,y)\), and the atom-free marginal parts have densities \(f_X^c(x)\) and \(f_Y^c(y)\), the fourth condition can be checked through a density factorization: it holds exactly when \(g(x,y)=f_X^c(x)f_Y^c(y)\) for almost every \((x,y)\in S_X\times S_Y\). Here “almost every” is with respect to two-dimensional Lebesgue measure. This statement concerns the atom-free joint and marginal parts, not the full mixed laws as though they had densities everywhere.
For the other combinations, define \(h_x(y)\) to be the density of the measure \(P(X=x,Y\in dy)\) on \(S_Y\), and \(k_y(x)\) to be the density of \(P(X\in dx,Y=y)\) on \(S_X\), whenever these measures have densities. The corresponding checks are \(h_x(y)=p_x f_Y^c(y)\) for almost every \(y\), and \(k_y(x)=q_y f_X^c(x)\) for almost every \(x\). The atom–atom check remains the point-mass equality in condition 1. Each comparison corresponds to one of the four parts of the theorem.
Worked Example: Matching Atom Probabilities Is Not Enough
Let the joint law be the following mixture: with probability \(0.25\), set \(X=Y=0\); with probability \(0.25\), set \(X=0\) and draw \(Y=U\); with probability \(0.25\), set \(Y=0\) and draw \(X=U\); with probability \(0.25\), set \(X=Y=U\). In each continuous case, \(U\) is uniform on \((0,1)\).
Both marginals have an atom of size \(0.5\) at zero and a continuous component of total probability \(0.5\), uniform on \((0,1)\). The atom–atom probability is \(0.25\), which matches \(0.5(0.5)\). Also, \(P(X=0,Y\in(0,0.5))=0.25(0.5)=0.125\), matching the independent value \(P(X=0)P(Y\in(0,0.5))=0.5(0.25)=0.125\).
But on the continuous–continuous set \(A=B=(0,0.5)\), the joint event occurs only in the last mixture component, where both variables equal the same \(U\). Therefore, \(P(X\in A,Y\in B)=0.25(0.5)=0.125\). The product of the marginal probabilities is \(P(X\in A)P(Y\in B)=0.25(0.25)=0.0625\). These are unequal, so \(X\) and \(Y\) are dependent.
The joint continuous component here is concentrated on the diagonal \(X=Y\); it does not have a two-dimensional density. A density-only comparison would miss this singular component entirely. The rectangle-probability criterion still detects it.
Common Pitfall and Practical Use
A common mistake is to compare only atom probabilities and the densities wherever they exist, then conclude that the variables are independent. That reasoning is incomplete if it omits atom–continuous probabilities, or if a continuous–continuous part is singular and has no joint density. Independence requires factorization for every pair of Borel sets, not just a match at atoms and at points where a density is available.
A reliable check is to identify the atomic sets first, then examine atom–atom probabilities, atom–continuous probabilities in both directions, and continuous–continuous probabilities. For a particular problem, a single mismatch in any one part proves dependence. To establish independence, however, the required factorization must hold for all sets in each part, not merely for a few illustrative intervals.
Check Your Understanding
For each question, identify which part of the mixed-law criterion is relevant.
- If \(P(X=a)=0.3\) and \(P(Y=b)=0.4\), what joint probability at \((a,b)\) is required by independence?
- Suppose \(P(X=a)=0.5\) and \(P(Y\in B)=0.2\) for an atom-free set \(B\). What value must \(P(X=a,Y\in B)\) have under independence?
- Why does matching the marginal distributions of \(X\) and \(Y\) not establish that they are independent?
- What does a density factorization on the atom-free parts fail to check if the joint law also has atoms?
- Why can a joint law concentrated on a diagonal fail to be detected by comparing only two-dimensional densities?