Dependence from a Shared Random Input

Introduction

Suppose $Z$ is independent of $X$, and define $Y=X+Z$. The noise $Z$ is independent of $X$, but $Y$ contains $X$ itself. That shared input can make $X$ and $Y$ dependent: knowing $X$ changes the conditional distribution of $Y$. To see why, it is useful to write down the joint law and compare that conditional distribution with the marginal law of $Y$.

The distinction is important in models that combine a signal with noise. An independent error term does not, by itself, make the signal independent of the noisy observation. The calculations below make that dependence explicit for discrete, continuous, and normal examples.

Joint and Conditional Laws for a Shared Input

Definition: Let $X$ and $Z$ be independent random variables, and define $Y=X+Z$. Then $X$ is a shared random input for the pair $(X,Y)$: it appears both as the first variable and as part of the second. The independence assumption is between $X$ and $Z$, not between $X$ and $Y$.

In the discrete case, write $p_X(x)=P(X=x)$ and $p_Z(z)=P(Z=z)$. For possible values $x,y$,

$$ P(X=x,Y=y)=p_X(x)p_Z(y-x). $$

Indeed, the event $\{X=x,Y=y\}$ is exactly $\{X=x,Z=y-x\}$, and independence lets us multiply the probabilities. Where $p_X(x)>0$, dividing by the marginal probability $P(X=x)$ gives

P(Y=y\mid X=x)=p_Z(y-x).

Thus, given $X=x$, the sum $Y$ has the distribution of $Z$ shifted by $x$.

The same relationship holds for continuous variables with densities. If $X$ and $Z$ are independent and have densities $f_X$ and $f_Z$, the change of variables from $(x,z)$ to $(x,y)=(x,x+z)$ has Jacobian determinant 1. Since $z=y-x$, the joint density is

f_{X,Y}(x,y)=f_X(x)f_Z(y-x).

Where $f_X(x)>0$, the corresponding conditional density is

f_{Y\mid X}(y\mid x)=\frac{f_{X,Y}(x,y)}{f_X(x)}=f_Z(y-x).

For continuous $X$, conditional distributions given $X=x$ are understood as regular conditional distributions and the density formula holds for almost every $x$ with respect to the distribution of $X$. A conditional distribution is not obtained by assigning positive probability to the event $\{X=x\}$, which usually has probability zero.

Worked Example: Two Bernoulli Inputs

Let $X$ and $Z$ be independent Bernoulli random variables with success probability $1/2$, and let $Y=X+Z$. Find the joint distribution and check whether $X$ and $Y$ are independent.

Each of the four pairs $(X,Z)$ has probability $1/4$. The relation $Y=X+Z$ maps them as follows:

\begin{aligned} (X,Z)=(0,0)&\ \Longrightarrow\ (X,Y)=(0,0),\\ (X,Z)=(0,1)&\ \Longrightarrow\ (X,Y)=(0,1),\\ (X,Z)=(1,0)&\ \Longrightarrow\ (X,Y)=(1,1),\\ (X,Z)=(1,1)&\ \Longrightarrow\ (X,Y)=(1,2). \end{aligned}

Therefore $P(X=0,Y=0)=1/4$. The marginal probabilities are $P(X=0)=1/2$ and $P(Y=0)=1/4$, so independence would require $P(X=0,Y=0)=(1/2)(1/4)=1/8$. Since $1/4\ne1/8$, $X$ and $Y$ are dependent.

The conditional law makes the same fact visible: $P(Y=0\mid X=0)=1/2$, whereas $P(Y=0\mid X=1)=0$. In contrast, the added variable $Z$ is independent of $X$ by assumption. The dependence arises because $Y$ includes $X$.

Conditional Distributions Reveal the Dependence

Result: If $X$ and $Z$ are independent and $Y=X+Z$, then, for every $x$ with $P(X=x)>0$ in the discrete case, $$ P(Y\le y\mid X=x)=F_Z(y-x). $$ If $X$ and $Z$ have densities, the conditional density is $f_{Y\mid X}(y\mid x)=f_Z(y-x)$ for almost every $x$ with respect to the distribution of $X$.

In the discrete case, $\{Y\le y\}$, conditional on $X=x$, is the event $\{Z\le y-x\}$. Independence means the conditional distribution of $Z$ given $X=x$ is its marginal distribution, so $P(Y\le y\mid X=x)=P(Z\le y-x)=F_Z(y-x)$. For densities, the joint density derived above factors as $f_X(x)f_Z(y-x)$; dividing by $f_X(x)$ gives the stated conditional density wherever that division is defined. This proves both formulas.

Independence of $X$ and $Y$ would require the conditional distribution of $Y$ given $X=x$ to equal the marginal distribution of $Y$, for almost every $x$ (or for each $x$ with positive probability in the discrete case). Consequently, if those conditional distributions vary with $x$ on a set of positive probability, then $X$ and $Y$ are dependent. A shift in the conditional distribution is a direct way to investigate this, without assuming that a covariance exists.

Worked Example: Two Independent Uniform Variables

Let $X$ and $Z$ be independent and uniform on $(0,1)$, and set $Y=X+Z$. Find the conditional distribution of $Y$ given $X=x$, then verify dependence using events.

Given $X=x$, $Y=x+Z$. Since $Z$ is uniform on $(0,1)$, $Y\mid X=x$ is uniform on $(x,x+1)$, with conditional density 1 on that interval. In particular, for $0<x<1$,

P(Y>1\mid X=x)=P(Z>1-x)=x.

The conditional probability changes with $x$. To compare joint and marginal probabilities directly, first calculate $P(Y>1)=1/2$: the event is $\{Z>1-X\}$, and its probability is $\int_0^1 x\,dx=1/2$. Also $P(X>3/4)=1/4$. But

\begin{aligned} P(X>3/4,Y>1) &=\int_{3/4}^{1}P(Y>1\mid X=x)f_X(x)\,dx\\ &=\int_{3/4}^{1}x\,dx\\ &=\left[\frac{x^2}{2}\right]_{3/4}^{1} =\frac12-\frac{9}{32} =\frac{7}{32}. \end{aligned}

If $X$ and $Y$ were independent, this joint probability would be $P(X>3/4)P(Y>1)=(1/4)(1/2)=1/8=4/32$. Since $7/32\ne4/32$, they are dependent. The calculation illustrates how an event involving $Y$ can become more likely as the value of the shared input $X$ increases.

A Covariance Check When Moments Exist

When $X$ and $Z$ have finite second moments, covariance supplies a concise additional check. Independence of $X$ and $Z$ implies $\operatorname{Cov}(X,Z)=0$. Since $Y=X+Z$, linearity of covariance gives

\operatorname{Cov}(X,Y) =\operatorname{Cov}(X,X+Z) =\operatorname{Var}(X)+\operatorname{Cov}(X,Z) =\operatorname{Var}(X).

Here the finite second moments ensure that all the displayed covariances exist. If $X$ is nonconstant with finite variance, then $\operatorname{Var}(X)>0$, so the covariance of $X$ and $Y$ is nonzero. Independence of variables with finite second moments implies zero covariance; therefore $X$ and $Y$ cannot be independent.

Worked Example: A Normal Signal Plus Normal Noise

Let $X\sim N(0,1)$ and $Z\sim N(0,1)$ be independent, and let $Y=X+Z$. Derive the conditional distribution of $Y$ given $X=x$, and use covariance to confirm dependence.

The density of $Z$ is $f_Z(z)=(2\pi)^{-1/2}\exp(-z^2/2)$. The conditional density formula gives

f_{Y\mid X}(y\mid x) =\frac{1}{\sqrt{2\pi}}\exp\left(-\frac{(y-x)^2}{2}\right).

This is the density of $N(x,1)$. The marginal distribution of $Y$ is $N(0,2)$, since it is a sum of independent normal variables with means 0 and variances 1 and 1. In particular, the conditional distribution depends on $x$, while the marginal distribution does not. Also,

\operatorname{Cov}(X,Y)=\operatorname{Var}(X)=1.

This nonzero covariance confirms dependence. The noise $Z$ is independent of $X$, but the observation $Y$ is not: its conditional mean is $x$, reflecting the shared signal.

Common Mistake and Why the Distinction Matters

A tempting but incorrect argument is: “$Z$ is independent of $X$, and $Y=X+Z$, so $Y$ is independent of $X$.” The conclusion does not follow. The independent quantity is the added noise $Z$; the sum $Y$ also contains $X$. To check the claim, calculate a joint probability or a conditional distribution. If $P(Y\le y\mid X=x)$ changes with $x$, then $X$ and $Y$ are dependent.

The covariance calculation is useful when second moments exist, but it is not a definition of independence. Zero covariance alone does not establish independence, and covariance is unavailable when the required moments do not exist. In those situations, comparing joint and marginal probabilities, or comparing conditional and marginal distributions, remains useful.

This distinction matters whenever an observation is formed by adding noise to a quantity of interest. Independent measurement error does not erase the information carried from the underlying quantity into the observation. Keeping track of which variables are independent—and which are linked by a shared input—prevents incorrect conclusions about the joint model.

Key takeaway: If $Z$ is independent of $X$, then $Y=X+Z$ has a conditional distribution given $X=x$ equal to the distribution of $Z$ shifted by $x$. That conditional law generally varies with $x$, revealing dependence between $X$ and $Y$.

Check Your Understanding

Use joint probabilities, conditional distributions, or covariance as appropriate.

If $X$ and $Z$ are independent discrete variables and $Y=X+Z$, express $P(X=x,Y=y)$ using their mass functions.
For independent $X$ and $Z$ with densities, what is the conditional density of $Y=X+Z$ given $X=x$? State where the formula applies.
If $X,Z$ are independent Bernoulli$(1/2)$ variables and $Y=X+Z$, calculate $P(Y=1\mid X=0)$ and $P(Y=1\mid X=1)$.
Suppose $X$ and $Z$ are independent with finite second moments and $\operatorname{Var}(X)>0$. Derive $\operatorname{Cov}(X,X+Z)$, and explain what it implies.
Why does independence of an added noise variable $Z$ and an input $X$ not imply independence of $X$ and $X+Z$?