Transforming Random Variables by Change of Variables

How a Transformation Changes a Distribution

A random variable can be transformed without changing the underlying experiment: a discrete variable might be squared, or two continuous variables might be replaced by their sum and ratio. The distribution of the new variable is determined by how the original probability is arranged over the values that map to it. For discrete variables, probabilities from all preimages are added. For continuous variables, the derivative or Jacobian adjusts for how the transformation stretches or compresses space.

Let \((\Omega,\mathcal{F},\mathbb{P})\) be a probability space. If \(X\) is a random variable and \(g\) is a measurable function, the transformed random variable is \(Y=g(X)\). Its distribution is defined by \(\mathbb{P}(Y\in B)=\mathbb{P}(X\in g^{-1}(B))\) for each measurable set \(B\). The challenge is to turn this definition into a mass function or density that can be used directly.

Discrete Transformations: Add the Masses in Each Fiber

Proof. The event \(\{Y=y\}\) is exactly the union of the events \(\{X=x\}\) over all \(x\in S\) satisfying \(g(x)=y\). These events are pairwise disjoint, since \(X\) cannot equal two different values at once. Countable additivity therefore gives \(\mathbb{P}(Y=y)=\sum_{x:g(x)=y}\mathbb{P}(X=x)\). If \(y\) is not in the image of \(g\), its preimage is empty and the probability is zero. \(\square\)

One-Dimensional Continuous Transformations

For a continuous random variable, there is no positive probability at an individual point to add up. Instead, a density describes probability over small intervals. If \(g\) is strictly monotone and differentiable, nearby values of \(X\) are mapped to nearby values of \(Y=g(X)\). The absolute derivative of the inverse accounts for the change in interval length.

Proof. First suppose \(g\) is increasing. For an interior point \(y\), the distribution function satisfies \(F_Y(y)=\mathbb{P}(g(X)\le y)=F_X(g^{-1}(y))\). Differentiating by the chain rule gives \(f_Y(y)=f_X(g^{-1}(y))(g^{-1})'(y)\); the inverse derivative is positive and equals \(1/g'(g^{-1}(y))\). If \(g\) is decreasing, the event becomes \(\{X\ge g^{-1}(y)\}\), so \(F_Y(y)=1-F_X(g^{-1}(y))\). Differentiating gives \(f_Y(y)=f_X(g^{-1}(y))[-(g^{-1})'(y)]\). Here the inverse derivative is negative and has absolute value \(1/|g'(g^{-1}(y))|\). In either case the stated absolute-derivative formula follows. Outside \(J\), \(Y\) has no values, so its density is zero. \(\square\)

When There Is More Than One Inverse

Strict monotonicity matters: without it, one output value can come from several input values. When a differentiable transformation has finitely many inverse branches and nonzero derivative at the preimages, each branch contributes a density term. In one dimension, at a regular value \(y\), the formula is $$ f_Y(y)=\sum_{\{x:g(x)=y\}}\frac{f_X(x)}{|g'(x)|}. $$ This is the same local stretching adjustment as before, summed over all the ways to produce \(y\). The formula applies on regions where the preimages are regular; isolated endpoints or critical values require separate attention but do not affect a density if they have probability zero.

Joint Transformations: The Jacobian

For two variables, a transformation can change both coordinates at once. The corresponding adjustment is the absolute determinant of the derivative matrix of the inverse transformation. It measures how areas in the transformed coordinates correspond to areas in the original coordinates.

Proof. For a measurable region \(A\subseteq E\), the definition of a joint density gives \(\mathbb{P}((Y_1,Y_2)\in A)=\int_{T^{-1}(A)} f_X(x_1,x_2)\,dx_1dx_2\). Apply the multivariable change-of-variables formula from calculus to the one-to-one map \(x=G(y)\). The integral becomes \(\int_A f_X(G(y_1,y_2))|\det DG(y_1,y_2)|\,dy_1dy_2\). Since this identity holds for every measurable \(A\), the integrand on \(E\) is a joint density for \((Y_1,Y_2)\). Outside \(E\), the transformed pair cannot take values, so the density is zero. \(\square\)

A Common Pitfall: Losing a Preimage

A transformation formula must account for every inverse branch. For the continuous square example, using only \(x=\sqrt{y}\) would give the incorrect density \(\tfrac{1}{4\sqrt{y}}\) on \(0<y<1\). Its total integral is \(\int_0^1\tfrac{1}{4\sqrt{y}}\,dy=\tfrac{1}{2}\), not \(1\). The missing half is the contribution from \(x=-\sqrt{y}\). The corrected density adds both branches and integrates to one. In a discrete transformation the same mistake appears as omitting one input value from the fiber.

Support is another essential part of the calculation. A formula obtained from a derivative or Jacobian only applies where the transformed value is attainable. For example, the sum and ratio transformation above has \(u>0\) and \(0<v<1\), not every point in the plane. Values outside this region must have density zero. Finally, a continuous density is determined up to changes on sets of area or length zero, so choices at boundary points do not change the distribution.