Tutorials › Stochastic Processes › Characteristic Functions Of Independent Linear Combinations

Home  /  Tutorials  /  Generating Functions and Transforms

Generating Functions and Transforms · Tutorial 39 of 994

Characteristic Functions Of Independent Linear Combinations

Learn to compute the characteristic function of an independent linear combination by rescaling each variable’s characteristic function and multiplying the results.

Intermediate 9 min read

What You'll Learn

  • State the characteristic function formula for a real random variable and its affine rescaling.
  • Derive the product formula for a finite linear combination of independent random variables.
  • Apply the formula to sums, differences, and combinations with zero coefficients.
  • Compute characteristic functions for Bernoulli and exponential examples.
  • Check when independence is essential and recognize a common product-rule error.

Rescaling and Combining Characteristic Functions

A characteristic function turns a random variable into a complex-valued function of a real parameter. For a linear combination, the main calculation has two parts: a coefficient rescales the argument of the characteristic function, and independence lets us multiply the resulting functions. This gives a compact way to calculate the characteristic function of a sum or difference without first finding the combination’s probability distribution.

Work on a probability space \((\Omega,\mathcal{F},\mathbb{P})\), and let \(X\) be a real-valued random variable. Its characteristic function is \(\varphi_X(t)=\mathbb{E}[e^{itX}]\), for \(t\in\mathbb{R}\), where \(i^2=-1\). This expectation is defined for every real \(t\), since \(|e^{itX}|=1\). If \(a,b\in\mathbb{R}\), the characteristic function of \(aX+b\) is \(\varphi_{aX+b}(t)=e^{itb}\varphi_X(at)\): the constant contributes a factor, while the coefficient \(a\) changes the argument from \(t\) to \(at\).

Theorem (Independent linear combinations): Let \(X_1,\ldots,X_m\) be mutually independent real-valued random variables on one probability space, and let \(a_1,\ldots,a_m\in\mathbb{R}\) be constants. For \(Z=\sum_{r=1}^{m}a_rX_r\), its characteristic function is \(\varphi_Z(t)=\prod_{r=1}^{m}\varphi_{X_r}(a_rt)\) for every \(t\in\mathbb{R}\). No moment or density assumptions are required. A coefficient may be zero; in that case its factor is \(\varphi_{X_r}(0)=1\).

Proof. Fix \(t\in\mathbb{R}\). By the definition of \(Z\) and the rule for exponentials of sums, \(e^{itZ}=\prod_{r=1}^{m}e^{it a_rX_r}\). Each factor is a bounded measurable function of \(X_r\). Mutual independence therefore gives factorization of the expectation of this product into the product of the expectations. Thus \(\mathbb{E}[e^{itZ}]=\prod_{r=1}^{m}\mathbb{E}[e^{i(a_rt)X_r}]=\prod_{r=1}^{m}\varphi_{X_r}(a_rt)\). If \(a_r=0\), the corresponding expectation is \(\mathbb{E}[1]=1\), as stated. \(\square\)

The theorem’s assumptions are worth keeping visible. The variables must be mutually independent, and the coefficients must be constants; the formula is not a rule for dependent variables or random coefficients. The formula needs no finite mean or variance: boundedness of the complex exponential is enough for every expectation in the proof to exist.

Corollary (Sums and differences): If \(X\) and \(Y\) are independent, then \(\varphi_{X+Y}(t)=\varphi_X(t)\varphi_Y(t)\) and \(\varphi_{X-Y}(t)=\varphi_X(t)\varphi_Y(-t)\). More generally, \(\varphi_{aX+bY}(t)=\varphi_X(at)\varphi_Y(bt)\) for real constants \(a,b\).

Proof. Apply the theorem with \(m=2\). For a sum use coefficients \(1,1\), giving \(\varphi_X(t)\varphi_Y(t)\). For a difference use coefficients \(1,-1\), giving \(\varphi_X(t)\varphi_Y(-t)\). For \(aX+bY\), the theorem gives the stated general formula. \(\square\)

The minus sign in the difference formula is important: it changes the argument of \(Y\)’s characteristic function. For a real-valued \(Y\), \(\varphi_Y(-t)=\overline{\varphi_Y(t)}\), since \(e^{-itY}\) is the complex conjugate of \(e^{itY}\), and expectation commutes with complex conjugation for bounded variables. Thus a difference of independent variables uses the characteristic function of one variable and the complex conjugate of the other at the same argument.

Worked Applications

Worked Example: A Difference of Independent Bernoulli Variables

Let \(X\) and \(Y\) be independent Bernoulli random variables with success probability \(1/2\). Thus each takes values \(0\) and \(1\), each with probability \(1/2\). Find the characteristic function of \(Z=2X-Y\), and check it directly from the possible values of \(Z\).

For either variable, \(\varphi_X(s)=\varphi_Y(s)=\tfrac12(1+e^{is})\): the value \(0\) contributes \(1/2\), and the value \(1\) contributes \(e^{is}/2\). The coefficient formula gives

$$ \varphi_Z(t)=\varphi_X(2t)\varphi_Y(-t) =\frac14(1+e^{2it})(1+e^{-it}) =\frac14(1+e^{2it}+e^{-it}+e^{it}). $$

For a direct check, the four equally likely pairs \((X,Y)=(0,0),(0,1),(1,0),(1,1)\) give \(Z=0,-1,2,1\), respectively. Hence \(\varphi_Z(t)=\tfrac14(e^{0}+e^{-it}+e^{2it}+e^{it})\), exactly the product-form expression. At \(t=0\), this equals \(\tfrac14(1+1+1+1)=1\), as every characteristic function must. The calculation also shows why the coefficient \(2\) changes the first argument to \(2t\), while the negative coefficient changes the second to \(-t\).

Worked Example: A Weighted Sum of Independent Normal Variables

Let \(X\) and \(Y\) be independent normal random variables with means \(0\) and variances \(1\) and \(4\), respectively. Compute the characteristic function of \(Z=3X-2Y\).

The characteristic function of a normal variable with mean \(0\) and variance \(\sigma^2\) is \(e^{-\sigma^2t^2/2}\). Therefore \(\varphi_X(3t)=e^{-(3t)^2/2}=e^{-9t^2/2}\). For \(Y\), whose variance is \(4\), \(\varphi_Y(-2t)=e^{-4(-2t)^2/2}=e^{-8t^2}\). Independence gives

$$ \varphi_Z(t)=e^{-9t^2/2}e^{-8t^2} =e^{-(9/2+8)t^2} =e^{-25t^2/2}. $$

Check: The variance calculation from independence is \(\operatorname{Var}(3X-2Y)=9\operatorname{Var}(X)+4\operatorname{Var}(Y)=9+16=25\); the cross-covariance term is zero because \(X\) and \(Y\) are independent and have finite variances. The computed function is the familiar normal-form characteristic function with variance \(25\). The calculation here establishes the characteristic function; identifying distributions from matching characteristic functions uses the uniqueness theorem.

Worked Example: The Difference of Independent Exponential Variables

Let \(X\) and \(Y\) be independent exponential random variables, each with rate \(1\), so each has density \(e^{-x}\) for \(x\ge 0\) and \(0\) otherwise. Compute the characteristic function of \(D=X-Y\).

For either variable, integrate the defining expectation over its density: \[ \varphi_X(t)=\int_0^\infty e^{itx}e^{-x}\,dx =\int_0^\infty e^{-(1-it)x}\,dx =\frac{1}{1-it}. \] The integral converges because the real part of \(1-it\) is \(1>0\); evaluating the antiderivative at the endpoints gives \(0-(-1/(1-it))=1/(1-it)\). The same calculation gives \(\varphi_Y(t)=1/(1-it)\), and therefore \(\varphi_Y(-t)=1/(1+it)\). Applying the difference formula,

$$ \varphi_D(t)=\frac{1}{1-it}\frac{1}{1+it} =\frac{1}{(1-it)(1+it)} =\frac{1}{1+t^2}. $$

Check: Multiplying the denominator gives \((1-it)(1+it)=1+t^2\). The result equals \(1\) at \(t=0\), and it is real and even in \(t\), as expected for the difference of two identically distributed independent variables. The result is computed from characteristic functions alone; no density for \(D\) is needed.

Why Independence Cannot Be Omitted

The product rule comes from factoring an expectation by independence, not merely from writing a sum inside an exponential. For dependent variables, the identity \(e^{it(X+Y)}=e^{itX}e^{itY}\) remains true, but it does not imply \(\mathbb{E}[e^{itX}e^{itY}]=\mathbb{E}[e^{itX}]\mathbb{E}[e^{itY}]\). The latter factorization needs independence (or some other condition that ensures this particular factorization).

For a concrete check, let \(X=Y\), where \(X\) takes \(1\) and \(-1\) with probability \(1/2\) each. Then \(\varphi_X(t)=\cos(t)\), while \(X+Y=2X\) has characteristic function \(\cos(2t)\). At \(t=\pi/2\), the actual characteristic function is \(\cos(\pi)=-1\), whereas the product of the marginal characteristic functions would be \(\cos^2(\pi/2)=0\). The product formula fails because \(X\) and \(Y\) are dependent.

Common mistake. Adding characteristic functions instead of multiplying them is also incorrect. For independent Bernoulli variables \(X,Y\) with success probability \(1/2\), set \(S=X+Y\). At \(t=\pi/2\), each characteristic function is \(\varphi_X(t)=\varphi_Y(t)=\tfrac12(1+i)\). The wrong calculation adds them and gives \(1+i\). The correct calculation multiplies: \[ \varphi_S(\pi/2)=\left(\frac{1+i}{2}\right)^2 =\frac{1+2i+i^2}{4} =\frac{i}{2}. \] Checking directly, \(S\) takes \(0,1,2\) with probabilities \(1/4,1/2,1/4\), so its characteristic function at \(\pi/2\) is \(\tfrac14+\tfrac12i-\tfrac14=i/2\). This confirms the correction.

For routine calculations, the key distinctions are: multiply across independent variables, rescale each argument by that variable’s coefficient, and use a negative argument for a negative coefficient. A zero coefficient contributes \(1\), not an extra random term. These rules give the characteristic function of the linear combination directly; they do not by themselves establish a converse or identify a distribution from its characteristic function.

Check Your Understanding

Use the coefficient rule, and show the argument substitutions in each calculation.

  1. If \(X,Y\) are independent, write the characteristic function of \(4X+Y\) in terms of \(\varphi_X\) and \(\varphi_Y\).
  2. If \(X,Y\) are independent with \(\varphi_X(t)=e^{-t^2}\) and \(\varphi_Y(t)=e^{-3t^2}\), compute the characteristic function of \(X-2Y\).
  3. Explain why the characteristic function of \(aX+bY\) still has a product formula when \(a=0\), and state what the \(X\)-factor equals.
  4. For independent fair Bernoulli variables \(X,Y\), list the possible values of \(X-Y\) and use them to check its characteristic function at \(t=0\).
  5. Give a dependent pair for which the product rule for a sum fails, and identify the factorization step that is no longer justified.

Previous

Next

Back to Tutorials