Tutorials › Multivariate Statistics (Graduate) › Affine Transformations and Sums Through Characteristic Functions

Home  /  Tutorials  /  Multivariate Laws and Transformations

Multivariate Laws and Transformations · Tutorial 60 of 975

Affine Transformations and Sums Through Characteristic Functions

Learn to transform and combine random-vector laws through characteristic functions, then identify the resulting distributions.

Beginner 9 min read

What You'll Learn

  • Apply the characteristic-function formula for an affine transformation of a random vector.
  • Derive the characteristic function of a sum of independent random vectors.
  • Combine affine maps and independent sums in one formula.
  • Identify transformed Gaussian and summed Poisson distributions from their characteristic functions.
  • Use a characteristic function to recover the support and probabilities of a transformed discrete variable.
  • Recognize why independence is essential to multiplying characteristic functions.

Why Characteristic Functions Help With Transformations

A random vector can be transformed in many ways: coordinates can be shifted, rescaled, mixed by a matrix, or added to another random vector. Finding the transformed density directly may require a change of variables, and a linear map need not even be one-to-one. Characteristic functions give a single calculation that applies to continuous, discrete, and mixed laws.

For a random vector \(X\) taking values in \(\mathbb{R}^p\), its characteristic function is the complex-valued function of \(t\in\mathbb{R}^p\) defined below. The expectation always exists because the complex exponential has absolute value one.

Definition: The characteristic function of \(X\in\mathbb{R}^p\) is \(\varphi_X(t)=E[\exp(i t^{\mathsf T}X)]\), for \(t\in\mathbb{R}^p\), where \(i^2=-1\). It records the distribution of \(X\): two probability distributions on \(\mathbb{R}^p\) are equal if and only if their characteristic functions agree at every \(t\).

The two rules developed here are direct consequences of this definition. An affine transformation changes the argument of the original characteristic function and adds a phase factor for the shift. For an independent sum, the characteristic functions multiply. The uniqueness property then turns these calculations into distributional identifications.

Affine Transformations

Let \(X\in\mathbb{R}^p\), let \(A\) be a fixed \(d\times p\) real matrix, and let \(b\in\mathbb{R}^d\). The transformed vector \(Y=AX+b\) takes values in \(\mathbb{R}^d\). The formula below does not require \(A\) to be invertible, nor does it require \(X\) to have a density.

Theorem: For \(Y=AX+b\), the characteristic function of \(Y\) is \(\varphi_Y(t)=\exp(i t^{\mathsf T}b)\varphi_X(A^{\mathsf T}t)\), for every \(t\in\mathbb{R}^d\).

Proof. By the definition of a characteristic function and the identity \(t^{\mathsf T}AX=(A^{\mathsf T}t)^{\mathsf T}X\),

$$ \begin{aligned} \varphi_Y(t) &=E\!\left[\exp\{i t^{\mathsf T}(AX+b)\}\right]\\ &=\exp(i t^{\mathsf T}b)\, E\!\left[\exp\{i(A^{\mathsf T}t)^{\mathsf T}X\}\right]\\ &=\exp(i t^{\mathsf T}b)\varphi_X(A^{\mathsf T}t). \end{aligned} $$

The expectation is well-defined: the random factor inside it has absolute value one. This proves the formula. \(\square\)

The transpose in the argument is important: \(t\) is \(d\)-dimensional, while the argument supplied to \(\varphi_X\) must be \(p\)-dimensional. If \(b=0\), only the argument changes; a nonzero shift contributes the factor \(\exp(i t^{\mathsf T}b)\).

Worked Example: An Affine Transformation of a Gaussian Vector

Suppose \(X\) is a bivariate normal vector with characteristic function \(\varphi_X(s)=\exp\{i s^{\mathsf T}\mu-\tfrac12s^{\mathsf T}\Sigma s\}\), where \(\mu=(1,-2)^{\mathsf T}\) and \(\Sigma=\begin{pmatrix}4&1\\1&2\end{pmatrix}\). Set \(A=\begin{pmatrix}1&2\\-1&1\end{pmatrix}\), \(b=(3,0)^{\mathsf T}\), and \(Y=AX+b\). Find the law of \(Y\).

Use the affine formula with \(s=A^{\mathsf T}t\). Substitution into the Gaussian characteristic function gives

$$ \begin{aligned} \varphi_Y(t) &=\exp(i t^{\mathsf T}b)\, \exp\{i(A^{\mathsf T}t)^{\mathsf T}\mu -\tfrac12(A^{\mathsf T}t)^{\mathsf T}\Sigma(A^{\mathsf T}t)\}\\ &=\exp\{i t^{\mathsf T}(A\mu+b) -\tfrac12t^{\mathsf T}(A\Sigma A^{\mathsf T})t\}. \end{aligned} $$

The transformed location parameter is \(A\mu+b=(1-4,-1-2)^{\mathsf T}+(3,0)^{\mathsf T}=(0,-3)^{\mathsf T}\). For the quadratic term, the rows of \(A\) are \(r_1=(1,2)\) and \(r_2=(-1,1)\). Direct multiplication gives \(r_1\Sigma r_1^{\mathsf T}=16\), \(r_1\Sigma r_2^{\mathsf T}=-1\), and \(r_2\Sigma r_2^{\mathsf T}=4\), so

$$ A\Sigma A^{\mathsf T}= \begin{pmatrix}16&-1\\-1&4\end{pmatrix}. $$

This matrix is positive definite: its first diagonal entry is positive and its determinant is \(16\cdot4-(-1)^2=63>0\). The resulting characteristic function is therefore that of the bivariate normal law with location \((0,-3)^{\mathsf T}\) and covariance parameter \(\begin{pmatrix}16&-1\\-1&4\end{pmatrix}\). By uniqueness of characteristic functions, it identifies the law of \(Y\).

Sums of Independent Random Vectors

Now let \(X\) and \(Z\) both take values in \(\mathbb{R}^p\). Their sum also takes values in \(\mathbb{R}^p\). Independence makes the characteristic function of this sum especially simple.

Theorem: If \(X\) and \(Z\) are independent random vectors in \(\mathbb{R}^p\), then \(\varphi_{X+Z}(t)=\varphi_X(t)\varphi_Z(t)\) for every \(t\in\mathbb{R}^p\).

Proof. For each fixed \(t\), the variables \(\exp(i t^{\mathsf T}X)\) and \(\exp(i t^{\mathsf T}Z)\) are independent, since each is a measurable function of one of the independent vectors. They are bounded, so their expectations exist. Factorization of expectations for independent variables yields

$$ \begin{aligned} \varphi_{X+Z}(t) &=E[\exp\{i t^{\mathsf T}(X+Z)\}]\\ &=E[\exp(i t^{\mathsf T}X)\exp(i t^{\mathsf T}Z)]\\ &=E[\exp(i t^{\mathsf T}X)]E[\exp(i t^{\mathsf T}Z)]\\ &=\varphi_X(t)\varphi_Z(t). \end{aligned} $$

Thus the characteristic function of the sum is the product shown. \(\square\)

The same reasoning combines affine maps and sums. If \(X_1,\ldots,X_m\) are independent, \(A_j\) are fixed matrices with a common number of rows \(d\), and \(b\in\mathbb{R}^d\), then \(W=b+\sum_{j=1}^m A_jX_j\) has characteristic function

$$ \varphi_W(t)=\exp(i t^{\mathsf T}b) \prod_{j=1}^m\varphi_{X_j}(A_j^{\mathsf T}t), \qquad t\in\mathbb{R}^d. $$

Indeed, apply the affine formula to the sum and factor its expectation using the independence of the \(X_j\). This formula is useful when each summand has a familiar characteristic function.

Worked Example: Adding Independent Poisson Vectors

Let \(U=(N_1,N_2)^{\mathsf T}\) and \(V=(M_1,M_2)^{\mathsf T}\). Assume all four counts are independent, with \(N_1\sim\operatorname{Poisson}(2)\), \(N_2\sim\operatorname{Poisson}(3)\), \(M_1\sim\operatorname{Poisson}(4)\), and \(M_2\sim\operatorname{Poisson}(1)\). Identify the law of \(S=U+V\).

A Poisson variable \(N\) with rate \(\lambda\) has characteristic function \(\exp\{\lambda(e^{it}-1)\}\), as follows by summing its probability mass function:

$$ E[e^{itN}] =\sum_{k=0}^{\infty}e^{itk}e^{-\lambda}\frac{\lambda^k}{k!} =e^{-\lambda}\exp(\lambda e^{it}) =\exp\{\lambda(e^{it}-1)\}. $$

The coordinates within each vector are independent, so \(\varphi_U(s,t)=\exp\{2(e^{is}-1)+3(e^{it}-1)\}\) and \(\varphi_V(s,t)=\exp\{4(e^{is}-1)+(e^{it}-1)\}\). The vectors \(U\) and \(V\) are independent, so their sum has characteristic function

$$ \begin{aligned} \varphi_S(s,t) &=\varphi_U(s,t)\varphi_V(s,t)\\ &=\exp\{6(e^{is}-1)+4(e^{it}-1)\}. \end{aligned} $$

This is the joint characteristic function of two independent Poisson variables with rates 6 and 4. Uniqueness identifies the distribution of \(S\); in particular, its first coordinate is \(N_1+M_1\sim\operatorname{Poisson}(6)\), and its second is \(N_2+M_2\sim\operatorname{Poisson}(4)\).

Worked Example: A Shifted Linear Combination of Bernoulli Variables

Let \(B_1\sim\operatorname{Bernoulli}(1/3)\) and \(B_2\sim\operatorname{Bernoulli}(1/2)\) be independent, and define \(T=5+2B_1-B_2\). Find its characteristic function and distribution.

For a Bernoulli variable with success probability \(q\), its characteristic function is \(1-q+qe^{it}\), because it equals 0 with probability \(1-q\) and 1 with probability \(q\). Applying the affine-and-sum formula gives

$$ \varphi_T(t) =e^{i5t}\left(\frac23+\frac13e^{i2t}\right) \left(\frac12+\frac12e^{-it}\right). $$

To read off the distribution, enumerate the four possible pairs. If \((B_1,B_2)\) is \((0,0),(0,1),(1,0),(1,1)\), then \(T\) is respectively \(5,4,7,6\). Independence gives the corresponding probabilities \(\frac23\cdot\frac12=\frac13\), \(\frac23\cdot\frac12=\frac13\), \(\frac13\cdot\frac12=\frac16\), and \(\frac13\cdot\frac12=\frac16\). Thus \(P(T=4)=P(T=5)=1/3\), while \(P(T=6)=P(T=7)=1/6\). The probabilities sum to \(1/3+1/3+1/6+1/6=1\), as required.

Assumptions, Scope, and a Common Pitfall

These formulas are broad: an affine map can be noninvertible, the input can be discrete or continuous, and no density is needed. What matters for the sum formula is not that the summands have the same distribution, but that they are independent and have the same vector dimension.

  • Wrong argument to the affine characteristic function. Replacing \(\varphi_X(A^{\mathsf T}t)\) by \(\varphi_X(At)\) ignores the dimensions and generally gives the wrong transformation. The correct argument follows from \(t^{\mathsf T}AX=(A^{\mathsf T}t)^{\mathsf T}X\).
  • Multiplying without independence. The identity \(\varphi_{X+Z}=\varphi_X\varphi_Z\) needs independence. Without it, the exact expression is \(E[\exp\{i t^{\mathsf T}X+i t^{\mathsf T}Z\}]\), which need not factor. The joint characteristic function, evaluated at \((t,t)\), accounts for their dependence.
  • Confusing the calculation with an identification. A proposed law is established only after its characteristic function matches the transformed or product expression. The uniqueness theorem is what licenses the distributional conclusion.

Characteristic functions turn transformations into algebra: transpose the matrix in the argument, include the shift factor, and multiply only when independence permits. These rules make it possible to identify many resulting laws without deriving a transformed density.

Check Your Understanding

Use the formulas and reasoning from this tutorial to answer the following.

  1. If \(X\in\mathbb{R}^p\) and \(Y=AX+b\in\mathbb{R}^d\), what is the dimension of the argument supplied to \(\varphi_X\)?
  2. State the characteristic function of \(AX+b\) and identify which factor represents the shift.
  3. Why does the characteristic function of a sum of independent vectors equal a product?
  4. Two independent Poisson variables have rates 3 and 5. Use their characteristic functions to identify the law of their sum.
  5. What information would you need beyond the separate characteristic functions to handle a sum of dependent random vectors?

Previous

Next

Back to Tutorials
Clinical Biostats