Multivariate Affine Transformations of Characteristic Functions

Why Transform a Characteristic Function?

A random vector can be transformed by multiplying it by a matrix and adding a fixed vector. The transformed variable may be a scalar projection, a collection of linear contrasts, or a lower-dimensional summary. Characteristic functions make these operations especially direct: the matrix moves to the argument of the original characteristic function, while the translation contributes a complex exponential factor.

This rule is useful even when a random vector has no density and even when the transformation collapses several dimensions into fewer dimensions. To use it without dimension errors, keep track of the spaces in which the random vector, its image, and the argument of each characteristic function lie.

Definition and Affine Transformation Formula

Definition: If \(X\) is a random vector taking values in \(\mathbb{R}^p\), its characteristic function is the function \(\varphi_X:\mathbb{R}^p\to\mathbb{C}\) defined by

\varphi_X(s)=E\!\left[e^{i s^{\mathsf T}X}\right],\qquad s\in\mathbb{R}^p,

where \(i^2=-1\) and \(s^{\mathsf T}X\) is the usual inner product. The expectation always exists because the integrand has absolute value one.

Let \(A\) be an \(m\times p\) real matrix and \(b\in\mathbb{R}^m\). Then \(Y=AX+b\) is an \(\mathbb{R}^m\)-valued random vector, so its characteristic function has an argument \(t\in\mathbb{R}^m\). The transpose \(A^{\mathsf T}\) maps that argument back to \(\mathbb{R}^p\).

Theorem: For any random vector \(X\) in \(\mathbb{R}^p\), any real \(m\times p\) matrix \(A\), and any \(b\in\mathbb{R}^m\), the affine image \(Y=AX+b\) has characteristic function

\varphi_Y(t)=e^{i t^{\mathsf T}b}\,\varphi_X(A^{\mathsf T}t), \qquad t\in\mathbb{R}^m.

No nonsingularity, density, or moment assumption is required.

Proof. By the definition of the characteristic function and the identity \(t^{\mathsf T}(AX)=(A^{\mathsf T}t)^{\mathsf T}X\),

\begin{aligned} \varphi_Y(t) &=E\!\left[e^{i t^{\mathsf T}Y}\right]\\ &=E\!\left[e^{i t^{\mathsf T}(AX+b)}\right]\\ &=E\!\left[e^{i t^{\mathsf T}b}e^{i (A^{\mathsf T}t)^{\mathsf T}X}\right]\\ &=e^{i t^{\mathsf T}b}\,E\!\left[e^{i (A^{\mathsf T}t)^{\mathsf T}X}\right]\\ &=e^{i t^{\mathsf T}b}\,\varphi_X(A^{\mathsf T}t). \end{aligned}

The factor \(e^{i t^{\mathsf T}b}\) can be taken outside the expectation because it is constant with respect to \(X\). All expectations are well-defined: each exponential inside the expectation has absolute value one. This proves the formula. \(\square\)

Scalar Projections and Linear Contrasts

A scalar projection is the special case \(m=1\). Write the \(1\times p\) matrix \(A\) as \(c^{\mathsf T}\), where \(c\in\mathbb{R}^p\). For \(Y=c^{\mathsf T}X+b\), a scalar argument \(u\in\mathbb{R}\) gives

\varphi_Y(u)=e^{iub}\,\varphi_X(uc).

Thus the scalar characteristic function is obtained by evaluating the vector characteristic function along the line \(uc\) in its argument space. A linear contrast is just such a projection with a coefficient vector chosen to compare coordinates or groups. The formula does not require the coordinates of \(X\) to be independent.

Worked Example: A Projection of a Discrete Random Vector

Suppose \(X=(X_1,X_2)^{\mathsf T}\) takes the values \((1,0)^{\mathsf T}\), \((0,1)^{\mathsf T}\), and \((1,1)^{\mathsf T}\) with probabilities \(0.2\), \(0.3\), and \(0.5\), respectively. Let \(Y=3+2X_1-X_2\). Find \(\varphi_Y(u)\) both directly and using the affine transformation formula.

The coefficient vector is \(c=(2,-1)^{\mathsf T}\), and \(b=3\). The three possible values of \(c^{\mathsf T}X\) are \(2\), \(-1\), and \(1\), so \(Y\) takes the values \(5\), \(2\), and \(4\) with the same respective probabilities. Directly,

\varphi_Y(u)=0.2e^{i5u}+0.3e^{i2u}+0.5e^{i4u}.

The joint characteristic function of \(X\), evaluated at \(s=(s_1,s_2)^{\mathsf T}\), is

\varphi_X(s)=0.2e^{i s_1}+0.3e^{i s_2}+0.5e^{i(s_1+s_2)}.

The transformation formula evaluates this at \(uc=(2u,-u)^{\mathsf T}\) and multiplies by \(e^{i3u}\):

\begin{aligned} e^{i3u}\varphi_X(2u,-u) &=e^{i3u}\left(0.2e^{i2u}+0.3e^{-iu}+0.5e^{iu}\right)\\ &=0.2e^{i5u}+0.3e^{i2u}+0.5e^{i4u}. \end{aligned}

This agrees with the direct calculation. In particular, the joint law did not need to factor into separate marginal laws.

Worked Example: A Three-Coordinate Linear Contrast

Let \(X\in\mathbb{R}^3\) take the values \((1,0,2)^{\mathsf T}\), \((0,1,1)^{\mathsf T}\), and \((-1,2,0)^{\mathsf T}\) with probabilities \(1/4\), \(1/2\), and \(1/4\). Consider the contrast \(Y=2+X_1-2X_2+X_3\). Compute its characteristic function.

The contrast vector is \(c=(1,-2,1)^{\mathsf T}\). Its values on the three support points are

\begin{aligned} c^{\mathsf T}(1,0,2)^{\mathsf T}&=1-0+2=3,\\ c^{\mathsf T}(0,1,1)^{\mathsf T}&=0-2+1=-1,\\ c^{\mathsf T}(-1,2,0)^{\mathsf T}&=-1-4+0=-5. \end{aligned}

After adding 2, the corresponding values of \(Y\) are \(5\), \(1\), and \(-3\). Therefore

\varphi_Y(u)=\frac14 e^{i5u}+\frac12 e^{iu}+\frac14 e^{-i3u}.

Equivalently, the joint characteristic function is

\varphi_X(s_1,s_2,s_3) =\frac14 e^{i(s_1+2s_3)} +\frac12 e^{i(s_2+s_3)} +\frac14 e^{i(-s_1+2s_2)}.

Substituting \((s_1,s_2,s_3)=(u,-2u,u)\) and multiplying by \(e^{i2u}\) gives

\begin{aligned} e^{i2u}\varphi_X(u,-2u,u) &=e^{i2u}\left(\frac14 e^{i3u} +\frac12 e^{-iu} +\frac14 e^{-i5u}\right)\\ &=\frac14 e^{i5u}+\frac12 e^{iu}+\frac14 e^{-i3u}, \end{aligned}

which verifies the result using the joint characteristic function.

Singular Transformations and Lower-Dimensional Support

The matrix \(A\) need not be square or invertible. If \(A\) has fewer than \(m\) linearly independent columns, the image \(AX+b\) lies in a proper affine subspace of \(\mathbb{R}^m\). Its distribution then cannot have an ordinary density with respect to \(m\)-dimensional volume, but its characteristic function is still given by the same formula.

Result: For \(Y=AX+b\), every vector \(h\in\mathbb{R}^m\) satisfying \(A^{\mathsf T}h=0\) obeys

h^{\mathsf T}Y=h^{\mathsf T}b\quad\text{almost surely}, \qquad \varphi_Y(t+h)=e^{i h^{\mathsf T}b}\varphi_Y(t).

If \(A\) has rank less than \(m\), such a nonzero \(h\) exists, and the image is confined to a proper affine subspace.

Proof. Since \(Y=AX+b\),

h^{\mathsf T}Y=h^{\mathsf T}AX+h^{\mathsf T}b =(A^{\mathsf T}h)^{\mathsf T}X+h^{\mathsf T}b =h^{\mathsf T}b

for every outcome for which \(X\) is defined, and hence almost surely. Also \(A^{\mathsf T}(t+h)=A^{\mathsf T}t\), so the affine transformation formula yields

\varphi_Y(t+h) =e^{i(t+h)^{\mathsf T}b}\varphi_X(A^{\mathsf T}(t+h)) =e^{i h^{\mathsf T}b}\varphi_Y(t).

Finally, if \(\operatorname{rank}(A)<m\), the null space of \(A^{\mathsf T}\) contains a nonzero vector \(h\). The relation \(h^{\mathsf T}(Y-b)=0\) places \(Y\) in the hyperplane defined by that equation, a proper affine subspace of \(\mathbb{R}^m\). \(\square\)

Worked Example: A Singular Affine Image

Let \(X=(X_1,X_2)^{\mathsf T}\) be uniform on the four points \((1,1)^{\mathsf T}\), \((1,-1)^{\mathsf T}\), \((-1,1)^{\mathsf T}\), and \((-1,-1)^{\mathsf T}\). Define

A=\begin{pmatrix}1&2\\2&4\end{pmatrix}, \qquad b=\begin{pmatrix}1\\-1\end{pmatrix}, \qquad Y=AX+b.

The second row of \(A\) is twice the first, so \(A\) has rank one and is singular. The joint characteristic function of \(X\), computed by averaging over its four support points, is

\begin{aligned} \varphi_X(s_1,s_2) &=\frac14\left(e^{i(s_1+s_2)}+e^{i(s_1-s_2)} +e^{i(-s_1+s_2)}+e^{-i(s_1+s_2)}\right)\\ &=\cos(s_1)\cos(s_2). \end{aligned}

For \(t=(t_1,t_2)^{\mathsf T}\), the transpose multiplication gives \(A^{\mathsf T}t=(t_1+2t_2,\,2t_1+4t_2)^{\mathsf T}\), and \(t^{\mathsf T}b=t_1-t_2\). Hence

\varphi_Y(t_1,t_2) =e^{i(t_1-t_2)} \cos(t_1+2t_2)\cos(2t_1+4t_2).

The support restriction can be checked directly. If \(W=X_1+2X_2\), then \(Y_1=1+W\) and \(Y_2=-1+2W\), so \(Y_2=2Y_1-3\). Thus \(Y\) lies on a line in \(\mathbb{R}^2\). The vector \(h=(-2,1)^{\mathsf T}\) satisfies \(A^{\mathsf T}h=0\) and \(h^{\mathsf T}b=-3\), giving the same constraint \(Y_2-2Y_1=-3\).

Composition and Practical Checks

Affine transformations can be applied in stages without changing the final formula. If \(Y=AX+b\) and \(Z=CY+d\), where \(C\) and \(d\) have compatible dimensions, then \(Z=(CA)X+(Cb+d)\). The characteristic-function calculation confirms that the two-stage and one-stage descriptions agree:

Result:

\begin{aligned} \varphi_Z(t) &=e^{i t^{\mathsf T}d}\varphi_Y(C^{\mathsf T}t)\\ &=e^{i t^{\mathsf T}d}e^{i(C^{\mathsf T}t)^{\mathsf T}b} \varphi_X(A^{\mathsf T}C^{\mathsf T}t)\\ &=e^{i t^{\mathsf T}(Cb+d)} \varphi_X((CA)^{\mathsf T}t). \end{aligned}

This is precisely the affine formula for \(Z=(CA)X+(Cb+d)\).

When second moments exist, the same affine map also gives \(E[Y]=AE[X]+b\) and \(\operatorname{Cov}(Y)=A\operatorname{Cov}(X)A^{\mathsf T}\). These identities follow by centering \(Y\): \(Y-E[Y]=A(X-E[X])\), then expanding the covariance. A singular \(A\) can therefore produce a singular covariance matrix, consistent with the lower-dimensional support described above.

Common Mistake

A frequent error is to substitute \(At\) into \(\varphi_X\). That product may not even have the required dimension: \(t\) is in \(\mathbb{R}^m\), whereas the argument of \(\varphi_X\) must be in \(\mathbb{R}^p\). The correct argument is \(A^{\mathsf T}t\), which belongs to \(\mathbb{R}^p\). Another common omission is the translation factor \(e^{i t^{\mathsf T}b}\). A reliable check is to expand \(t^{\mathsf T}(AX+b)\) before taking the expectation.

Do not assume that an affine image has a density simply because the original vector does. An invertible square map can preserve full-dimensional support, but a rank-deficient map can collapse that support onto a line, plane, or other proper affine subspace. The characteristic-function formula applies in either case.

Key takeaway: For \(Y=AX+b\), the affine transformation rule is \(\varphi_Y(t)=e^{i t^{\mathsf T}b}\varphi_X(A^{\mathsf T}t)\). The transpose maps the argument to the original vector's dimension; the exponential accounts for translation; and no inverse or density is needed.

Check Your Understanding

Use dimensions and the affine transformation formula to answer the following.

If \(X\in\mathbb{R}^4\) and \(Y=AX+b\in\mathbb{R}^2\), what are the dimensions of \(A\), \(t\), and \(A^{\mathsf T}t\)?
Write the characteristic function of the scalar contrast \(Y=5-3X_1+X_2\) in terms of \(\varphi_X\), where \(X\in\mathbb{R}^2\).
Why does the affine transformation formula remain valid if \(A\) is singular?
If \(A^{\mathsf T}h=0\), what fixed linear constraint does \(Y=AX+b\) satisfy?
For \(Y=AX+b\), state the covariance transformation when \(X\) has finite second moments.