Marginals and Linear Projections of Elliptical Vectors

Why Linear Projections Matter

A multivariate random vector is often observed through only a few coordinates or through combinations such as a total score, a contrast, or a set of linear summaries. For an elliptical vector, these operations preserve the elliptical characteristic-function form. This gives a direct way to identify the location and scatter of the projected vector without first finding its density.

The key calculation is to replace the argument of the original characteristic function by the transpose of the projection matrix applied to the new argument. The resulting quadratic form determines the projected scatter. Coordinate marginals are an important special case: selecting coordinates extracts the corresponding entries of the location vector and the corresponding principal submatrix of the scatter matrix.

Linear Projections and Their Characteristic Functions

Let $X$ be a $p$-dimensional elliptical random vector with location $\mu\in\mathbb{R}^p$, symmetric positive-definite scatter matrix $\Sigma$, and characteristic-function generator $\psi$. Thus, for every $t\in\mathbb{R}^p$, its characteristic function has the form

\varphi_X(t)=E[\exp(i t^\top X)] =\exp(i t^\top\mu)\,\psi(t^\top\Sigma t), \qquad \psi(0)=1.

Here the same scalar function $\psi$ is evaluated at a nonnegative quadratic form. The scatter matrix specifies the shape and orientation in this representation; it need not equal the covariance matrix unless the relevant moments exist and the appropriate scaling is applied.

Definition: For an $m\times p$ matrix $A$ and a fixed vector $b\in\mathbb{R}^m$, the affine projection of $X$ is $Y=AX+b$. Its projected location is $A\mu+b$, and its projected scatter candidate is $A\Sigma A^\top$. A linear projection is the special case $b=0$.

If $A$ has full row rank, then $A\Sigma A^\top$ is positive definite, so it is a nonsingular scatter matrix for the $m$-dimensional projected vector. If $A$ does not have full row rank, this matrix is singular; the projection is then confined to a lower-dimensional subspace. The formula below still gives its characteristic function, but the nonsingular elliptical density description does not apply.

Theorem: Suppose $X$ has characteristic function $\varphi_X(t)=\exp(i t^\top\mu)\psi(t^\top\Sigma t)$, where $\Sigma$ is positive definite. For any $m\times p$ matrix $A$ and fixed $b\in\mathbb{R}^m$, $Y=AX+b$ has characteristic function $$ \varphi_Y(s)=\exp\!\big(i s^\top(A\mu+b)\big)\, \psi\!\big(s^\top A\Sigma A^\top s\big), \qquad s\in\mathbb{R}^m. $$ If $A$ has full row rank, $Y$ is an $m$-dimensional elliptical vector with location $A\mu+b$, scatter $A\Sigma A^\top$, and generator $\psi$.

Proof. By the definition of the characteristic function and $Y=AX+b$,

\begin{aligned} \varphi_Y(s) &=E[\exp(i s^\top(AX+b))]\\ &=\exp(i s^\top b)\,E[\exp(i (A^\top s)^\top X)]\\ &=\exp(i s^\top b)\, \exp(i (A^\top s)^\top\mu)\, \psi\!\big((A^\top s)^\top\Sigma(A^\top s)\big)\\ &=\exp\!\big(i s^\top(A\mu+b)\big)\, \psi\!\big(s^\top A\Sigma A^\top s\big). \end{aligned}

The third equality uses the stated characteristic-function form for $X$. If $A$ has full row rank, then for every nonzero $s\in\mathbb{R}^m$, $A^\top s\ne0$. Positive definiteness of $\Sigma$ gives

s^\top A\Sigma A^\top s=(A^\top s)^\top\Sigma(A^\top s)>0.

Thus $A\Sigma A^\top$ is positive definite, and the displayed characteristic function has the elliptical form in dimension $m$, with the claimed parameters. $\square$

A useful moment consequence follows when $X$ has a finite mean and covariance. Linearity of expectation and covariance gives

E[Y]=A E[X]+b,\qquad \operatorname{Cov}(Y)=A\operatorname{Cov}(X)A^\top.

In particular, if $\operatorname{Cov}(X)=c\Sigma$ for some finite scalar $c$, then $\operatorname{Cov}(Y)=cA\Sigma A^\top$. The scatter transformation itself does not require finite moments; the covariance statement does.

Coordinate Marginals

Choose $m$ coordinates from $X$ using an $m\times p$ selection matrix $P_I$, whose rows are the coordinate row vectors for the selected indices $I$. Then $X_I=P_I X$. The projection theorem gives location $P_I\mu$ and scatter $P_I\Sigma P_I^\top$. In ordinary vector notation, these are the selected entries of $\mu$ and the principal submatrix of $\Sigma$ indexed by $I$.

Result: Every coordinate subvector of a nonsingular elliptical vector is elliptical. Its characteristic function is obtained by restricting the original characteristic function to arguments whose unselected coordinates are zero. Its location is the selected subvector of $\mu$, and its scatter is the matching principal submatrix of $\Sigma$.

To see the restriction directly, take a vector $s\in\mathbb{R}^m$ of arguments for the selected coordinates. The corresponding argument in $\mathbb{R}^p$ is $P_I^\top s$, with zeros in the unselected positions. Substitution yields

\varphi_{X_I}(s) =\varphi_X(P_I^\top s) =\exp\!\big(i s^\top P_I\mu\big)\, \psi\!\big(s^\top P_I\Sigma P_I^\top s\big).

Because a coordinate-selection matrix has full row rank, its projected scatter is positive definite. This identifies the marginal law even when a density calculation would be inconvenient. If a joint density exists, the marginal density can also be obtained by integrating out unselected coordinates, but its density generator need not have the same expression as the joint density generator. The characteristic-function generator in the formula above is unchanged.

Worked Projections

Worked Example: A Two-Dimensional Projection With a Checked Cross-Term

Let $X\sim N_3(\mu,\Sigma)$, with

\mu=\begin{pmatrix}0\\1\\2\end{pmatrix}, \qquad \Sigma= \begin{pmatrix} 7&1&2\\ 1&2&2\\ 2&2&4 \end{pmatrix}, \qquad A= \begin{pmatrix} 1&1&0\\ 0&1&1 \end{pmatrix}.

The scatter matrix is positive definite: its leading principal minors are $7$, $7\cdot2-1^2=13$, and $\det(\Sigma)=24$, all positive. Find the distributional parameters of $Y=AX$, and calculate its characteristic function.

First, the projected location is

A\mu= \begin{pmatrix} 1\\3 \end{pmatrix}.

For the projected scatter, calculate both diagonal entries and the cross-entry. The row vectors of $A$ are $r_1=(1,1,0)$ and $r_2=(0,1,1)$. Direct multiplication gives

\Sigma r_2^\top= \begin{pmatrix}3\\4\\6\end{pmatrix}, \qquad r_1\Sigma r_2^\top=3+4=7.

The diagonal entries are $r_1\Sigma r_1^\top=7+1+1+2=11$ and $r_2\Sigma r_2^\top=2+2+2+4=10$. Therefore

A\Sigma A^\top= \begin{pmatrix}11&7\\7&10\end{pmatrix}.

Since a normal vector has generator $\psi(q)=\exp(-q/2)$, the characteristic function of $Y$, for $s=(s_1,s_2)^\top$, is

\begin{aligned} \varphi_Y(s) &=\exp\!\left(i(s_1+3s_2)\right) \exp\!\left[-\frac12 \left(11s_1^2+14s_1s_2+10s_2^2\right)\right]. \end{aligned}

The coefficient of $s_1s_2$ is twice the off-diagonal scatter entry, so it is $2(7)=14$. Hence $Y\sim N_2\!\left((1,3)^\top,\begin{pmatrix}11&7\\7&10\end{pmatrix}\right)$.

Worked Example: Selecting Two Coordinate Marginals

Let $X\sim N_3(\mu,\Sigma)$, where

\mu=\begin{pmatrix}2\\0\\-1\end{pmatrix}, \qquad \Sigma= \begin{pmatrix} 4&1&0\\ 1&9&3\\ 0&3&5 \end{pmatrix}.

Consider the marginal vector $Y=(X_1,X_3)^\top$. Select coordinates 1 and 3 with

P_I= \begin{pmatrix} 1&0&0\\ 0&0&1 \end{pmatrix}.

The marginal location and scatter are

P_I\mu=\begin{pmatrix}2\\-1\end{pmatrix}, \qquad P_I\Sigma P_I^\top= \begin{pmatrix}4&0\\0&5\end{pmatrix}.

The normal characteristic function is therefore

\varphi_Y(s_1,s_2) =\exp\!\left(i(2s_1-s_2)\right) \exp\!\left[-\frac12(4s_1^2+5s_2^2)\right].

Thus $Y$ is a bivariate normal marginal with the displayed location and scatter. The zero off-diagonal entry follows from the selected entries of the original scatter matrix; it is not a conditional calculation. The positive definiteness check for $\Sigma$ is $4>0$, $4\cdot9-1=35>0$, and $\det(\Sigma)=139>0$, so the stated three-dimensional normal model is valid.

Worked Example: A Scalar Contrast

Let $X\sim N_2(\mu,\Sigma)$, with $\mu=(1,4)^\top$ and

\Sigma=\begin{pmatrix}3&1\\1&2\end{pmatrix}.

Find the distribution of the contrast $Y=2X_1-X_2$. Here $A=(2,-1)$, so its location is

A\mu=2(1)-4=-2.

For the scatter, first compute

\Sigma A^\top= \begin{pmatrix}3&1\\1&2\end{pmatrix} \begin{pmatrix}2\\-1\end{pmatrix} = \begin{pmatrix}5\\0\end{pmatrix}, \qquad A\Sigma A^\top=(2,-1)\begin{pmatrix}5\\0\end{pmatrix}=10.

Consequently,

\varphi_Y(t)=\exp(-2it)\exp(-10t^2/2) =\exp(-2it-5t^2), \qquad Y\sim N(-2,10).

The scalar scatter is positive, as required for a nondegenerate projected normal distribution. The matrix $\Sigma$ is positive definite because its leading principal minors are $3$ and $3\cdot2-1=5$.

Common Mistakes and Why the Result Is Useful

Dropping part of a cross-entry. To compute $r_1\Sigma r_2^\top$, first multiply $\Sigma r_2^\top$, then take the dot product with all entries of $r_1$. In the first example that dot product is $3+4=7$, not $3$. The quadratic form uses twice the off-diagonal entry, giving a cross-term of $14s_1s_2$.
Using the wrong scatter submatrix for a coordinate marginal. Selecting coordinates 1 and 3 means retaining rows and columns 1 and 3, including their mutual covariance entry. It does not mean using the conditional distribution given the omitted coordinate.
Calling scatter covariance without checking moments. The projection theorem applies through characteristic functions and does not require finite second moments. A covariance statement requires those moments to exist and must use $\operatorname{Cov}(X)$, not automatically $\Sigma$.
Assuming every projection has a full-dimensional density. A matrix with fewer than $m$ independent rows can yield a singular scatter. The characteristic-function transformation still holds, but the projected law is not a nonsingular $m$-dimensional elliptical distribution.

These formulas are useful whenever a multivariate model is summarized by linear contrasts or reduced to a subset of measured variables. They preserve the elliptical structure while making the transformed location and scatter explicit. In practice, checking the matrix product $A\Sigma A^\top$ carefully is often the main calculation.

Check Your Understanding

Use the projection formula and show the key matrix products in your reasoning.

If $Y=AX+b$, what are its location and scatter in terms of those of $X$?
Why does full row rank of $A$ guarantee that $A\Sigma A^\top$ is positive definite when $\Sigma$ is positive definite?
For a coordinate marginal, how do you obtain its scatter matrix from the original scatter matrix?
Suppose $X$ has finite covariance $C$. What is the covariance of $AX+b$, and when can it be written as a scalar multiple of the projected scatter?
What does a zero cross-term in the projected quadratic form tell you, and what does it not establish by itself?