Vector Location-Scale Transformations

From Scalar Changes to Vector Transformations

A scalar location-scale transformation shifts and rescales a random variable. For a random vector, a matrix can also stretch, compress, reflect, or mix its coordinates. The same compact expression describes all these operations, but the matrix’s rank matters: an invertible matrix transforms the full-dimensional space without collapsing it, while a rank-deficient matrix sends the vector into a lower-dimensional affine subspace.

Let \(\mathbf Z\) be a random vector in \(\mathbb R^k\), let \(\boldsymbol\mu\in\mathbb R^p\), and let \(A\) be a fixed \(p\times k\) real matrix. The resulting vector \(\mathbf X\) lies in \(\mathbb R^p\). The entries of \(\boldsymbol\mu\) set the location, while \(A\) determines how the coordinates of \(\mathbf Z\) are transformed.

Means, Covariances, and Characteristic Functions

The transformation rules for moments follow from linearity. Write \(E[\mathbf Z]\) for the vector of coordinatewise expectations and \(\operatorname{Cov}(\mathbf Z)\) for the matrix whose \((i,j)\) entry is \(\operatorname{Cov}(Z_i,Z_j)\). A finite mean is enough for the mean rule; the covariance rule requires finite second moments.

To verify the mean formula, write the \(i\)th coordinate as \(X_i=\mu_i+\sum_{j=1}^k A_{ij}Z_j\). Taking expectations term by term gives \(E[X_i]=\mu_i+\sum_{j=1}^k A_{ij}E[Z_j]\), which is the \(i\)th coordinate of \(\boldsymbol\mu+A E[\mathbf Z]\). For the covariance, set \(\mathbf m=E[\mathbf Z]\). Since \(\mathbf X-E[\mathbf X]=A(\mathbf Z-\mathbf m)\), its \((i,\ell)\) covariance entry is

This is exactly the \((i,\ell)\) entry of \(A\operatorname{Cov}(\mathbf Z)A^T\). The finite second-moment assumption ensures these expectations and covariance entries are finite. \(\square\)

For \(\mathbf t\in\mathbb R^p\), define the characteristic function by \(\varphi_{\mathbf X}(\mathbf t)=E[\exp(i\mathbf t^T\mathbf X)]\). Characteristic functions always exist, whether or not moments do.

Indeed, \(\mathbf t^T\mathbf X=\mathbf t^T\boldsymbol\mu+\mathbf t^TA\mathbf Z=\mathbf t^T\boldsymbol\mu+(A^T\mathbf t)^T\mathbf Z\). Exponentiating, taking expectations, and factoring out the nonrandom term gives the result. No density or moment condition is needed. \(\square\)

Support Under an Affine Map

The support of a random vector is the set of points for which every open neighborhood has positive probability. It describes where the distribution is concentrated, including boundary points of probability zero. For a continuous affine map, the transformed support is the closure of the image of the original support.

For completeness, \(\mathbf Z\) belongs to its support with probability one: the complement can be covered by a countable collection of open balls with rational centers and radii, each of which has probability zero if it misses the support. If a point \(\mathbf y\) is outside the closure of \(g(S)\), it has an open neighborhood disjoint from \(g(S)\). Since \(\mathbf Z\in S\) almost surely, \(\mathbf X\) has probability zero in that neighborhood, so \(\mathbf y\) is not in the transformed support.

Conversely, suppose \(\mathbf y\) is in the closure of \(g(S)\). Every open neighborhood \(O\) of \(\mathbf y\) contains some \(g(\mathbf z)\) with \(\mathbf z\in S\). By continuity of \(g\), there is an open neighborhood \(U\) of \(\mathbf z\) whose image is contained in \(O\). The definition of support gives \(P(\mathbf Z\in U)>0\), hence \(P(\mathbf X\in O)>0\). Thus \(\mathbf y\) is in the transformed support, proving the equality. Finally, every \(A\mathbf z\) lies in \(\operatorname{col}(A)\), which proves the containment. \(\square\)

When \(A\) is square and invertible, it maps open sets to open sets and has a continuous inverse. The support is then exactly \(\boldsymbol\mu+A S\), with no need for a closure beyond the usual closed support \(S\). If \(\mathbf Z\) has a density \(f_{\mathbf Z}\), the transformed density is \(f_{\mathbf X}(\mathbf x)=f_{\mathbf Z}(A^{-1}(\mathbf x-\boldsymbol\mu))/|\det A|\). If \(A\) is rank deficient, its image lies in a proper affine subspace of \(\mathbb R^p\). In that case the transformed distribution cannot have a density with respect to \(p\)-dimensional Lebesgue measure: that subspace has \(p\)-dimensional measure zero, yet it contains probability one.

Common Pitfalls and Why Rank Matters

A frequent mistake is to treat a rank-deficient transformation as though it merely produced a very thin full-dimensional distribution. It does not: all its probability lies exactly in \(\boldsymbol\mu+\operatorname{col}(A)\). This has practical consequences. For example, a covariance matrix from such a vector is singular, and standard formulas requiring an inverse covariance matrix cannot be applied without additional treatment.

• Using the wrong dimensions. If \(A\) is \(p\times k\), then \(A^T\mathbf t\) has dimension \(k\), while \(\mathbf X\) and \(\boldsymbol\mu\) have dimension \(p\).
• Forgetting the transpose in the covariance rule. The correct transformation is \(A\operatorname{Cov}(\mathbf Z)A^T\), a \(p\times p\) matrix.
• Assuming a rank-\(r\) map fills its image. The transformed support is the closure of the image of the actual support, not necessarily the entire affine subspace or even its full image region.
• Applying a full-dimensional density formula to a singular map. The determinant-based formula requires a square invertible matrix; a rank-deficient image has zero ambient volume.

The mean, covariance, and characteristic-function formulas require different assumptions: a finite mean for the first, finite second moments for the second, and no moment assumptions for the third. Keeping these conditions distinct makes the rules valid for discrete, continuous, and mixed distributions alike.