Location-Scale Group Actions in Gaussian Models

Why Study Location-Scale Transformations?

A multivariate Gaussian analysis should not depend on arbitrary choices of units or origin. Measuring blood pressure in a different unit changes the numerical coordinates of every observation; shifting the origin changes their location. These operations alter the likelihood’s numerical value, but they need not alter comparisons between corresponding statistical models. The reason is a shared transformation rule for the observations, parameters, and likelihood.

Here, a location-scale transformation means an affine transformation with an invertible linear part. It provides a precise way to express how Gaussian models behave under changes of coordinates, and it gives a standard criterion—equivariance—for checking whether an estimator respects those changes.

The Affine Action on Samples and Parameters

Definition: Let $a\in\mathbb{R}^p$ and let $B$ be an invertible $p\times p$ matrix. The affine transformation $g=(a,B)$ sends a vector $x$ to $g(x)=a+Bx$. On a sample $x_1,\ldots,x_n$, it acts componentwise: $y_i=a+Bx_i$. If $X_i\sim N(\mu,\Sigma)$, where $\Sigma$ is positive definite, the corresponding parameter transformation is $$ g(\mu,\Sigma)=(a+B\mu,\;B\Sigma B^{\mathsf T}). $$

The transformed vector has distribution $Y_i\sim N(a+B\mu,B\Sigma B^{\mathsf T})$. The covariance remains positive definite because for every nonzero $v$, $v^{\mathsf T}B\Sigma B^{\mathsf T}v=(B^{\mathsf T}v)^{\mathsf T}\Sigma(B^{\mathsf T}v)>0$. The transformations form a group: composing two affine transformations gives another one, and each has an inverse because $B$ is invertible.

For clarity, write $L_x(\mu,\Sigma)$ for the joint likelihood of the original sample and $L_y(\mu',\Sigma')$ for that of the transformed sample. The Jacobian of $x_i\mapsto a+Bx_i$ has absolute determinant $|\det B|$ for each observation, so the Jacobian for the whole sample is $|\det B|^n$. Thus the transformed density includes the reciprocal factor.

Theorem: For $y_i=a+Bx_i$, with $B$ invertible, and $(\mu',\Sigma')=g(\mu,\Sigma)$, $$ L_y(\mu',\Sigma')=|\det B|^{-n}L_x(\mu,\Sigma). $$ The factor depends on the data transformation, but not on the Gaussian parameters.

To see the density relationship directly, let $x_i=B^{-1}(y_i-a)$. The Gaussian density at $y_i$, under mean $\mu'=a+B\mu$ and covariance $\Sigma'=B\Sigma B^{\mathsf T}$, is the density at $x_i$ under $(\mu,\Sigma)$, multiplied by $|\det B|^{-1}$. Multiplication over the $n$ independent observations gives the displayed likelihood identity. This Jacobian factor is essential: likelihood values do change under a change of coordinates.

Worked Example: Transforming a Gaussian Mean and Covariance

Suppose $p=2$, $\mu=(1,2)^{\mathsf T}$, and $$ \Sigma=\begin{pmatrix}4&1\\1&3\end{pmatrix},\qquad B=\begin{pmatrix}2&1\\0&1\end{pmatrix},\qquad a=\begin{pmatrix}3\\-1\end{pmatrix}. $$ Find the transformed mean and covariance.

The determinant of $B$ is $2$, so $B$ is invertible. First, $$ B\mu=\begin{pmatrix}2&1\\0&1\end{pmatrix} \begin{pmatrix}1\\2\end{pmatrix} =\begin{pmatrix}4\\2\end{pmatrix}, \qquad \mu'=a+B\mu=\begin{pmatrix}7\\1\end{pmatrix}. $$ For the covariance, $$ B\Sigma=\begin{pmatrix}9&5\\1&3\end{pmatrix}, \qquad B^{\mathsf T}=\begin{pmatrix}2&0\\1&1\end{pmatrix}, $$ so $$ \Sigma'=B\Sigma B^{\mathsf T} =\begin{pmatrix}23&5\\5&3\end{pmatrix}. $$ This matrix is symmetric and positive definite: its leading entry is $23>0$, and its determinant is $23\cdot3-5\cdot5=44>0$. The affine transformation shifts the mean, while its linear part changes both covariance coordinates and scale.

Likelihood-Ratio Invariance

A likelihood ratio compares the best likelihood under one parameter set with the best likelihood under another. To compare models after transforming the data, the parameter sets must be transformed as well. Let $\Theta$ be a parameter space and $\Theta_0\subseteq\Theta$ a null parameter set. Define the likelihood ratio by $$ \Lambda(x)= \frac{\sup_{\theta\in\Theta_0}L_x(\theta)} {\sup_{\theta\in\Theta}L_x(\theta)}. $$ Here $\theta$ denotes the Gaussian parameters, or any chosen parameterization of them. The corresponding transformed sets are $g\Theta_0$ and $g\Theta$.

Theorem: Suppose the affine transformation is one-to-one on the Gaussian parameter space, and compare the transformed sample $y_i=a+Bx_i$ using the corresponding sets $g\Theta_0\subseteq g\Theta$. Whenever the suprema defining the likelihood ratio are finite and the denominator is positive, $$ \Lambda_y(g\Theta_0,g\Theta)=\Lambda_x(\Theta_0,\Theta). $$ Consequently, any test statistic that is a function only of this likelihood ratio, such as $-2\log\Lambda$, is unchanged.

Proof. The parameter transformation is a bijection, so as $\theta$ ranges over $\Theta_0$, $g\theta$ ranges over $g\Theta_0$; the same holds for $\Theta$. By the likelihood identity, $$ \sup_{\theta'\in g\Theta_0}L_y(\theta') =|\det B|^{-n}\sup_{\theta\in\Theta_0}L_x(\theta), \qquad \sup_{\theta'\in g\Theta}L_y(\theta') =|\det B|^{-n}\sup_{\theta\in\Theta}L_x(\theta). $$ The common positive factor cancels in the ratio, proving the result. Since the ratios are equal and positive, applying $-2\log(\cdot)$ also gives equal statistics. $\square$

Worked Example: A Likelihood Ratio After a Change of Coordinates

Let $X_1,X_2$ be independent $N(\mu,I_2)$ observations, and test the simple null hypothesis $\mu=(0,0)^{\mathsf T}$ against the unrestricted mean model with known covariance $I_2$. Suppose the observed vectors are $x_1=(1,0)^{\mathsf T}$ and $x_2=(3,2)^{\mathsf T}$. Compute the likelihood ratio, then verify it after an affine change of coordinates.

The sample mean is $\bar{x}=(2,1)^{\mathsf T}$. With covariance $I_2$, the log likelihood at $\mu$, apart from terms not depending on $\mu$, is $-\frac12\sum_{i=1}^2(x_i-\mu)^{\mathsf T}(x_i-\mu)$. The unrestricted maximum occurs at $\bar{x}$. The difference between the null and unrestricted log likelihoods is therefore $-\frac n2\bar{x}^{\mathsf T}\bar{x}=-\frac22(2^2+1^2)=-5$. Hence $\Lambda_x=\exp(-5)$.

Now take $a=(1,-1)^{\mathsf T}$ and $B=\begin{pmatrix}2&1\\0&1\end{pmatrix}$. The transformed observations are $y_1=(3,-1)^{\mathsf T}$ and $y_2=(9,1)^{\mathsf T}$, with mean $\bar{y}=(6,0)^{\mathsf T}$. The null mean transforms to $a=(1,-1)^{\mathsf T}$, and the known covariance transforms to $$ \Sigma'=BB^{\mathsf T}=\begin{pmatrix}5&1\\1&1\end{pmatrix}. $$ The difference between the transformed sample mean and transformed null mean is $\bar{y}-a=(5,1)^{\mathsf T}=B\bar{x}$. Its squared Mahalanobis distance is $$ (\bar{y}-a)^{\mathsf T}(\Sigma')^{-1}(\bar{y}-a) =\bar{x}^{\mathsf T}B^{\mathsf T}(BB^{\mathsf T})^{-1}B\bar{x} =\bar{x}^{\mathsf T}\bar{x}=5, $$ because $B^{\mathsf T}(BB^{\mathsf T})^{-1}B=I_2$ for invertible $B$. The transformed log-likelihood difference is again $-\frac22(5)=-5$, so the likelihood ratio remains $\exp(-5)$.

Equivariance of Estimators

Definition: An estimator $T$ of a parameter $\psi$ is equivariant under a transformation group if transforming the data and parameter transforms the estimate in the same way: $T(gx)=gT(x)$. For a location estimator, this means $T(a+Bx_1,\ldots,a+Bx_n)=a+BT(x_1,\ldots,x_n)$. For a scatter-matrix estimator $C$, it means $C(a+Bx_1,\ldots,a+Bx_n)=BC(x_1,\ldots,x_n)B^{\mathsf T}$.

Equivariance describes the behavior of an estimator, not the behavior of a likelihood ratio. It is a structural property: the numerical answer changes in the correct coordinate system when the data are re-expressed. The Gaussian sample mean and, when it exists, the unrestricted maximum-likelihood covariance estimate have this property.

Theorem: For a Gaussian sample, the sample mean $\bar{X}$ and the maximum-likelihood covariance estimate $$ \widehat{\Sigma}=\frac1n\sum_{i=1}^n(X_i-\bar{X})(X_i-\bar{X})^{\mathsf T} $$ are affine equivariant. The covariance statement applies when the unrestricted covariance estimate exists as a positive-definite matrix.

Proof. For the transformed observations $Y_i=a+BX_i$, their sample mean is $$ \bar{Y}=\frac1n\sum_{i=1}^n(a+BX_i)=a+B\bar{X}. $$ Thus the sample mean is location equivariant. Also $Y_i-\bar{Y}=B(X_i-\bar{X})$, so $$ \widehat{\Sigma}_Y =\frac1n\sum_{i=1}^n B(X_i-\bar{X})(X_i-\bar{X})^{\mathsf T}B^{\mathsf T} =B\widehat{\Sigma}_X B^{\mathsf T}. $$ This proves affine equivariance of the covariance estimate. The existence qualification matters: if the centered sample scatter is singular, the unrestricted positive-definite Gaussian covariance maximum may not exist. $\square$

More generally, likelihood maximizers inherit equivariance when the parameter transformation is a bijection and the likelihood changes only by a parameter-independent positive factor. The likelihood identity maps every maximizer for the original data to a maximizer for the transformed data. If the maximizer is unique, the maximum-likelihood estimator is equivariant. If there are multiple maximizers, the set of maximizers transforms accordingly, but an arbitrary rule for selecting one need not be equivariant.

Worked Example: Checking Mean and Scatter Equivariance Numerically

Consider the three two-dimensional observations $x_1=(0,0)^{\mathsf T}$, $x_2=(2,0)^{\mathsf T}$, and $x_3=(0,2)^{\mathsf T}$. Find their mean and maximum-likelihood scatter estimate, then apply $a=(1,-1)^{\mathsf T}$ and $B=\begin{pmatrix}2&0\\0&3\end{pmatrix}$.

The mean is $\bar{x}=(2/3,2/3)^{\mathsf T}$. The centered vectors are $(-2/3,-2/3)^{\mathsf T}$, $(4/3,-2/3)^{\mathsf T}$, and $(-2/3,4/3)^{\mathsf T}$. Summing their outer products gives $$ \sum_{i=1}^3(x_i-\bar{x})(x_i-\bar{x})^{\mathsf T} =\begin{pmatrix}8/3&-4/3\\-4/3&8/3\end{pmatrix}. $$ Dividing by $n=3$ yields $$ \widehat{\Sigma}_x=\begin{pmatrix}8/9&-4/9\\-4/9&8/9\end{pmatrix}. $$ The transformed mean is $a+B\bar{x}=(7/3,1)^{\mathsf T}$. The transformed scatter must be $$ B\widehat{\Sigma}_xB^{\mathsf T} =\begin{pmatrix}32/9&-8/3\\-8/3&8\end{pmatrix}. $$ These values are also obtained by centering the transformed observations: their centered vectors are $B(x_i-\bar{x})$, so each outer product is $B(x_i-\bar{x})(x_i-\bar{x})^{\mathsf T}B^{\mathsf T}$. The numerical result therefore verifies both parts of the equivariance rule.

Common Pitfall and Practical Meaning

Pitfall: “The likelihood ratio is invariant, so I can transform the data and keep the same numerical null hypothesis.” This is generally false. A null hypothesis about a mean or covariance must be carried along by the parameter transformation. In the likelihood-ratio example, the null mean $(0,0)^{\mathsf T}$ became $(1,-1)^{\mathsf T}$; retaining a zero transformed mean would compare a different model. The correction is to transform both the sample and every parameter set in the comparison. Only then do the shared Jacobian factors cancel.

These results matter whenever data are expressed in different units, rotated, or otherwise re-coordinatized. Likelihood-ratio conclusions for corresponding models do not depend on that representation, while equivariant estimates report the matching transformed answer. This does not mean every statistical procedure is invariant or equivariant: the property must be established for the particular statistic or estimator, and the hypotheses must be mapped consistently.

Check Your Understanding

Use the definitions and results above to answer the following questions.

Under $Y=a+BX$, what are the transformed Gaussian mean and covariance, and why does the covariance remain positive definite?
Why does a transformed $n$-observation likelihood acquire the factor $|\det B|^{-n}$?
What must happen to a null parameter set for the likelihood ratio to remain invariant after transforming the data?
State the affine equivariance identities for the sample mean and the maximum-likelihood covariance estimate.
Why does uniqueness matter when concluding that a selected maximum-likelihood estimator is equivariant?