Affine Invariance of One-Sample Mean Inference

Why Affine Invariance Matters

A multivariate mean can be expressed in different coordinate systems: measurements may be converted to new units, shifted to a different origin, or replaced by invertible linear combinations. Such a change should not alter the evidence about a mean hypothesis if the hypothesis and covariance are expressed in the new coordinates as well. Affine invariance makes this principle precise.

The key is to transform the whole inferential problem, not just the observed mean. Under an invertible affine transformation, the sample mean and hypothesized mean shift and rotate or rescale together, while the covariance changes by the corresponding matrix rule. Mahalanobis distances, Hotelling's statistic, and their confidence ellipsoids are preserved in this coordinated transformation.

Transforming the Data and Parameters

Definition: Let $X_1,\ldots,X_n$ be random vectors in $\mathbb{R}^p$. An invertible affine transformation is the map $x\mapsto Ax+b$, where $A$ is an invertible $p\times p$ matrix and $b\in\mathbb{R}^p$. Define $Y_i=AX_i+b$. If $E[X_i]=\mu$ and $\operatorname{Cov}(X_i)=\Sigma$, then $E[Y_i]=A\mu+b$ and $\operatorname{Cov}(Y_i)=A\Sigma A^T$.

Write $\bar X=n^{-1}\sum_{i=1}^n X_i$ and $S_X=(n-1)^{-1}\sum_{i=1}^n(X_i-\bar X)(X_i-\bar X)^T$. Directly from the definitions,

\bar Y=A\bar X+b,\qquad S_Y=AS_XA^T.

The translation cancels from every centered observation: $Y_i-\bar Y=A(X_i-\bar X)$. The same covariance rule applies to a known population covariance $\Sigma_Y=A\Sigma A^T$. Because $A$ is invertible, $\Sigma_Y$ is positive definite whenever $\Sigma$ is, and

\Sigma_Y^{-1}=A^{-T}\Sigma^{-1}A^{-1}.

A null hypothesis $H_0:\mu=\mu_0$ must become $H_0:A\mu+b=A\mu_0+b$ in the transformed coordinates. Since $A$ is invertible, this is an equivalent null hypothesis: the transformation gives a one-to-one correspondence between possible means, and it does not change which parameter values satisfy the null.

Invariance of the Known-Covariance Statistic

For a specified positive-definite covariance matrix $\Sigma$, the squared Mahalanobis statistic for testing $H_0:\mu=\mu_0$ is

Q=n(\bar X-\mu_0)^T\Sigma^{-1}(\bar X-\mu_0).

Under the affine transformation, the difference between the transformed sample mean and transformed null mean is $A(\bar X-\mu_0)$. The statistic in the transformed coordinates is therefore

\begin{aligned} Q_Y &=n\{A(\bar X-\mu_0)\}^T(A\Sigma A^T)^{-1}\{A(\bar X-\mu_0)\}\\ &=n(\bar X-\mu_0)^TA^TA^{-T}\Sigma^{-1}A^{-1}A(\bar X-\mu_0)\\ &=n(\bar X-\mu_0)^T\Sigma^{-1}(\bar X-\mu_0)=Q. \end{aligned}

The cancellation uses $A^TA^{-T}=I$ and $A^{-1}A=I$. It is an algebraic equality for every data set; it does not depend on a probability model. Calibration of the statistic, however, does depend on the sampling model.

Theorem: Suppose $X_1,\ldots,X_n$ are independent and identically distributed as $N(\mu,\Sigma)$, with known positive-definite $\Sigma$. Under $H_0:\mu=\mu_0$, $Q$ has a chi-square distribution with $p$ degrees of freedom. For every invertible affine transformation of the observations, with the covariance and null transformed accordingly, the value of $Q$, its null distribution, and the resulting test decision are unchanged.

For the distribution claim, under the null, $\sqrt n(\bar X-\mu_0)\sim N(0,\Sigma)$. A positive-definite square root $\Sigma^{-1/2}$ standardizes this vector, so $\Sigma^{-1/2}\sqrt n(\bar X-\mu_0)\sim N(0,I_p)$. Its squared Euclidean length is $Q$, hence $Q\sim\chi^2_p$. The algebra above gives the identical statistic after transformation, so the same critical value and p-value apply. The independent multivariate normal assumption is essential for this exact finite-sample calibration; the algebraic invariance alone does not establish a chi-square law. $\square$

Worked Example: A Known-Covariance Test After a Change of Coordinates

Suppose independent observations are multivariate normal with known covariance $\Sigma=\begin{pmatrix}4&0\\0&1\end{pmatrix}$. Let $n=10$, test $H_0:\mu=\mu_0$, and suppose the observed difference is $\bar x-\mu_0=(2,1)^T$. Apply $A=\begin{pmatrix}1&1\\0&1\end{pmatrix}$ and $b=(3,-2)^T$.

The original statistic is

Q=10\left(\frac{2^2}{4}+\frac{1^2}{1}\right)=10(1+1)=20.

The transformed difference is $A(2,1)^T=(3,1)^T$. The transformed covariance and its inverse are

A\Sigma A^T= \begin{pmatrix}5&1\\1&1\end{pmatrix}, \qquad (A\Sigma A^T)^{-1} =\frac14\begin{pmatrix}1&-1\\-1&5\end{pmatrix}.

Thus the transformed quadratic form is

10(3,1)\frac14 \begin{pmatrix}1&-1\\-1&5\end{pmatrix} \begin{pmatrix}3\\1\end{pmatrix} =10(3,1)\begin{pmatrix}1/2\\1/2\end{pmatrix} =20.

Under the stated normal model and null, $Q\sim\chi^2_2$. At level $0.05$, the upper-tail critical value is approximately $5.991$; since $20>5.991$, both coordinate systems reject. This decision follows from the specified sampling model as well as the equal statistic values.

Invariance with Estimated Covariance

When the covariance is unknown, the one-sample Hotelling statistic uses the sample covariance:

T^2=n(\bar X-\mu_0)^TS_X^{-1}(\bar X-\mu_0).

This expression is defined when $S_X$ is invertible. With $S_Y=AS_XA^T$, invertibility is preserved under invertible $A$, and $S_Y^{-1}=A^{-T}S_X^{-1}A^{-1}$. Repeating the quadratic-form calculation gives

\begin{aligned} T_Y^2 &=n\{A(\bar X-\mu_0)\}^T(AS_XA^T)^{-1}\{A(\bar X-\mu_0)\}\\ &=n(\bar X-\mu_0)^TS_X^{-1}(\bar X-\mu_0)=T^2. \end{aligned}

Theorem: If $X_1,\ldots,X_n$ are independent and identically distributed as $N(\mu,\Sigma)$, with positive-definite $\Sigma$, and $n>p$, then $S_X$ is invertible with probability one. Under $H_0:\mu=\mu_0$, $$ \frac{n-p}{p(n-1)}T^2\sim F_{p,n-p}. $$ The same $T^2$, and therefore the same exact $F$-calibrated decision, results after any invertible affine transformation when the null is transformed with the data.

The exact Hotelling calibration follows from the normal sampling laws for the sample mean and sample covariance: $\bar X$ and $S_X$ are independent, $\sqrt n(\bar X-\mu_0)\sim N(0,\Sigma)$ under the null, and $(n-1)S_X$ has a Wishart distribution with $n-1$ degrees of freedom and scale $\Sigma$. These conditions yield the stated $F$ law. The affine calculation proves that the statistic itself is identical in both coordinate systems. In particular, the identical exact null distribution gives identical p-values and decisions. $\square$

Worked Example: Hotelling's Statistic Under a Linear Transformation

Let $n=8$, $p=2$, and suppose independent observations come from a bivariate normal distribution with unknown positive-definite covariance. For the observed data, take $\bar x-\mu_0=(1,2)^T$ and $S_X=\begin{pmatrix}4&1\\1&2\end{pmatrix}$.

The determinant of $S_X$ is $8-1=7$, so $S_X^{-1}=\frac17\begin{pmatrix}2&-1\\-1&4\end{pmatrix}$. Multiplication gives $S_X^{-1}(1,2)^T=(0,1)^T$; the quadratic form is $2$, and $T^2=8(2)=16$. The exact $F$ statistic is

\frac{8-2}{2(8-1)}16=\frac{48}{7}\approx6.857, \qquad \text{with }(2,6)\text{ degrees of freedom}.

Now choose $A=\begin{pmatrix}2&1\\1&1\end{pmatrix}$, whose determinant is $1$, so it is invertible. The transformed difference is $(4,3)^T$, and

AS_XA^T=\begin{pmatrix}22&13\\13&8\end{pmatrix}, \qquad (AS_XA^T)^{-1}=\frac17\begin{pmatrix}8&-13\\-13&22\end{pmatrix}.

Applying this inverse to $(4,3)^T$ gives $(-1,2)^T$, so the transformed quadratic form is $4(-1)+3(2)=2$. Consequently $T_Y^2=8(2)=16$, and the transformed $F$ statistic remains $48/7$. The normality and $n>p$ assumptions justify the exact $F_{2,6}$ calibration.

Confidence Ellipsoids Transform as Sets

A confidence ellipsoid is not unchanged in its plotted coordinates: a stretch, shear, or rotation changes its shape and location. What is invariant is its inferential content: the transformed confidence set is exactly the affine image of the original one. For known covariance, the level $1-\alpha$ set is

C_X=\left\{m:n(\bar x-m)^T\Sigma^{-1}(\bar x-m)\leq \chi^2_{p,1-\alpha}\right\},

where $\chi^2_{p,1-\alpha}$ is the $1-\alpha$ quantile of the chi-square distribution with $p$ degrees of freedom. In the transformed coordinates, $\bar y=A\bar x+b$, $\Sigma_Y=A\Sigma A^T$, and a candidate mean $m_Y$ corresponds to $m=A^{-1}(m_Y-b)$. Substitution into the transformed quadratic form reduces it to the original one. Hence $C_Y=\{Am+b:m\in C_X\}$, with the same coverage under the normal model.

For unknown covariance, replace $\Sigma$ by $S_X$ and the chi-square quantile by $p(n-1)F_{p,n-p;1-\alpha}/(n-p)$, where $F_{p,n-p;1-\alpha}$ is the corresponding $F$ quantile. The same substitution shows that the transformed confidence ellipsoid is precisely the affine image of the original. This is the set-level counterpart of Hotelling-statistic invariance.

Worked Example: Mapping a Known-Covariance Confidence Ellipsoid

Suppose the known covariance is $I_2$, $n=4$, and the observed mean is $\bar x=(1,0)^T$. Under independent bivariate normal sampling, the 95% confidence ellipsoid is

4\{(m_1-1)^2+m_2^2\}\leq 5.991.

Transform the data using $z=Ax+b$, where $A=\begin{pmatrix}2&0\\0&1\end{pmatrix}$ and $b=(3,-1)^T$. The center becomes $A\bar x+b=(5,-1)^T$, and the covariance becomes $AI_2A^T=\begin{pmatrix}4&0\\0&1\end{pmatrix}$. Thus the transformed ellipsoid is

4\left\{\frac{(m_{Y,1}-5)^2}{4}+(m_{Y,2}+1)^2\right\}\leq5.991.

To check the set mapping, write $m_Y=Am+b$. Then $m_{Y,1}-5=2(m_1-1)$ and $m_{Y,2}+1=m_2$. Substituting gives exactly $4\{(m_1-1)^2+m_2^2\}\leq5.991$. The ellipse is wider in the first transformed coordinate, but it represents the same confidence statement in transformed units.

Common Pitfall and Practical Meaning

A frequent error is to transform the observed mean but leave the covariance matrix unchanged. In the first worked example, using the transformed difference $(3,1)^T$ with the old covariance $\operatorname{diag}(4,1)$ would give $10(9/4+1)=32.5$, not $20$. That calculation compares quantities in inconsistent coordinate systems. The correction is to transform the covariance as $A\Sigma A^T$, or $S_X$ as $AS_XA^T$, and transform the null mean by the same affine map.

Another error is to infer a chi-square or $F$ reference distribution from invariance alone. Invariance is an algebraic fact; an exact rejection rule also requires its sampling assumptions. For example, the known-covariance chi-square calibration above requires independent multivariate normal observations with known covariance. Without an appropriate distributional assumption or a justified approximation, one may still report that the statistic is affine invariant, but not claim the stated exact calibrated decision.

Key takeaway: An invertible affine re-expression preserves one-sample mean inference when the observations, null mean, and covariance are all transformed together. The numerical test statistic and calibrated decision remain the same under the appropriate sampling model, while the confidence ellipsoid is carried to its exact affine image.

Check Your Understanding

Use the definitions and results above to check both the algebra and the role of assumptions.

For $Y_i=AX_i+b$, derive the formulas for $\bar Y$ and $S_Y$, and explain why the translation disappears from the sample covariance.
Why must the null mean transform from $\mu_0$ to $A\mu_0+b$, rather than to $A\mu_0$?
State the sampling conditions that justify the exact chi-square calibration for the known-covariance statistic.
Explain why an invertible affine transformation preserves the numerical Hotelling statistic when $S_X$ is invertible.
Does the confidence ellipsoid retain the same coordinate equation after a stretch? Describe what remains unchanged and what changes.