Eigenvector Perturbation Under Spectral Gaps

Why Eigenvector Sensitivity Depends on Gaps

A principal direction is an eigenvector of a covariance matrix, so a small change in the matrix can change the direction even when its eigenvalues change only slightly. The size and direction of that change matter when interpreting components: a direction that responds strongly to small perturbations is less stable as a scientific summary. For a simple eigenvalue, the first-order change has a precise form. Its coefficients are the perturbation’s off-diagonal components in the eigenbasis, divided by gaps between the target eigenvalue and the others.

Throughout, $A$ is a real symmetric $p\times p$ matrix. Write its eigenvalues as $\lambda_1,\ldots,\lambda_p$, with corresponding orthonormal eigenvectors $v_1,\ldots,v_p$. The order of the eigenvalues is arbitrary unless stated otherwise. A perturbation $E$ is also real symmetric, so $A(t)=A+tE$ remains symmetric for real $t$. The eigenvalue perturbation result gives the first-order change in an eigenvalue; here the target is the associated normalized eigenvector.

Definition and First-Order Formula

Definition: An eigenvalue $\lambda_j$ of $A$ is simple if it has multiplicity one. Its spectral gap from the rest of the spectrum is $$ \delta_j=\min_{k\ne j}|\lambda_j-\lambda_k|. $$ For a simple eigenvalue, $\delta_j>0$. A normalized eigenvector $v_j(t)$ for the corresponding eigenvalue of $A(t)$ is locally oriented by requiring $v_j(t)^\mathsf{T}v_j(0)>0$.

The orientation condition resolves the local sign ambiguity: $v$ and $-v$ are both eigenvectors, but the condition chooses the one pointing toward the unperturbed vector. It does not impose an additional restriction on the eigenvector’s direction. Locally, a simple eigenvalue has a differentiable eigenvalue branch and a differentiable normalized eigenvector branch under a differentiable symmetric matrix perturbation.

Theorem: Let $A(t)$ be a continuously differentiable real symmetric matrix-valued function near $t=0$, and suppose $\lambda_j$ is a simple eigenvalue of $A(0)$, with unit eigenvector $v_j$. Let $v_j(t)$ be the locally oriented, unit eigenvector branch for the corresponding eigenvalue $\lambda_j(t)$. Then $$ v_j'(0)=\sum_{k\ne j}\frac{v_k^\mathsf{T}A'(0)v_j}{\lambda_j-\lambda_k}\,v_k. $$ In particular, for $A(t)=A+tE$, replace $A'(0)$ by $E$. The first-order expansion is $v_j(t)=v_j+t\,v_j'(0)+o(t)$.

Here is a justification both for the local differentiable branch and for the formula. Define $F(x,\ell,t)=(A(t)x-\ell x,\ (x^\mathsf{T}x-1)/2)$. At $(v_j,\lambda_j,0)$, $F=0$. The derivative of $F$ with respect to $(x,\ell)$, applied to $(h,\mu)$, is $((A(0)-\lambda_j I)h-\mu v_j,\ v_j^\mathsf{T}h)$. This linear map is invertible. Indeed, if its output is zero, multiplying its first component by $v_j^\mathsf{T}$ gives $-\mu=0$; the second component gives $h\perp v_j$. On that orthogonal complement, $A(0)-\lambda_j I$ is invertible because $\lambda_j$ is simple. Thus $h=0$ as well. The implicit function theorem therefore supplies a differentiable local solution $(v_j(t),\lambda_j(t))$. Continuity permits the stated local orientation.

Differentiate $A(t)v_j(t)=\lambda_j(t)v_j(t)$ at zero and write $w=v_j'(0)$, $E=A'(0)$. The result is $$ E v_j+A(0)w=\lambda_j'(0)v_j+\lambda_j w. $$ Differentiating $v_j(t)^\mathsf{T}v_j(t)=1$ gives $v_j^\mathsf{T}w=0$. Now multiply the differentiated eigen-equation on the left by $v_k^\mathsf{T}$, for $k\ne j$. Since $v_k^\mathsf{T}A(0)=\lambda_k v_k^\mathsf{T}$ and $v_k^\mathsf{T}v_j=0$, $$ v_k^\mathsf{T}E v_j+\lambda_k v_k^\mathsf{T}w =\lambda_j v_k^\mathsf{T}w. $$ Consequently, $v_k^\mathsf{T}w=(v_k^\mathsf{T}E v_j)/(\lambda_j-\lambda_k)$. The vectors $v_k$, $k\ne j$, span the space orthogonal to $v_j$, and $w$ lies in that space. Expanding $w$ in this basis proves the formula. $\square$

Reading the Formula: Direction and Size

Each coefficient describes mixing between the target eigenvector and another eigenvector. The numerator $v_k^\mathsf{T}E v_j$ measures how much the perturbation connects those two directions. A perturbation that is diagonal in the eigenbasis has no such off-diagonal terms and therefore causes no first-order change in eigenvector direction. The denominator records the signed separation between the corresponding eigenvalues: a small gap allows a larger rotation for the same numerator.

This interpretation yields a useful norm bound. Use the Euclidean vector norm and its induced matrix operator norm, denoted by $\|\cdot\|_2$. Since the terms in the eigenvector expansion are orthogonal,

\begin{aligned} \|v_j'(0)\|_2^2 &=\sum_{k\ne j}\frac{|v_k^\mathsf{T}E v_j|^2}{|\lambda_j-\lambda_k|^2}\\ &\le \frac{1}{\delta_j^2}\sum_{k\ne j}|v_k^\mathsf{T}E v_j|^2\\ &\le \frac{\|E v_j\|_2^2}{\delta_j^2} \le \frac{\|E\|_2^2}{\delta_j^2}. \end{aligned}

For the second inequality, the sum is the squared norm of the projection of $E v_j$ onto the space perpendicular to $v_j$, which cannot exceed $\|E v_j\|_2^2$. Taking square roots proves $\|v_j'(0)\|_2\le \|E\|_2/\delta_j$. The bound is conservative: a small gap alone does not guarantee high sensitivity, because the relevant numerator may be zero. It identifies a possible amplification, not a claim that every perturbation rotates the eigenvector substantially.

Worked Example: A Two-Dimensional Rotation

Let $A=\begin{pmatrix}5&0\\0&2\end{pmatrix}$ and $E=\begin{pmatrix}0&1\\1&0\end{pmatrix}$. Find the first-order change in the eigenvector for the eigenvalue $5$.

The target eigenpair is $(\lambda_1,v_1)=(5,e_1)$, and the other eigenpair is $(2,e_2)$. The eigenvector formula has one term: $$ v_1'(0)=\frac{e_2^\mathsf{T}E e_1}{5-2}e_2 =\frac{1}{3}e_2. $$ Thus the unit eigenvector initially moves toward $e_2$, and $v_1(t)=e_1+(t/3)e_2+o(t)$. To check the direction directly, the perturbed matrix is $\begin{pmatrix}5&t\\t&2\end{pmatrix}$. Its top eigenvector can be written $(\cos\theta,\sin\theta)^\mathsf{T}$, where $\tan(2\theta)=2t/3$ near $t=0$. Therefore $\theta'(0)=1/3$, agreeing with the perturbation formula.

Worked Example: A Perturbation That Changes Eigenvalues but Not Directions

Let $A=\begin{pmatrix}4&0\\0&1\end{pmatrix}$ and $E=\begin{pmatrix}2&0\\0&-1\end{pmatrix}$. Consider the eigenvector for the eigenvalue $4$.

The unperturbed eigenvectors are $e_1,e_2$, and the only coefficient in the derivative is $$ \frac{e_2^\mathsf{T}E e_1}{4-1}=0, $$ because $E e_1=2e_1$ is orthogonal to $e_2$. Hence $v_1'(0)=0$. In fact, $A+tE$ is diagonal, with eigenvalues $4+2t$ and $1-t$; for sufficiently small $t$, its eigenvectors remain $e_1,e_2$. This example shows why the numerator matters as well as the gap: the eigenvalues move, but the directions do not.

Worked Example: Several Competing Directions

Let $A=\operatorname{diag}(6,4,1)$, target the eigenvalue $4$, and set $E=\begin{pmatrix}0&2&0\\2&0&3\\0&3&0\end{pmatrix}$. Calculate the derivative of the unit eigenvector $e_2$.

The gaps from $4$ to the other eigenvalues are $4-6=-2$ and $4-1=3$. The two numerators are $e_1^\mathsf{T}E e_2=2$ and $e_3^\mathsf{T}E e_2=3$, so $$ v_2'(0)=\frac{2}{-2}e_1+\frac{3}{3}e_3=-e_1+e_3. $$ The derivative has norm $\sqrt{2}$. The nearest gap is $\delta_2=2$. For a check of the general bound, $E$ has eigenvalues $0,\sqrt{13},-\sqrt{13}$, so $\|E\|_2=\sqrt{13}$. Thus $\sqrt{2}\le \sqrt{13}/2$, as required. The opposite signs of the two coefficients follow from the signed denominators, not from a normalization convention.

Worked Example: A Small Gap Can Amplify a Directional Change

Take $A=\operatorname{diag}(2.1,2,0)$, target $\lambda_2=2$ with eigenvector $e_2$, and let $E_{12}=E_{21}=1$, with all other entries zero. The gaps are $2-2.1=-0.1$ and $2-0=2$. The only nonzero numerator is $e_1^\mathsf{T}E e_2=1$, giving $$ v_2'(0)=\frac{1}{-0.1}e_1=-10e_1. $$ The derivative norm is $10$, even though the perturbation has operator norm $1$. This is consistent with the bound $\|E\|_2/\delta_2=1/0.1=10$. The calculation describes the local derivative: it does not imply that a finite perturbation of arbitrary size produces a rotation of $10t$ radians.

Common Mistake: Ignoring Sign and Normalization

A common error is to differentiate the eigen-equation and then treat $v_j'(0)$ as though it could have a component along $v_j$. Differentiating unit length rules that out: $v_j^\mathsf{T}v_j'(0)=0$. Another error is to compare eigenvectors from two separate computations without fixing their signs. If one numerical routine returns $v_j$ and another returns $-v_j$, their raw difference can look large even when they represent the same direction. Orienting locally by positive inner product removes that artificial discrepancy; the derivative formula then describes the actual first-order directional change.

The formula also requires a simple target eigenvalue. If the target is repeated, an individual eigenvector within its eigenspace is not uniquely determined, so the displayed denominators cannot be used for that target. The question must instead concern the eigenspace or a chosen basis with additional structure. For a simple eigenvalue, the gap gives a local scale for sensitivity, while the perturbation’s entries in the eigenbasis determine which directions are activated.

Key takeaway: For a simple eigenvalue, the first-order change in its normalized eigenvector is an orthogonal sum of other eigenvectors. Each coefficient is the perturbation’s cross-direction component divided by the signed eigenvalue gap. Small gaps can amplify directional sensitivity, but only when the perturbation couples the relevant directions.

Check Your Understanding

Use the formula and the role of the spectral gap to answer these questions.

For a symmetric perturbation $E$, what condition makes the first-order derivative of an eigenvector $v_j$ equal to zero?
If $v_k^\mathsf{T}E v_j$ is fixed and nonzero, what happens to the magnitude of its contribution as $|\lambda_j-\lambda_k|$ decreases?
Why must the derivative of a unit eigenvector be orthogonal to the eigenvector itself?
For $A=\operatorname{diag}(7,3)$, $E_{12}=E_{21}=2$, and target eigenvector $e_1$, calculate $v_1'(0)$.
Why can comparing two eigenvectors without aligning their signs falsely suggest a large change in direction?