Sample Size Planning for Principal Directions

Why Plan for a Principal Direction?

A sample eigenvector is an estimate of a population direction, but its accuracy depends on more than the number of observations. In particular, a direction is difficult to estimate when its eigenvalue is close to a neighboring eigenvalue: the data then provide relatively little separation between the corresponding directions. Sample-size planning makes this dependence explicit, so that a target angular accuracy can be related to eigenvalue gaps and sampling variability.

The calculations below are first-order approximations, not guarantees of finite-sample performance. They are useful for comparing designs and establishing a starting sample size. A small estimated gap, a heavy-tailed distribution, or a required high probability of success may call for a more cautious plan.

First-Order Angular Error

Let \(X_1,\ldots,X_n\) be independent and identically distributed random vectors in fixed dimension \(p\), with mean \(\mu\), covariance matrix \(\Sigma\), and finite fourth moments. Write the eigenvalues of \(\Sigma\) in decreasing order as \(\lambda_1\ge\cdots\ge\lambda_p\), with corresponding orthonormal eigenvectors \(u_1,\ldots,u_p\). Fix an index \(r\) for which \(\lambda_r\) is simple: \(\lambda_r\ne\lambda_s\) for every \(s\ne r\). Let \(\hat u_r\) be a unit eigenvector of the sample covariance matrix for the corresponding sample eigenvalue.

Because an eigenvector and its negative represent the same direction, use the sign-invariant angle \(\alpha_r=\arccos(|\hat u_r^\top u_r|)\), which lies between zero and \(\pi/2\). The first-order displacement of \(\hat u_r\) lies in directions orthogonal to \(u_r\). Its coordinate in direction \(u_s\) is driven by the sample covariance entry connecting \(u_s\) and \(u_r\), divided by their population eigenvalue gap.

Deriving the Gap-Weighted Variance

The key calculation is an eigenvector perturbation. Let \(\hat\Sigma\) be the sample covariance and write \(E_n=\hat\Sigma-\Sigma\). For a symmetric matrix with a simple eigenvalue, the first derivative of its unit eigenvector, with sign chosen so that \(\hat u_r^\top u_r\ge 0\), has the form \(\sum_{s\ne r}u_s(u_s^\top E_nu_r)/(\lambda_r-\lambda_s)\). Here is the derivation that identifies both the gap and the random quantity in the formula.

For the sample covariance, the entry \(u_s^\top E_nu_r\) is, to first order, the sample average of centered products. In detail, put \(Y_{is}=u_s^\top(X_i-\mu)\). The covariance entry in the population eigenbasis is the sample covariance of the \(Y\)'s, and its population value is zero for \(s\ne r\). Estimating the mean rather than using \(\mu\) changes this entry only by a product of sample-mean errors, which is of smaller order than \(n^{-1/2}\). Therefore \[ \sqrt n\,u_s^\top E_nu_r =\frac{1}{\sqrt n}\sum_{i=1}^n Y_{is}Y_{ir}+o_p(1). \]

Each product has mean zero because \(u_s^\top\Sigma u_r=0\). The multivariate central limit theorem applies to the vector of these products: its coordinates have finite second moments because \(X\) has finite fourth moments. The eigenvector perturbation formula is differentiable at a simple eigenvalue, so the delta method gives the following asymptotic result.

To see the variance calculation directly, each summand \(Y_{is}Y_{ir}\) has mean zero and variance \(v_{sr}\). The coefficient along \(u_s\) consequently has asymptotic variance \(v_{sr}/(\lambda_r-\lambda_s)^2\). Since the \(u_s\)'s are orthonormal, the squared length of the tangent displacement is the sum of the squared coordinates; covariance between distinct coordinates does not contribute to that squared length's expectation. For small displacements, angular error equals tangent length to first order, since \(\sin(\alpha_r)\approx\alpha_r\).

The theorem is a distributional, first-order statement. Using \(A_r/n\) as an approximation to the actual finite-sample mean-square error additionally relies on the usual mean-square regularity of the approximation; convergence in distribution alone does not establish convergence of second moments. Sample-size calculations below are explicitly asymptotic planning approximations.

Planning for a Root-Mean-Square Target

Suppose the desired root-mean-square angular error is at most \(\delta\) radians. Substituting the first-order approximation gives \(\sqrt{A_r/n}\le\delta\), or \(n\ge A_r/\delta^2\). The smallest integer meeting this approximate threshold is the ceiling of the quotient. Angles in these formulas are in radians.

The Gaussian simplification is useful but not universal. In general, the variance of a product of principal-coordinate scores depends on their joint fourth moments, not just on their variances. A plan based on eigenvalues alone can therefore misrepresent uncertainty when the data are not Gaussian.

Gaps and Probability-Based Targets

Every contribution to \(A_r\) is divided by a squared eigenvalue gap. This means a neighboring eigenvalue can dominate the plan even if other eigenvalues are well separated. In addition to RMS planning, a design may specify a probability target, such as requiring the angle to be below \(\delta\) with probability about \(0.95\). The asymptotic distribution can translate that request into a sample size, provided the limiting covariance is known or credibly estimated.

In two dimensions there is only one tangent coordinate. Under the asymptotic normal approximation, the sign-invariant angle is approximately the absolute value of a mean-zero normal variable with variance \(A_r/n\). A two-sided probability target can then be planned using a normal quantile. In higher dimensions, the tangent vector is multivariate normal in the limit, and the probability target depends on the distribution of its norm; it is not generally obtained by applying a one-dimensional normal quantile to the RMS.

Assumptions, Pitfalls, and Practical Use

These formulas assume a fixed dimension, independent observations from a stable population, a simple target eigenvalue, and finite fourth moments for the asymptotic covariance calculation. They do not directly handle a repeated eigenvalue: within a repeated eigenspace, an individual eigenvector is not uniquely defined, and its gap denominator vanishes. In that setting the estimand must be an eigenspace rather than a single direction.

• Using the gap instead of its square. The first-order coordinate error divides by \(\lambda_r-\lambda_s\), so its variance divides by \((\lambda_r-\lambda_s)^2\).
• Assuming Gaussian product variances for all data. Outside the Gaussian case, calculate or estimate \(v_{sr}\) from fourth moments or use a justified model for them.
• Treating an RMS target as a probability guarantee. The RMS criterion and the requirement of a specified probability within an angular tolerance are not interchangeable.
• Ignoring uncertainty in planning inputs. Population gaps and fourth moments are usually unknown. Pilot estimates can be unstable, especially near a small gap; sensitivity calculations across plausible values are prudent.
• Reading a first-order threshold as an exact sample size. Approximation error, finite-sample behavior, and model misspecification may matter. A safety margin or simulation under plausible models can be appropriate.

The practical value of the calculation is its diagnostic clarity: it separates the role of the eigenvalue geometry from the role of sampling variation. A direction with a large gap and modest product variance may require relatively few observations, while a near-tie can demand far more. Reporting the assumed gaps, variance coefficient, angular tolerance, and planning criterion makes the resulting sample-size recommendation interpretable and auditable.