Inference with Few Spatial Units or Networks

Why the Number of Independent Units Matters

Spatial and network data can contain many observations but little independent information. A data set may record thousands of people, locations, or network nodes while dependence is concentrated within only a few regions or networks. In that setting, the effective amount of information for inference may be closer to the number of independent units than to the number of rows in the data.

Spatial HAC and network-robust covariance estimators address dependence by estimating covariance from spatially or network-related observations. Their usual justifications rely on large-sample conditions: for example, an expanding spatial domain with sufficiently weak dependence, or a network with enough nodes and suitably controlled dependence. When the number of independent regions or networks is small, these approximations can be poor even if each region contains many observations. A standard error that looks precise is not evidence that the required independent information is present.

The independence claim must match the data-generating structure. If shocks spill across the boundaries used to define regions, those regions are not independent merely because they have different labels. A network partition can fail for the same reason when important links cross between networks. The earlier material on Spatial HAC and Network-Robust Standard Errors develops covariance estimators for dependence; here the question is whether their large-sample approximation has enough independent units to work well.

What Fails When There Are Few Units?

Consider the linear model \(y=X\beta+u\), where \(y\) is an \(n\)-vector, \(X\) is an \(n\times k\) regressor matrix of full column rank, and \(u\) is an \(n\)-vector of disturbances. Suppose observations are partitioned into \(G\) independent clusters, with arbitrary dependence allowed within each cluster. For the cluster-robust covariance estimator, define the cluster score \(S_g=X_g'\hat{u}_g\), where \(X_g\) and \(\hat{u}_g\) contain the rows and OLS residuals for cluster \(g\). The uncorrected sandwich estimator is

This estimator relies on the cluster scores providing an adequate approximation to the sampling variation of the estimator. A central limit argument typically needs the number of independent score contributions to grow, along with conditions controlling their size and dependence. If \(G\) stays small, the sum contains only a few independent contributions; increasing the number of observations within each cluster does not make that sum contain more independent clusters.

This is the central finite-sample problem. The estimated covariance can be downward biased, and the usual normal approximation for a coefficient statistic can have rejection rates far from the intended significance level. Unequal cluster sizes, high leverage, and a few influential cluster scores can make the problem more severe. Neither a large observation count nor a small reported standard error resolves these concerns.

Finite-Sample Adjustments and Degrees of Freedom

A common adjustment multiplies the uncorrected cluster covariance estimator by a degrees-of-freedom factor. One conventional form is

The factor is greater than one when \(G>1\) and \(n>k\), so it inflates the estimated covariance. This correction is simple and widely reported, but it is not an exact solution to few-cluster inference. In particular, a general cluster-robust test should not automatically use a standard normal reference distribution. A common small-sample practice is to compare a studentized statistic with a \(t\) distribution using \(G-1\) degrees of freedom. This is a useful convention, not a universal finite-sample theorem: performance can still be poor when clusters are highly unequal or influential.

For a scalar contrast \(c'\beta\), the estimated standard error is \(\sqrt{c'\widehat{V}c}\), and the statistic for testing \(H_0:c'\beta=r\) is

More refined corrections, such as CR2, adjust residual contributions to account for leverage and use a contrast-specific degrees-of-freedom approximation, often of Satterthwaite type. These can improve small-sample behavior under their stated working assumptions. The adjustment and degrees of freedom depend on the design and the contrast; there is no single CR2 multiplier or degrees-of-freedom value appropriate for every spatial or network problem. A researcher should report which correction and reference distribution were used, rather than describing a generic “robust” standard error as automatically reliable.

Look for Concentrated Cluster Information

The count \(G\) alone does not reveal whether the information is spread evenly across clusters. One useful descriptive check is to inspect the cluster score contributions for the coefficient or contrast of interest. If a small number of contributions dominate, the effective information may be much more limited than the nominal cluster count suggests. A simple concentration summary is

where \(q_g\) are nonnegative magnitudes of the relevant cluster contributions. This summary equals \(G\) when all magnitudes are equal and decreases when contributions are concentrated. It is a diagnostic, not a formal replacement for degrees of freedom: it does not by itself determine a valid critical value or p-value.

Resampling: Match the Resampling Unit to the Dependence

Resampling can approximate a finite-sample reference distribution, but it must preserve the dependence structure that motivates the covariance estimator. Resampling individual observations independently is generally inappropriate when observations within a region or network are dependent. Two approaches are commonly considered, with different requirements.

• Wild cluster bootstrap: retain the observed regressors and cluster structure, impose the null hypothesis when generating bootstrap outcomes for a test, and multiply cluster-level residual contributions by random weights. Refit the model and recompute the studentized statistic in each replication. This preserves within-cluster patterns in the residuals while varying contributions across clusters. Null-imposed, studentized versions are often preferred for testing, but bootstrap refinements do not guarantee accurate inference with arbitrarily few or highly influential clusters.
• Spatial block resampling: resample blocks of spatial units rather than isolated observations when dependence extends across nearby locations. Blocks must be constructed so that the resampling scheme reasonably represents the dependence and provides enough distinct blocks. Very large blocks leave too few resampling units; very small blocks can break important dependence. With only a few regions, there may be no block size that makes the approximation convincing.

A resampling method is not automatically valid because it produces many replications. The number of replications reduces Monte Carlo error in estimating the chosen bootstrap distribution; it does not repair a mismatch between that distribution and the actual sampling process. If regions share substantial cross-boundary dependence, a cluster bootstrap that treats them as independent may remain invalid. Likewise, exact randomization inference is justified only when the assignment mechanism or an appropriate exchangeability assumption supports the permutations being used.

A Practical Inference Checklist

Common Mistakes and Why the Issue Matters

• Counting observations as independent units. A study with 5,000 people in five regions does not have 5,000 independent regional shocks. Inference about region-level dependence may be governed mainly by five regions.
• Treating a correction as a cure. Multiplying a covariance matrix by a finite-sample factor adjusts its scale; it does not establish independence, eliminate high leverage, or make a normal approximation exact.
• Resampling at the wrong level. Independently resampling people can destroy within-region dependence. Resampling regions is also questionable if the regions are not independent.
• Calling a diagnostic a degrees-of-freedom rule. A concentration measure such as \(G_{\mathrm{eff}}\) can flag imbalance, but it does not specify a valid test distribution.

Few-unit inference matters because policy or scientific conclusions can depend heavily on whether uncertainty is understated. The sound response is not to search for one universally correct adjustment. It is to make the independent-unit assumptions explicit, diagnose the concentration of information, use a finite-sample method suited to the design, and report the remaining limitations. When methods disagree, that disagreement is substantive evidence that the data provide limited support for precise inference.