Triangular-Array Regression Designs

Why Triangular Arrays Matter

In many regression limits, observations are treated as independent draws from one fixed distribution. That assumption can be too restrictive: the design, error variance, or both may change with sample size. A triangular array allows the distribution of observation $i$ in a sample of size $n$ to depend on $n$, while retaining independence within each sample. The central question is whether the accumulated design and score still have stable limits.

The OLS argument has two distinct parts. The empirical Gram matrix must be well-conditioned at its chosen scale, and the regression score must satisfy a central limit condition at a compatible scale. Neither follows merely from independence or from convergence of expected quantities. The conditions below make those requirements explicit.

Model and Array Conditions

Definition: A triangular-array regression model with fixed parameter dimension $p$ consists, for each $n$, of observations $(y_{ni},x_{ni})$, $i=1,\ldots,n$, satisfying $$ y_{ni}=x_{ni}'\beta+u_{ni}, $$ where $x_{ni}\in\mathbb R^p$, $\beta\in\mathbb R^p$ is fixed, and the pairs $(x_{ni},u_{ni})$ are independent across $i$ within each row $n$. Their distributions may vary with both $i$ and $n$. Assume $E[u_{ni}\mid x_{ni}]=0$, so $E[x_{ni}u_{ni}]=0$.

Write the empirical Gram matrix, its expectation, and the regression score as

\widehat G_n=\frac{1}{n}\sum_{i=1}^n x_{ni}x_{ni}',\qquad Q_n=E[\widehat G_n],\qquad S_n=\sum_{i=1}^n x_{ni}u_{ni}.

For the usual square-root-of-sample-size limit, a useful set of conditions is:

$Q_n\to Q$, where $Q$ is positive definite.
$\widehat G_n\xrightarrow{p}Q$.
$\Omega_n=n^{-1}\sum_{i=1}^n E[u_{ni}^2x_{ni}x_{ni}']\to\Omega$, where $\Omega$ is positive definite.
For every $\varepsilon>0$, the vector Lindeberg condition holds: $$ \frac{1}{n}\sum_{i=1}^n E\!\left[\|x_{ni}u_{ni}\|^2 \mathbf{1}\{\|x_{ni}u_{ni}\|>\varepsilon\sqrt n\}\right]\longrightarrow 0. $$

The first two conditions concern the design; the last two concern the score. The Lindeberg condition rules out a non-negligible share of the score variance being carried by a few terms of order $\sqrt n$. A convenient sufficient condition for empirical Gram convergence is proved below.

Proposition: Suppose the vectors $x_{ni}$ are independent within each row, $Q_n\to Q$, and $$ \frac{1}{n^2}\sum_{i=1}^n E[\|x_{ni}\|^4]\longrightarrow 0. $$ Then $\widehat G_n\xrightarrow{p}Q$.

Proof. Fix coordinates $j,k\in\{1,\ldots,p\}$. Independence within row $n$ gives

\operatorname{Var}\!\left((\widehat G_n)_{jk}\right) =\frac{1}{n^2}\sum_{i=1}^n \operatorname{Var}(x_{ni,j}x_{ni,k}) \leq \frac{1}{n^2}\sum_{i=1}^n E[\|x_{ni}\|^4] \longrightarrow 0.

Chebyshev's inequality therefore implies $(\widehat G_n)_{jk}-(Q_n)_{jk}\xrightarrow{p}0$. Since $Q_n\to Q$, each entry of $\widehat G_n$ converges in probability to the corresponding entry of $Q$. There are finitely many entries because $p$ is fixed, so the entire matrix converges in probability. $\square$

The OLS Limit Under a Common Rate

Theorem: In the triangular-array model, suppose $Q_n\to Q$ and $\widehat G_n\xrightarrow{p}Q$ for positive definite $Q$; $\Omega_n\to\Omega$ for positive definite $\Omega$; $E[x_{ni}u_{ni}]=0$; and the vector Lindeberg condition above holds. Then OLS is defined with probability tending to one, and $$ \sqrt n(\hat\beta-\beta)\xrightarrow{d} N\!\left(0,Q^{-1}\Omega Q^{-1}\right). $$

Proof. Set $Z_{ni}=x_{ni}u_{ni}/\sqrt n$. The $Z_{ni}$ are independent within row, have mean zero, and satisfy

\sum_{i=1}^n\operatorname{Cov}(Z_{ni}) =\frac{1}{n}\sum_{i=1}^n E[u_{ni}^2x_{ni}x_{ni}'] =\Omega_n\longrightarrow\Omega.

For any fixed $a\in\mathbb R^p$, the scalar array $a'Z_{ni}$ satisfies the scalar Lindeberg condition: $|a'Z_{ni}|\leq\|a\|\|Z_{ni}\|$, so a threshold on $|a'Z_{ni}|$ implies the corresponding vector norm exceeds a fixed positive threshold (when $a\neq0$). The Lindeberg–Feller central limit theorem and the Cramér–Wold device give

\frac{S_n}{\sqrt n}=\sum_{i=1}^n Z_{ni} \xrightarrow{d}N(0,\Omega).

Because $Q$ is positive definite and $\widehat G_n\xrightarrow{p}Q$, $\widehat G_n$ is nonsingular with probability tending to one. The OLS normal equations then yield the exact identity

\sqrt n(\hat\beta-\beta) =\widehat G_n^{-1}\frac{S_n}{\sqrt n}.

Continuity of matrix inversion at $Q$ gives $\widehat G_n^{-1}\xrightarrow{p}Q^{-1}$. Slutsky's theorem now gives the stated limit, whose covariance is $Q^{-1}\Omega(Q^{-1})'=Q^{-1}\Omega Q^{-1}$, since $Q$ is symmetric. $\square$

The theorem gives a stable normalization when the limiting matrices are positive definite: the coefficient error has order $n^{-1/2}$, and its limiting covariance after multiplication by $\sqrt n$ is fixed. Equivalently, with $\Sigma=Q^{-1}\Omega Q^{-1}$, the deterministically standardized vector $\Sigma^{-1/2}\sqrt n(\hat\beta-\beta)$ converges to $N(0,I_p)$. If design and score scales do not settle to fixed $Q$ and $\Omega$, this particular normalization need not work. One must then choose a sample-size-dependent normalization based on the design and score covariance and verify a central limit condition for the correspondingly normalized score; merely substituting changing matrices into the fixed-limit formula does not establish a limit.

Worked Examples

Worked Example: Changing Error Variances in a Two-Point Design

For each even $n$, use $n/2$ observations with $x_{ni}=(1,1)'$ and $n/2$ with $x_{ni}=(1,-1)'$. Let the errors be independent, with $u_{ni}\sim N(0,4)$ for the first design point and $u_{ni}\sim N(0,1)$ for the second. Find the limiting covariance of $\sqrt n(\hat\beta-\beta)$.

The empirical Gram matrix is nonrandom, and equal representation of the two design points gives

\widehat G_n=Q_n=Q =\frac12\begin{pmatrix}1&1\\1&1\end{pmatrix} +\frac12\begin{pmatrix}1&-1\\-1&1\end{pmatrix} =\begin{pmatrix}1&0\\0&1\end{pmatrix}.

The score covariance per observation is

\Omega_n=\Omega =\frac12\,4\begin{pmatrix}1&1\\1&1\end{pmatrix} +\frac12\,1\begin{pmatrix}1&-1\\-1&1\end{pmatrix} =\begin{pmatrix}2.5&1.5\\1.5&2.5\end{pmatrix}.

Both limiting matrices are positive definite: $Q=I_2$, and the eigenvalues of $\Omega$ are $4$ and $1$. The errors are Gaussian with bounded variances and the regressors are bounded, so the Lindeberg condition holds. Thus the limiting covariance is $Q^{-1}\Omega Q^{-1}=\Omega$. The nonzero off-diagonal entry reflects the unequal error variances across the two design points.

Worked Example: Verifying Gram and Score Conditions with Varying Distributions

For a scalar regression through the origin, let $x_{ni}\sim N(0,1+i/n)$, independently within each row, and let $u_{ni}$ be independent Rademacher errors (each equals $1$ or $-1$ with probability $1/2$), independent of all regressors. Verify the conditions for the theorem.

The mean Gram value is

Q_n=\frac1n\sum_{i=1}^n(1+i/n) =1+\frac{n+1}{2n}\longrightarrow\frac32.

A centered normal variable with variance $v$ has fourth moment $3v^2$. Here $v=1+i/n\leq2$, so $E[x_{ni}^4]\leq12$, and

\operatorname{Var}(\widehat G_n) \leq\frac1{n^2}\sum_{i=1}^n E[x_{ni}^4] \leq\frac{12}{n}\longrightarrow0.

Hence $\widehat G_n\xrightarrow{p}3/2$. Also $E[u_{ni}^2]=1$, so $\Omega_n=Q_n\to3/2$. Finally $E[(x_{ni}u_{ni})^4]=E[x_{ni}^4]\leq12$. For every $\varepsilon>0$, the inequality $z^2\mathbf{1}\{|z|>\varepsilon\sqrt n\}\leq z^4/(\varepsilon^2n)$ gives

\frac1n\sum_{i=1}^n E[(x_{ni}u_{ni})^2 \mathbf{1}\{|x_{ni}u_{ni}|>\varepsilon\sqrt n\}] \leq\frac{12}{\varepsilon^2n}\longrightarrow0.

The theorem applies. The limiting variance of $\sqrt n(\hat\beta-\beta)$ is $(3/2)^{-1}(3/2)(3/2)^{-1}=2/3$. The changing regressor distributions do not prevent the usual rate because their average second moment stabilizes and no single score term dominates.

Worked Example: Gram Convergence Is Not Enough

Consider scalar OLS through the origin. For each $n$, let $x_{n1}=\sqrt n$, let $x_{ni}=1$ for $i=2,\ldots,n$, and take independent Rademacher errors $u_{ni}$. The empirical Gram matrix and score variance per observation satisfy

\widehat G_n=\frac{n+(n-1)}{n}\longrightarrow2,\qquad \Omega_n=\frac{n+(n-1)}{n}\longrightarrow2.

Nevertheless, the first normalized score term is $x_{n1}u_{n1}/\sqrt n=u_{n1}$, which does not become small. Indeed,

\frac{S_n}{\sqrt n} =u_{n1}+\frac1{\sqrt n}\sum_{i=2}^n u_{ni} \xrightarrow{d}R+Z,

where $R$ is Rademacher, $Z\sim N(0,1)$, and they are independent. This follows because the second term converges to $Z$ by the ordinary central limit theorem and is independent of the first term for each $n$. The exact OLS identity therefore gives

\sqrt n(\hat\beta-\beta)\xrightarrow{d}\frac{R+Z}{2}.

This limit has variance $1/2$, matching the sandwich covariance calculation, but is not normal: its characteristic function is $\cos(t/2)e^{-t^2/8}$, not the characteristic function $e^{-t^2/4}$ of $N(0,1/2)$. The first term violates Lindeberg because its magnitude is always $1$ after score normalization. Convergence of Gram and variance matrices alone does not guarantee a Gaussian limit.

Stable Normalization Beyond the Common-Rate Case

The square-root-of-$n$ theorem presumes that average Gram and score covariance matrices approach finite positive definite limits. A more general triangular array may have changing scales. To make the normalization explicit, define $M_n=\sum_i E[x_{ni}x_{ni}']$ and $W_n=\sum_i E[u_{ni}^2x_{ni}x_{ni}']$, assuming both are positive definite. The deterministic sandwich covariance for the approximation $M_n^{-1}S_n$ is $V_n=M_n^{-1}W_nM_n^{-1}$. A candidate stable normalization is $V_n^{-1/2}$, not automatically $\sqrt n$.

For example, if the normalized summands $V_n^{-1/2}M_n^{-1}x_{ni}u_{ni}$ satisfy a vector Lindeberg condition and their covariance sums to $I_p$, their sum converges to $N(0,I_p)$. To transfer that limit to OLS, the empirical Gram matrix $\widehat M_n=\sum_i x_{ni}x_{ni}'$ must also be accurate enough that

V_n^{-1/2}(\widehat M_n^{-1}-M_n^{-1})S_n\xrightarrow{p}0.

This formulation separates score normalization from the approximation error caused by using an empirical rather than expected Gram matrix. Under the common-rate conditions, the simpler $\sqrt n$ theorem verifies these tasks through Gram convergence, score convergence, and Slutsky's theorem.

Common Pitfall and Why the Conditions Matter

A frequent error is to see $n^{-1}\sum_i E[x_{ni}x_{ni}']\to Q$ and $n^{-1}\sum_i E[u_{ni}^2x_{ni}x_{ni}']\to\Omega$, then conclude that OLS is asymptotically normal. Those are statements about averages and second moments; they do not prevent an individual score term from remaining large after normalization. The high-leverage example has both matrix limits but a nonnormal limit. The correction is to verify the score central limit condition, such as Lindeberg, as well as empirical Gram convergence. A triangular-array argument is reliable only when its design concentration, score limit, and normalization all use compatible scales.

Key takeaway: In independent triangular-array regression, OLS inherits a normal limit when the empirical Gram matrix stabilizes, the normalized score obeys a central limit theorem, and the chosen normalization matches both. Expected matrices alone are not enough.

Check Your Understanding

Use the definitions and limit arguments above to answer these questions.

Why does the triangular-array setup allow observation distributions to change with $n$, and what independence assumption is used in the Gram convergence proof?
State a moment condition that ensures $\widehat G_n-Q_n\xrightarrow{p}0$, and identify the inequality used to prove it.
In the common-rate theorem, where does the positive definiteness of $Q$ enter the OLS argument?
What does the Lindeberg condition rule out, and why do convergent Gram and score covariance matrices not rule it out by themselves?
For the changing-variance two-point design, explain why the limiting covariance has a nonzero off-diagonal entry.