Pointwise Lepski Selection in a Gaussian Regression Example

Why Select a Pointwise Scale?

At a fixed location, a regression estimate built from a narrow neighborhood uses relatively few observations: its variance is high, but its local averaging bias may be small. A wider neighborhood typically reduces variance while bringing in observations whose regression means may differ from the target value. Lepski’s method makes this tradeoff visible by comparing estimates across scales. If estimates at a wide scale remain close to estimates at finer scales, the data have not revealed a discrepancy large enough to rule out the wider scale.

We work through a concrete Gaussian regression model and a finite collection of nested local averages. The calculation is pointwise: the target is the value of the regression function at one fixed location, not a simultaneous estimate over the whole domain. The example makes explicit both how the thresholds are calibrated and what it means when a scale passes or fails the comparisons.

Model, Estimators, and Selection Rule

Suppose fixed design points \(x_i=i/128\), for \(i=1,\ldots,128\), are observed under the model \(Y_i=f(x_i)+\epsilon_i\), where the errors are independent and distributed as \(N(0,\sigma^2)\). We estimate \(f(0)\), using one-sided averages of the observations nearest to zero. The available sample sizes, or scales, are \(m\in\{4,16,64\}\).

The threshold for comparing a coarser scale \(m_j\) with a finer scale \(m_i\) is indexed by the finer scale. That choice reflects the larger noise level of the finer estimate. In this finite example, there are three comparisons: \(4\) versus \(16\), \(4\) versus \(64\), and \(16\) versus \(64\).

Deriving the Gaussian Comparison Threshold

The nested structure makes the noise variance of each difference especially simple. Write \(\hat f_m(0)=m^{-1}\sum_{i=1}^{m}f(x_i)+m^{-1}\sum_{i=1}^{m}\epsilon_i\). If \(m<k\), the first \(m\) errors occur in both averages. Independence and the common variance \(\sigma^2\) give

The final inequality says that the standard deviation of the random part of a comparison is no larger than \(\sigma/\sqrt{m}\), where \(m\) is the finer scale. We use this upper bound to choose one threshold that controls all three comparisons at once.

To prove the result, fix one comparison between \(m_i<m_j\). Its random part, call it \(Z\), is centered Gaussian with variance at most \(\sigma^2/m_i\). For a centered Gaussian variable with variance \(v\), the moment generating function is \(E[\exp(tZ)]=\exp(t^2v/2)\). For \(a>0\), Markov’s inequality therefore gives \(P(Z>a)\leq \exp(-ta+t^2v/2)\) for every \(t>0\). Taking \(t=a/v\) gives \(P(Z>a)\leq\exp(-a^2/(2v))\); applying the same argument to \(-Z\) yields \(P(|Z|>a)\leq2\exp(-a^2/(2v))\). With \(a=\lambda\sigma/\sqrt{m_i}\) and \(v\leq\sigma^2/m_i\), this is at most \(2\exp(-\lambda^2/2)\). The union bound over \(M\) comparisons makes the probability that any comparison exceeds its threshold at most \(2M\exp(-\lambda^2/2)\). Substituting \(\lambda=\sqrt{2\log(2M/\delta)}\) makes this upper bound equal to \(\delta\), proving the claim. \(\square\)

Worked Comparisons for Smooth and Rough Functions

The expectation of a local average is \(m^{-1}\sum_{i=1}^{m} f(x_i)\). Its bias for estimating \(f(0)\) is therefore the difference between this local average of the regression means and \(f(0)\). The next examples use the same design, error standard deviation, candidate scales, and thresholds. Their contrasting outcomes illustrate how scale comparisons respond to the regression function near the target.

What the Selection Does and Does Not Say

These calculations illustrate the basic interpretation. In the linear example, the changes in the observed averages are small relative to the thresholds, so a broad scale is retained. In the square-root example, the local averages move substantially as the neighborhood expands, so the broad scales are rejected. This is useful because the procedure does not require the analyst to supply the regression function’s smoothness in advance; it uses observed cross-scale differences as evidence about which candidate averages are compatible.

The comparison is not a direct test that a function is smooth, and passing comparisons does not certify that the selected estimate has negligible bias. The thresholds control Gaussian fluctuations of the comparisons, while the mean differences reflect bias. A chance fluctuation can make a difference pass or fail, and a candidate set that omits suitable scales may limit the usefulness of the selection. Here the error variance is treated as known and the design is fixed; changing those assumptions requires corresponding changes to the calibration.

Common Pitfall

A frequent mistake is to use the threshold associated with the coarser scale in every comparison, or to treat the threshold as the standard deviation of the difference exactly. For nested averages the exact variance is \(\sigma^2(1/m_i-1/m_j)\), but the rule above uses the convenient upper bound \(\sigma^2/m_i\). It is the finer scale \(m_i\) that determines this bound and the stated threshold. Another mistake is to conclude that a selected wide scale proves the function is smooth. The selection only says that the observed differences did not exceed the chosen thresholds; it is a data-dependent scale decision under a specified model, not a proof of a shape property.