Projection Onto A Bounded Parameter Space

Why Project an Estimate?

An estimator can take values that the parameter it estimates cannot take. A proportion, for example, is in \([0,1]\), but an estimator constructed from noisy data might fall below zero or above one. If the parameter is known to lie in a bounded interval, we can replace such an estimate by the nearest point in that interval. Under squared-error loss, this simple operation has a strong guarantee: for every possible estimate and every parameter value in the interval, the replacement is no farther from the parameter.

The guarantee is pointwise, before any averaging over the data. Consequently, it does not depend on unbiasedness, a particular sampling distribution, or a calculation of the estimator’s bias. The assumption that the true parameter lies in the interval is essential.

Projection Onto an Interval

This projection is the point in the interval closest to \(t\): if \(t\) is already in the interval, it stays unchanged; if it is below or above the interval, it is moved to the nearer endpoint. The map \(t\mapsto\Pi_{[a,b]}(t)\) is continuous, so if \(T\) is a measurable estimator, then \(\Pi_{[a,b]}(T)\) is also measurable.

For a scalar estimator \(T\) of a parameter \(\theta\), squared-error loss at a realized estimate \(t\) is \((t-\theta)^2\). Its risk under a model indexed by \(\theta\) is the expected loss, \(R(\theta,T)=E_\theta[(T-\theta)^2]\), when this expectation is finite. The central fact is that projection reduces the loss for each realized value, not merely on average.

Proof. There are three cases. If \(a\le t\le b\), the projection equals \(t\), so the losses are equal. If \(t<a\), then \(t<a\le\theta\), and \[ 0\le \theta-a < \theta-t. \] Both sides are nonnegative, so squaring gives \((a-\theta)^2<(t-\theta)^2\). Since \(\Pi_{[a,b]}(t)=a\), this is the claimed strict inequality. If \(t>b\), then \(\theta\le b<t\), and \[ 0\le b-\theta < t-\theta. \] Squaring gives \((b-\theta)^2<(t-\theta)^2\), and \(\Pi_{[a,b]}(t)=b\). These cases establish the claim. \(\square\)

From Pointwise Loss to Risk

Apply the pointwise inequality to each possible realized value of an estimator \(T\). This gives a risk comparison after taking expectations. The distinction between the pointwise statement and the risk statement is useful: the former needs no probability calculation, while the latter averages that same inequality over the sampling distribution.

Proof. For every outcome, substitute the realized value \(t=T\) into the pointwise projection inequality. Both losses are nonnegative measurable random variables, so taking expectations preserves their order. This proves the non-strict risk inequality; finiteness of \(R(\theta,T)\) also ensures the projected risk is finite.

For strictness, define the loss reduction \[ D=(T-\theta)^2-(\Pi_{[a,b]}(T)-\theta)^2. \] The pointwise result gives \(D\ge0\), and gives \(D>0\) whenever \(T\notin[a,b]\). Thus \(P_\theta(D>0)>0\). Since \(\{D>0\}=\bigcup_{k=1}^{\infty}\{D\ge1/k\}\), at least one event \(\{D\ge1/k\}\) has positive probability. For that \(k\), \(E_\theta[D]\ge P_\theta(D\ge1/k)/k>0\). The finite-risk assumption allows us to subtract the expectations of the two losses, so \[ R(\theta,T)-R(\theta,\Pi_{[a,b]}(T))=E_\theta[D]>0. \] This proves the strict risk comparison. \(\square\)

The strictness condition matters. If \(T\) already lies in \([a,b]\) with probability one, clipping changes nothing and the risks are equal. More generally, strict improvement at a particular \(\theta\) follows whenever the estimator has positive probability of falling outside the interval under that parameter value.

What the Guarantee Does—and Does Not—Say

When the parameter is known to lie in \([a,b]\), clipping is a valid way to ensure an estimator respects that constraint without increasing squared-error risk at any parameter value in the interval. The result holds uniformly over the interval: the same clipping rule works for every \(\theta\in[a,b]\). It does not say that the projected estimator is best among all estimators, or that it must strictly improve risk if it rarely or never leaves the interval.

The theorem also does not promise improvement if the true parameter might lie outside the interval. For example, take the interval \([0,1]\), the true value \(\theta=1.2\), and the estimate \(t=1.15\). The unprojected loss is \((1.15-1.2)^2=0.0025\), while clipping to \(1\) gives loss \((1-1.2)^2=0.04\), which is larger. The premise \(\theta\in[a,b]\) is therefore not a technical detail; it is what makes the distance comparison true.

Common Mistakes

• Confusing a pointwise comparison with a risk calculation. A comparison at one observed \(t\) is not itself an expected risk. The risk result follows by applying the pointwise inequality to every possible realization and then taking expectations.
• Assuming clipping preserves unbiasedness. Clipping can change an estimator’s expectation, as the biased-estimator example shows. The risk guarantee does not require unbiasedness.
• Forgetting the parameter-space condition. Projection is guaranteed not to increase squared-error loss only when the true \(\theta\) is in the interval being used.
• Claiming strict improvement without checking where \(T\) falls. If \(T\in[a,b]\) almost surely, then projection leaves it unchanged and the risks are equal.