Histogram Bias Under Hölder Smoothness

Why Smoothness Controls Histogram Bias

A regular-bin histogram represents a density by a constant on each bin. Even if the data set were infinitely large, this representation would generally not reproduce a density that varies within a bin. That discrepancy is the histogram’s approximation bias. The smoother the density and the narrower the bins, the smaller this error should be.

This tutorial makes that statement precise for densities satisfying a Hölder condition of order $0<\alpha\leq 1$. The main result bounds the integrated squared approximation bias by a constant times $h^{2\alpha}$, where $h$ is the bin width. The bound concerns the expected histogram, not the random fluctuations due to a finite sample.

Histogram Approximation and Hölder Smoothness

Let the density $f$ be supported on a fixed interval $I=[a,b]$, whose length is $D=b-a$. Partition $I$ into $m$ equal bins $B_1,\ldots,B_m$, each of width $h=D/m$. Endpoints shared by adjacent bins have probability and Lebesgue measure zero, so their assignment does not affect the calculations. For an independent sample of size $n$, the regular-bin histogram has height $N_j/(nh)$ on bin $B_j$, where $N_j$ is the number of observations in that bin. Since $E[N_j]=n\int_{B_j}f(x)\,dx$, its expected value on that bin is the average of $f$ over the bin.

Definition: The histogram approximation $f_h$ is the function that is constant on each bin $B_j$, with value $$ f_h(x)=\frac{1}{h}\int_{B_j} f(y)\,dy,\qquad x\in B_j. $$ It is the expected regular-bin histogram. Its integrated squared approximation bias is $$ \int_I \bigl(f_h(x)-f(x)\bigr)^2\,dx. $$ A density satisfies a Hölder condition of order $\alpha$ on $I$, with constant $L$, if $0<\alpha\leq 1$, $L<\infty$, and $$ |f(x)-f(y)|\leq L|x-y|^\alpha \quad\text{for all }x,y\in I. $$

When $\alpha=1$, this is the Lipschitz condition. For $0<\alpha<1$, it allows less regular functions while still limiting how quickly their values can change over short distances. The bound below needs only this condition; it does not require a derivative.

A Binwise Identity

The first step is an identity for any square-integrable function on one bin. It explains why the squared error from replacing a function by its bin average can be controlled by pairwise differences within the bin.

Result: If $B$ is an interval of width $h$, and $\bar f_B=h^{-1}\int_B f(x)\,dx$, then $$ \int_B \bigl(f(x)-\bar f_B\bigr)^2\,dx = \frac{1}{2h}\int_B\int_B \bigl(f(x)-f(y)\bigr)^2\,dy\,dx. $$

To verify the identity, expand the double integral. Since both integrations are over the same bin, $$ \int_B\int_B (f(x)-f(y))^2\,dy\,dx = 2h\int_B f(x)^2\,dx-2\left(\int_B f(x)\,dx\right)^2. $$ Dividing by $2h$ gives $\int_B f(x)^2\,dx-h^{-1}(\int_B f(x)\,dx)^2$. Expanding the left side of the claimed identity gives the same expression: $$ \int_B(f(x)-\bar f_B)^2\,dx = \int_B f(x)^2\,dx-\frac{1}{h}\left(\int_B f(x)\,dx\right)^2. $$ Thus the identity holds. $\square$

The Integrated Squared-Bias Bound

Theorem: Suppose $f$ is a density supported on $I=[a,b]$, with $D=b-a$, and satisfies the Hölder condition of order $0<\alpha\leq1$ with constant $L$. For a regular partition of $I$ into bins of width $h=D/m$, the histogram approximation satisfies $$ \int_I \bigl(f_h(x)-f(x)\bigr)^2\,dx \leq \frac{L^2D}{(2\alpha+1)(2\alpha+2)}h^{2\alpha}. $$

Consider one bin $B$. Apply the binwise identity and then the Hölder condition to its integrand: $$ \int_B(f(x)-\bar f_B)^2\,dx \leq \frac{L^2}{2h}\int_B\int_B |x-y|^{2\alpha}\,dy\,dx. $$ By translating the bin, the double integral is the same as the corresponding integral over $[0,h]$. Splitting at $y=x$ and integrating gives $$ \int_0^h\int_0^h |x-y|^{2\alpha}\,dy\,dx = 2\int_0^h\int_0^x (x-y)^{2\alpha}\,dy\,dx = \frac{2h^{2\alpha+2}}{(2\alpha+1)(2\alpha+2)}. $$ Therefore the contribution of this bin is at most $$ \frac{L^2 h^{2\alpha+1}}{(2\alpha+1)(2\alpha+2)}. $$ There are $m=D/h$ bins, and the integrated squared bias is the sum of their contributions. Multiplying the binwise bound by $D/h$ proves the theorem. $\square$

For fixed $D$, $L$, and $\alpha$, the result is an $O(h^{2\alpha})$ upper bound as the bin width decreases. It is an upper bound, not a claim that every density has bias of exactly this order: a constant density, for instance, has zero approximation bias for every partition. The dependence on sample size $n$ does not appear because this is the bias of the expected histogram.

Worked Examples

Worked Example: A Uniform Density Has No Approximation Bias

Let $f(x)=1$ on $[0,1]$, and divide the interval into $m$ regular bins of width $h=1/m$. This is a density because $\int_0^1 1\,dx=1$.

On every bin $B_j$, the average is $$ \frac{1}{h}\int_{B_j}1\,dy=\frac{h}{h}=1. $$ Thus $f_h(x)=1=f(x)$ throughout the interval, and $$ \int_0^1(f_h(x)-f(x))^2\,dx=0. $$ The density satisfies the Hölder condition for any $0<\alpha\leq1$, since $|f(x)-f(y)|=0$. The theorem’s nonnegative upper bound is valid but not tight in this case.

Worked Example: A Linear Density Attains the Bound

Let $f(x)=2x$ on $[0,1]$. It is a density because $\int_0^1 2x\,dx=1$, and it is Hölder of order $\alpha=1$ with $L=2$, since $|2x-2y|=2|x-y|$. Use $m$ regular bins of width $h=1/m$.

For bin $B_j=[(j-1)h,jh]$, the bin average is $$ \frac{1}{h}\int_{(j-1)h}^{jh}2x\,dx = \frac{j^2h^2-(j-1)^2h^2}{h} = (2j-1)h. $$ The midpoint of the bin is $(j-\tfrac12)h$, and $2(j-\tfrac12)h=(2j-1)h$. Hence, writing $c_j=(j-\tfrac12)h$, the difference between the approximation and the density on this bin is $2c_j-2x$. Its integrated square is $$ \int_{B_j}(2c_j-2x)^2\,dx = 4\int_{-h/2}^{h/2}u^2\,du = \frac{h^3}{3}. $$ Summing over $m=1/h$ bins gives total integrated squared bias $h^2/3$. The theorem gives $$ \frac{L^2D}{(2\alpha+1)(2\alpha+2)}h^{2\alpha} = \frac{4}{3\cdot4}h^2 = \frac{h^2}{3}. $$ So this density attains the bound exactly.

Worked Example: A Square-Root Density With Hölder Order One-Half

Let $f(x)=\tfrac32\sqrt{x}$ on $[0,1]$. Its integral is $\tfrac32\cdot\tfrac23=1$, so it is a density. For nonnegative $x,y$, $$ |\sqrt{x}-\sqrt{y}|\leq\sqrt{|x-y|}, $$ as follows, for example, by squaring and using $(\sqrt{x}-\sqrt{y})^2\leq|x-y|$. Thus $f$ satisfies a Hölder condition with $\alpha=\tfrac12$ and $L=\tfrac32$. Divide $[0,1]$ into two bins, so $h=\tfrac12$.

The mass in the first bin is $\int_0^{1/2}\tfrac32\sqrt{x}\,dx=(1/2)^{3/2}=1/(2\sqrt2)$, giving average height $1/\sqrt2$. The mass in the second bin is $1-1/(2\sqrt2)$, giving average height $2-1/\sqrt2$.

For a bin with average $c$, expanding the square shows that $\int_B(f-c)^2=\int_B f^2-hc^2$. Here $\int_0^1 f^2=\int_0^1 \tfrac94x\,dx=9/8$, while the sum of $hc^2$ over the two bins is $$ \frac12\left(\frac12+\left(2-\frac{1}{\sqrt2}\right)^2\right) = \frac52-\sqrt2. $$ The integrated squared bias is therefore $$ \frac98-\left(\frac52-\sqrt2\right) = \sqrt2-\frac{11}{8} \approx 0.0392. $$ The theorem’s bound is $\frac{(3/2)^2}{(2)(3)}(1/2)=3/16=0.1875$, which is larger, as required.

Interpreting the Bound and Avoiding a Common Mistake

The bound describes the error from using one constant per bin to represent a changing density. Hölder smoothness limits the variation of $f$ within each bin, and the binwise identity converts that variation into an integrated squared-error bound. Narrower bins improve this approximation bound at the rate $h^{2\alpha}$; smaller $\alpha$ permits rougher densities and gives a slower power of $h$.

A common mistake is to read this theorem as a bound on the integrated squared error of the random histogram itself. That error also includes sampling variability: the observed bin counts need not equal their expectations. Here $f_h$ is the expected histogram, so the result concerns only $\int_I(f_h-f)^2$. The theorem does not bound the random estimator’s error without an additional analysis of its variability.

Another tempting shortcut is to say that the histogram error is “about $Lh^\alpha$” and omit the distinction between pointwise and integrated error. A pointwise deviation bound, even when available, must be squared and integrated over the domain to yield an integrated squared-bias bound. The proof above carries out that integration bin by bin and supplies the resulting constant and dependence on $D$.

Key takeaway: For a density on a fixed interval satisfying a Hölder condition of order $0<\alpha\leq1$, the expected regular-bin histogram has integrated squared approximation bias at most $$ \frac{L^2D}{(2\alpha+1)(2\alpha+2)}h^{2\alpha}. $$ This is an approximation-bias statement, separate from the random sampling variation of the histogram.

Check Your Understanding

Use the definitions and derivation above to answer these questions.

Why is the expected height of the histogram on a bin equal to the average of the density over that bin?
State the Hölder condition used in the theorem and identify the allowed range of $\alpha$.
For a fixed interval and Hölder constant, how does the theorem’s bias bound change when $h$ is halved?
Why does the uniform density have zero approximation bias even though the general theorem gives a positive upper bound?
Explain why the integrated squared bias bound is not, by itself, a bound on the random histogram’s integrated squared error.