Transformed Intervals Near Probability Boundaries

Why Probability Boundaries Need Attention

For a binomial proportion, the parameter $p$ is restricted to $[0,1]$. Near zero or one, an interval constructed on the original proportion scale can behave poorly: the usual Wald interval has estimated standard error $\sqrt{\hat p(1-\hat p)/n}$, which is zero when all observations fall in one category. An arcsine-based interval uses a transformed scale and, after back-transformation, can give a nonzero range even when the observed proportion is zero or one.

That improvement has limits. The interval is based on a normal approximation, not an exact binomial calculation. In particular, a nondegenerate interval at a boundary does not by itself imply correct finite-sample coverage. The formulas below make both points explicit.

The Arcsine Interval and Its Endpoints

Definition: For a binomial sample $X\sim\operatorname{Bin}(n,p)$, let $\hat p=X/n$. Define the arcsine transformation $g:[0,1]\to[0,\pi]$ and its inverse $h:[0,\pi]\to[0,1]$ by $$ g(p)=2\arcsin\sqrt p, \qquad h(t)=\sin^2(t/2). $$ For a positive critical value $z$, the clipped arcsine interval is $$ \left[ h\!\left(\max\{0,g(\hat p)-z/\sqrt n\}\right), h\!\left(\min\{\pi,g(\hat p)+z/\sqrt n\}\right) \right]. $$

For a two-sided nominal 95% interval, the usual choice is $z=z_{0.975}\approx1.96$, the 97.5th percentile of the standard normal distribution. Clipping keeps the transformed endpoints in $[0,\pi]$; applying $h$ then keeps the proportion endpoints in $[0,1]$. Since $h$ is increasing on $[0,\pi]$, the lower transformed endpoint remains the lower proportion endpoint.

For a fixed $p$ strictly between zero and one, the binomial central limit theorem and the delta method motivate this construction. The derivative is $g'(p)=1/\sqrt{p(1-p)}$, while $\operatorname{Var}(\hat p)=p(1-p)/n$. Thus the first-order variance of $g(\hat p)$ is $1/n$. This argument concerns fixed interior parameter values; it does not establish exact coverage, or automatically justify the same approximation when $p$ approaches a boundary as $n$ changes.

Worked Example: Zero Observed Successes and Clipping

Suppose $X=0$, $n=1$, and the chosen critical value is $z=4$. Then $\hat p=0$, $g(\hat p)=0$, and $z/\sqrt n=4$. The clipped transformed endpoints are $$ \max\{0,0-4\}=0, \qquad \min\{\pi,0+4\}=\pi. $$

Back-transforming gives $[h(0),h(\pi)]=[0,1]$. In particular, the upper endpoint is $1$. It would be incorrect to use $\sin^2(4/2)\approx0.8268$ here: that expression back-transforms the un-clipped upper limit $4$, which lies outside the transformation’s range $[0,\pi]$.

What Happens at the Two Boundaries?

When $X=0$, the observed transformed estimate is zero. The lower endpoint is therefore zero, and the upper endpoint is $$ h\!\left(\min\{\pi,z/\sqrt n\}\right) = \sin^2\!\left(\frac{\min\{\pi,z/\sqrt n\}}{2}\right). $$ The minimum is essential: the simpler expression $\sin^2(z/(2\sqrt n))$ is valid only when $z/\sqrt n\leq\pi$. When $X=n$, the interval is the reflection of the $X=0$ interval around $1/2$: its lower endpoint is $1-\sin^2(\min\{\pi,z/\sqrt n\}/2)$, and its upper endpoint is $1$.

For fixed $z$, once $n$ is large enough that $z/\sqrt n\leq\pi$, the endpoint at $X=0$ is approximately $z^2/(4n)$. Thus the interval’s upper endpoint is positive but shrinks at rate $1/n$. This describes the size of the interval at the observed boundary; it is not a statement that its coverage is nominal for nearby parameter values.

Result: For fixed $z>0$, at $X=0$ the upper endpoint $U_n$ satisfies $$ U_n=\sin^2\!\left(\frac{z}{2\sqrt n}\right) \quad\text{for all sufficiently large }n, \qquad nU_n\longrightarrow \frac{z^2}{4}. $$ At $X=n$, the lower endpoint $L_n$ satisfies $n(1-L_n)\longrightarrow z^2/4$.

To verify the first limit, fixed $z$ implies $z/\sqrt n\leq\pi$ for all sufficiently large $n$, so clipping no longer changes the endpoint formula. Using $\lim_{u\to0}\sin(u)/u=1$, $$ nU_n =n\sin^2\!\left(\frac{z}{2\sqrt n}\right) =\frac{z^2}{4} \left[ \frac{\sin(z/(2\sqrt n))}{z/(2\sqrt n)} \right]^2 \longrightarrow\frac{z^2}{4}. $$ Reflection about $1/2$ gives $1-L_n=U_n$ when $X=n$, proving the second limit. $\square$

Worked Example: A Boundary Endpoint for a 95% Interval

Let $n=20$, $X=0$, and use $z=1.96$. Here $$ \frac{z}{\sqrt n}=\frac{1.96}{\sqrt{20}}\approx0.4383<\pi, $$ so the clipping does not alter the upper transformed endpoint. The interval is $$ \left[0,\sin^2\!\left(\frac{1.96}{2\sqrt{20}}\right)\right] \approx[0,0.0473]. $$

The raw Wald interval would have estimated standard error $\sqrt{0(1-0)/20}=0$, hence would be $[0,0]$. The arcsine interval instead extends to about $0.0473$. This illustrates a practical boundary improvement, not a finite-sample 95% coverage guarantee.

Back-Transformation Can Make the Interval Asymmetric

A symmetric interval on the transformed scale need not be symmetric on the proportion scale. The inverse $h(t)=\sin^2(t/2)$ is nonlinear. Consequently, even if the two transformed endpoints are equally far from $g(\hat p)$, their distances from $\hat p$ after back-transformation can differ.

Worked Example: Back-Transforming an Interior Estimate

Take $n=10$, $X=5$, and $z=1.96$. Then $\hat p=0.5$, $g(\hat p)=\pi/2$, and $z/\sqrt n\approx0.6198$. The transformed limits are approximately $0.9510$ and $2.1906$, both within $[0,\pi]$. Therefore the proportion limits are $$ \sin^2(0.9510/2)\approx0.210, \qquad \sin^2(2.1906/2)\approx0.790. $$

The interval is approximately $[0.210,0.790]$, centered at $0.5$ in this particular example because of symmetry about $\pi/2$. Away from $\hat p=0.5$, the inverse transformation generally produces unequal distances from the estimate. The endpoints should be computed by transforming each limit, not by treating the transformed-scale margin as a margin on the original scale.

Exact Finite-Sample Coverage Is a Separate Calculation

For a proposed true value $p$, the interval covers $p$ precisely when its transformed-scale interval contains $g(p)$. Clipping does not change this condition: $g(p)$ already belongs to $[0,\pi]$, so clipping either end of the interval cannot remove a point that was inside the original transformed interval.

Result: Let $C(X)$ be the clipped arcsine interval, and let $z>0$. Its exact coverage probability under $X\sim\operatorname{Bin}(n,p)$ is $$ P_p\{p\in C(X)\} = \sum_{k=0}^n \mathbf{1}\!\left\{ \left|g(k/n)-g(p)\right|\leq z/\sqrt n \right\} \binom{n}{k}p^k(1-p)^{n-k}. $$

To prove the formula, fix a possible count $k$. Before clipping, the transformed interval is centered at $g(k/n)$ with half-width $z/\sqrt n$. Because $g(p)\in[0,\pi]$, it belongs to the clipped interval if and only if $$ g(k/n)-z/\sqrt n\leq g(p)\leq g(k/n)+z/\sqrt n, $$ which is equivalent to $\left|g(k/n)-g(p)\right|\leq z/\sqrt n$. The event of coverage is therefore the union of exactly those count outcomes $k$ satisfying this inequality. Summing their binomial probabilities gives the displayed expression. $\square$

Worked Example: Exact Coverage Near Zero

Let $n=10$, $p=0.15$, and $z=1.96$. The transformed true value and half-width are $$ g(0.15)=2\arcsin\sqrt{0.15}\approx0.7954, \qquad z/\sqrt{10}\approx0.6198. $$ Thus a count $k$ covers $p$ when $g(k/10)$ lies between approximately $0.1756$ and $1.4152$. The values for $k=1,2,3,4$ are approximately $0.6435,0.9273,1.1593,1.3694$, respectively, so they satisfy the condition. For $k=0$, $g(0)=0$ is too small; for $k\geq5$, $g(k/10)\geq g(0.5)=\pi/2$, which is too large. Therefore coverage occurs exactly when $1\leq X\leq4$.

Under $X\sim\operatorname{Bin}(10,0.15)$, the exact coverage is $$ \begin{aligned} P(1\leq X\leq4) &=\binom{10}{1}(0.15)(0.85)^9 +\binom{10}{2}(0.15)^2(0.85)^8\\ &\quad+\binom{10}{3}(0.15)^3(0.85)^7 +\binom{10}{4}(0.15)^4(0.85)^6\\ &\approx0.3474254+0.2758967+0.1298337+0.0400957\\ &\approx0.7933. \end{aligned} $$

The exact coverage in this example is about 79.3%, not 95%. A normal-based transformed interval may improve on a degenerate Wald interval at an observed boundary, but this calculation shows why that improvement must not be confused with exact finite-sample coverage.

Interpretation and a Common Pitfall

The arcsine construction is useful because it respects the parameter space after back-transformation and avoids collapsing to a single point at an observed zero or one. Its endpoint behavior is also interpretable: for fixed critical value, the boundary endpoint is of order $1/n$. But the binomial distribution is discrete, and the set of counts producing coverage changes with $p$, $n$, and $z$. There is no general reason for the resulting finite-sample coverage to equal the nominal level.

Wrong reasoning: “The variance is stabilized, so the interval has exact 95% coverage, including near zero.”
Correction: Variance stabilization motivates a normal approximation for transformed estimates under appropriate asymptotic conditions. Exact coverage is a binomial probability, as the count-by-count formula shows.
Wrong reasoning: “At $X=0$, the upper endpoint is always $\sin^2(z/(2\sqrt n))$.”
Correction: The endpoint is $\sin^2(\min\{\pi,z/\sqrt n\}/2)$; omitting clipping can give an incorrect result when $z/\sqrt n>\pi$.

Key takeaway: Near a probability boundary, calculate the clipped transformed limits first and then back-transform them. A positive boundary endpoint can be a useful improvement over a degenerate Wald interval, but only an explicit coverage calculation establishes finite-sample coverage.

Check Your Understanding

Work through these questions using the endpoint and coverage formulas.

For $X=0$, $n=4$, and $z=1.96$, write the exact upper endpoint, including clipping.
Why does the arcsine interval at $X=0$ have a positive upper endpoint when $z/\sqrt n\leq\pi$, while the raw Wald interval collapses?
For fixed $z$, what is the limiting value of $n$ times the upper endpoint when $X=0$?
Explain why back-transforming a symmetric transformed interval does not generally produce an interval symmetric about $\hat p$.
What must be summed to find the exact coverage probability of an arcsine interval under a binomial model?