Deriving Variance-Stabilizing Transformations

Why Derive a Transformation?

An estimator can be more variable at some parameter values than at others. A transformation of the parameter scale can sometimes make its first-order asymptotic variance constant across the parameter space. The central calculation is short, but its interpretation matters: it is a local, large-sample statement, and the transformation is not uniquely determined.

Suppose \(\hat{\theta}\) estimates a scalar parameter \(\theta\), and for each fixed \(\theta\) its asymptotic distribution is

\sqrt{n}(\hat{\theta}-\theta)\xrightarrow{d}N(0,V(\theta)),

where \(V(\theta)>0\). The quantity \(V(\theta)/n\) is the variance in the corresponding first-order normal approximation to \(\hat{\theta}\). We seek a differentiable function \(g\) for which the transformed estimator \(g(\hat{\theta})\), targeting \(g(\theta)\), has a limiting variance that does not depend on \(\theta\).

Definition and Differential Equation

Definition: Let \(\theta\) range over an open interval \(I\), and suppose \(\sqrt{n}(\hat{\theta}-\theta)\xrightarrow{d}N(0,V(\theta))\), with \(0<V(\theta)<\infty\). A differentiable function \(g\) is a first-order variance-stabilizing transformation on \(I\) if the delta method gives \(g(\hat{\theta})\) an asymptotic variance \(K/n\), where \(K\) is a positive constant independent of \(\theta\).

The scalar delta method says that if \(g\) is differentiable at \(\theta\), then

\sqrt{n}\{g(\hat{\theta})-g(\theta)\} \xrightarrow{d} N\bigl(0,[g'(\theta)]^2V(\theta)\bigr).

Thus stabilization requires \([g'(\theta)]^2V(\theta)=K\) throughout \(I\). For a nonconstant, invertible transformation, take \(K>0\). Solving for the derivative yields the defining differential equation.

Key equation: If \(V(\theta)>0\), a first-order variance-stabilizing transformation satisfies \(g'(\theta)=c/\sqrt{V(\theta)}\), where \(c\ne 0\) is a constant. Equivalently, its solutions are \(g(\theta)=b+c\int_{\theta_0}^{\theta}du/\sqrt{V(u)}\), on any interval where the integral is defined. Here \(b\) is an arbitrary location constant and \(c\) is an arbitrary nonzero scale constant.

The square root is essential: the variance involves the square of the derivative, so solving for the derivative takes a square root. The sign of \(c\) chooses the orientation of the scale: positive \(c\) gives an increasing transformation, while negative \(c\) gives a decreasing one.

Derivation and Characterization

The next result states both why the equation works and why, under the stated conditions, it describes every such transformation. Assume \(V\) is continuous and strictly positive on the open interval \(I\). These conditions ensure that \(1/\sqrt{V}\) is continuous and that its integral defines a differentiable function there. Also assume the displayed asymptotic normal limit holds for each fixed \(\theta\in I\).

Theorem: Let \(g:I\to\mathbb{R}\) be continuously differentiable. Its delta-method asymptotic variance is a positive constant \(K\) at every \(\theta\in I\) if and only if, for some \(b\in\mathbb{R}\), \(c\ne0\), and fixed \(\theta_0\in I\), \(g(\theta)=b+c\int_{\theta_0}^{\theta}du/\sqrt{V(u)}\). The constant variance is \(c^2/n\).

Proof. By the delta method, the variance factor in the limiting distribution of \(\sqrt{n}\{g(\hat{\theta})-g(\theta)\}\) is \([g'(\theta)]^2V(\theta)\). If this is the positive constant \(K\), then

[g'(\theta)]^2V(\theta)=K \quad\Longrightarrow\quad |g'(\theta)|=\frac{\sqrt{K}}{\sqrt{V(\theta)}}.

Because \(g'\) is continuous and never zero, its sign cannot change on the connected interval \(I\). Otherwise it would equal zero somewhere by the intermediate value theorem. Therefore \(g'(\theta)=c/\sqrt{V(\theta)}\) for one fixed \(c\), with \(c=\sqrt{K}\) or \(c=-\sqrt{K}\). Integrating from \(\theta_0\) to \(\theta\) gives

g(\theta)-g(\theta_0) =c\int_{\theta_0}^{\theta}\frac{du}{\sqrt{V(u)}}.

Setting \(b=g(\theta_0)\) gives the stated form. Conversely, differentiating that form gives \(g'(\theta)=c/\sqrt{V(\theta)}\), and hence \([g'(\theta)]^2V(\theta)=c^2\) at every \(\theta\). The delta method then gives constant asymptotic variance \(c^2/n\). \(\square\)

The theorem also explains the freedom in choosing a solution. Adding \(b\) shifts every transformed value but does not change a derivative. Multiplying by \(c\) changes the variance by the constant factor \(c^2\), but does not make it parameter-dependent. Choosing a different reference point \(\theta_0\) only changes the additive constant, since the difference between two such integrals is constant in \(\theta\).

Worked Applications

Worked Example: A Variance Function on the Real Line

Suppose an estimator has the asymptotic distribution \(\sqrt{n}(\hat{\theta}-\theta)\xrightarrow{d}N(0,1+\theta^2)\), for \(\theta\in\mathbb{R}\). Find a stabilizing transformation and verify its variance factor.

Here \(V(\theta)=1+\theta^2\), which is positive and continuous on the whole real line. Taking \(c=1\) and \(\theta_0=0\), integrate:

g(\theta)=\int_0^\theta\frac{du}{\sqrt{1+u^2}} =\operatorname{asinh}(\theta).

Differentiation verifies the result: \(g'(\theta)=1/\sqrt{1+\theta^2}\). Therefore

[g'(\theta)]^2V(\theta) =\frac{1}{1+\theta^2}(1+\theta^2)=1.

The transformed estimator has first-order asymptotic variance \(1/n\). For example, at \(\theta=1\), the original variance factor is \(2\), while the squared derivative is \(1/2\); their product is \(1\). Every transformation \(b+c\operatorname{asinh}(\theta)\), with \(c\ne0\), also stabilizes the variance, at \(c^2/n\).

Worked Example: A Variance Function on a Restricted Interval

Suppose \(\theta\in(0,\infty)\) and the asymptotic variance function is \(V(\theta)=(1+\theta)^4\). Find a stabilizing transformation on this parameter interval.

Since \(\sqrt{V(\theta)}=(1+\theta)^2\), use \(\theta_0=1\) and integrate:

g(\theta) =\int_1^\theta\frac{du}{(1+u)^2} =\left[-\frac{1}{1+u}\right]_1^\theta =\frac{1}{2}-\frac{1}{1+\theta}.

Its derivative is \(g'(\theta)=1/(1+\theta)^2\), so

[g'(\theta)]^2V(\theta) =\frac{1}{(1+\theta)^4}(1+\theta)^4=1.

At \(\theta=1\), for instance, \(V(1)=16\) and \(g'(1)=1/4\), giving \(g'(1)^2V(1)=(1/16)16=1\). The transformation is strictly increasing, though its range is bounded: as \(\theta\) runs from \(0\) toward infinity, \(g(\theta)\) runs from \(-1/2\) toward \(1/2\). A bounded range does not prevent first-order stabilization.

Worked Example: Choosing a Convenient Scale

Suppose \(V(\theta)=e^{-2\theta}\) on \(\mathbb{R}\). A convenient primitive is \(g(\theta)=e^\theta\). Check whether it stabilizes the variance, and determine the effect of rescaling it to \(3e^\theta+5\).

For \(g(\theta)=e^\theta\), the derivative is \(e^\theta\), and \(\sqrt{V(\theta)}=e^{-\theta}\). Thus \(g'(\theta)=1/\sqrt{V(\theta)}\), and

[g'(\theta)]^2V(\theta) =e^{2\theta}e^{-2\theta}=1.

For \(\tilde g(\theta)=3e^\theta+5\), the derivative is \(3e^\theta\), so \(\tilde g'(\theta)^2V(\theta)=9\). Its asymptotic variance is therefore \(9/n\), still constant in \(\theta\). The added \(5\) has no effect; multiplying by \(3\) multiplies the variance by \(3^2\). At \(\theta=0\), the original variance factor is \(1\), and the rescaled transformation has squared derivative \(9\), confirming the factor \(9\).

Scope and Common Pitfalls

This construction depends on the variance function being finite and strictly positive on the interval of interest. If \(V\) vanishes or is not defined at a point, the differential equation may fail there, and the integral may not extend smoothly across it. In that case, the calculation may still work on separate intervals where \(V>0\), but a single differentiable transformation across the problematic point is not guaranteed.

A second limitation is the meaning of “variance” in the result. Convergence in distribution alone does not guarantee convergence of the estimators’ actual finite-sample variances. The delta-method conclusion is about the variance in the limiting normal distribution, or equivalently the first-order normal approximation. Additional moment conditions are needed to infer convergence of actual variances.

Using \(1/V(\theta)\) instead of \(1/\sqrt{V(\theta)}\). The variance after transformation is \([g'(\theta)]^2V(\theta)\), so the derivative must be proportional to the reciprocal square root.
Forgetting the arbitrary constants. Integration gives a family of solutions, not one uniquely determined function. Location and nonzero scale do not affect whether the variance is constant.
Claiming exact finite-sample stabilization. The delta method establishes first-order asymptotic stabilization, not generally a constant variance for every sample size.
Ignoring the parameter interval. The integral and the transformation must be considered where \(V\) is positive and the integral is defined; a formula valid on one interval need not extend through a boundary or singularity.

Key takeaway: If the estimator’s asymptotic variance is \(V(\theta)/n\), the delta method turns the stabilization problem into \([g'(\theta)]^2V(\theta)=K\). Integrating \(1/\sqrt{V(\theta)}\) gives all first-order stabilizing transformations, up to an arbitrary location and nonzero scale.

Check Your Understanding

Use the differential equation and its assumptions to answer these questions.

If \(V(\theta)=9\) throughout an interval, what form do the stabilizing transformations take, and what is their asymptotic variance?
Why does the differential equation involve the reciprocal square root of \(V(\theta)\), rather than its reciprocal?
If \(g\) stabilizes the variance with constant factor \(K\), what variance factor results from \(2g(\theta)-7\)?
What condition on \(V\) lets the integral formula define a differentiable transformation throughout an interval?
Does a constant delta-method variance imply that the transformed estimator has exactly the same finite-sample variance for every \(\theta\)? Explain.