Log-Log Models for Power-Law Relationships

Why Use a Log-Log Model?

Some relationships between positive measurements are curved when graphed on their original scales but become approximately linear after taking logarithms of both variables. A common example is a power-law relationship: the response changes in proportion to a power of the predictor. The log-log model lets us fit such a pattern with a straight-line regression and use its slope to estimate that power.

The transformation is useful only when it makes sense for the data and the scientific question. In particular, the predictor and response must both be positive to take their logarithms. The transformed scatterplot and residuals should also support a linear regression model; transforming both variables does not guarantee a good fit.

Definition: A power-law relationship has the form \(y=a x^b\), where \(x>0\), \(y>0\), \(a>0\), and \(b\) is a constant. Taking logarithms of both variables gives \(\log(y)=\log(a)+b\log(x)\), a straight-line relationship between the log-transformed response and predictor. The exponent \(b\) is the slope in that relationship.

From a Power Law to a Regression Line

For observed pairs \((x_i,y_i)\), define \(u_i=\ln(x_i)\) and \(v_i=\ln(y_i)\), where \(\ln\) denotes the natural logarithm. If the power law describes the underlying pattern, its transformed form is a line. To allow for observations that do not lie exactly on the pattern, write a regression model on the transformed scale:

v_i=\alpha+\beta u_i+\varepsilon_i

Here \(\alpha\) is the intercept, \(\beta\) is the slope, and \(\varepsilon_i\) is the error for observation \(i\). The corresponding power-law form is \(y_i=e^\alpha x_i^\beta e^{\varepsilon_i}\). Thus the fitted slope estimates the power exponent. For a descriptive fit, ordinary least squares (OLS) chooses the line that minimizes the sum of squared residuals in \(v=\ln(y)\), not in the original response \(y\).

The OLS line can be calculated when the transformed predictor values are not all identical. If inference about the line is intended, the usual linear-regression conditions apply on the log scale: the conditional mean of \(v\) is linear in \(u\), errors are independent with mean zero and constant variance, and normally distributed errors are additionally needed for exact small-sample \(t\)-based inference. The data collection must support the independence assumption. A transformed-scale residual plot can help assess linearity and unequal spread; the order of observations can matter when checking dependence. These conditions concern the transformed response, not the untransformed \(y\).

OLS formulas: With \(u_i=\ln(x_i)\) and \(v_i=\ln(y_i)\), the fitted slope and intercept are

\hat\beta_1= \frac{\sum_{i=1}^{n}(u_i-\bar u)(v_i-\bar v)} {\sum_{i=1}^{n}(u_i-\bar u)^2}, \qquad \hat\beta_0=\bar v-\hat\beta_1\bar u.

The fitted transformed line is \(\hat v=\hat\beta_0+\hat\beta_1u\). In power-law form, its fitted curve is \(\hat y=e^{\hat\beta_0}x^{\hat\beta_1}\).

Why the Slope Is the Power Exponent

The slope-exponent connection follows directly from the logarithm rule for powers. Starting with an exact power law \(y=a x^b\), take the natural logarithm of both sides:

\ln(y)=\ln(a x^b)=\ln(a)+\ln(x^b) =\ln(a)+b\ln(x).

This has the form of a straight line with log-response \(\ln(y)\) as the vertical variable and log-predictor \(\ln(x)\) as the horizontal variable. Its intercept is \(\ln(a)\), and its slope is \(b\). In a regression with variation around the power-law pattern, OLS estimates that slope with \(\hat\beta_1\). This algebra justifies calling the fitted slope the estimated power exponent; it does not establish that a power law is appropriate for any particular data set.

Worked Example: A Power Law That Fits Exactly

Suppose hypothetical measurements are \((x,y)=(1,4),(2,8),(4,16),(8,32)\). Fit a log-log line and identify its exponent.

Take natural logarithms. The transformed predictor values are \(0,\ln(2),2\ln(2),3\ln(2)\). Each response is four times its predictor, so \(v_i=\ln(y_i)=\ln(4)+u_i\). The transformed points lie exactly on a line with slope 1 and intercept \(\ln(4)\).

The OLS formula confirms the slope. The deviations of \(u_i\) from their mean are \(-1.5\ln(2),-0.5\ln(2),0.5\ln(2),1.5\ln(2)\). Since \(v_i=u_i+\ln(4)\), the deviations of \(v_i\) from their mean are the same. Therefore,

\hat\beta_1= \frac{5[\ln(2)]^2}{5[\ln(2)]^2}=1, \qquad \hat\beta_0=\ln(4).

The fitted power law is \(\hat y=4x^1\), or \(\hat y=4x\). The estimated exponent is 1. Because every transformed point lies exactly on the line, all four transformed residuals are zero. This example illustrates the algebra, rather than a typical noisy data set.

Fitting a Power Law to Noisy Data

When observations vary around the pattern, first transform each positive \(x_i\) and \(y_i\), then fit OLS to \((u_i,v_i)\). Calculate both sums in the slope formula using the transformed values. They must be the actual centered-product sum and centered-square sum; factors introduced by the log transformation cannot be dropped from one sum independently.

Worked Example: Estimating an Exponent From Noisy Measurements

Consider hypothetical positive observations \((x,y)=(1,10),(2,15),(4,23),(8,35)\). Fit a power-law model using natural logarithms.

The transformed predictor values are \(0,\ln(2),2\ln(2),3\ln(2)\), with mean \(\bar u=1.5\ln(2)\). The transformed response values are \(\ln(10),\ln(15),\ln(23),\ln(35)\), with mean \(\bar v\approx2.92537\). The centered predictor values are \(-1.5\ln(2),-0.5\ln(2),0.5\ln(2),1.5\ln(2)\).

Using those centered values, the numerator and denominator are:

\begin{aligned} \sum (u_i-\bar u)(v_i-\bar v) &=\ln(2)\left[-1.5\ln(10)-0.5\ln(15)+0.5\ln(23)+1.5\ln(35)\right]\\ &\approx 1.45066,\\ \sum (u_i-\bar u)^2 &=5[\ln(2)]^2\\ &\approx 2.40227. \end{aligned}

Thus the fitted slope is \(\hat\beta_1\approx1.45066/2.40227\approx0.6039\). The intercept is \(\hat\beta_0=\bar v-\hat\beta_1\bar u\approx2.2975\). The fitted line on the transformed scale is \(\widehat{\ln(y)}=2.2975+0.6039\ln(x)\), corresponding to the fitted power-law curve \(\hat y=e^{2.2975}x^{0.6039}\), or approximately \(\hat y=9.949x^{0.6039}\). The estimated exponent is about 0.604. The arithmetic gives a fitted line, but whether it is an adequate model still needs to be assessed using the transformed-scale pattern and residuals.

Checking the Fit and Understanding the Log Base

A straight-line fit in transformed coordinates should be judged on that scale. Plot \(v_i=\ln(y_i)\) against \(u_i=\ln(x_i)\), and examine whether a straight line describes the pattern. Check the residuals \(v_i-\hat v_i\) for structure such as curvature or changing spread. A strong curved pattern means the fitted line may not capture the relationship; a fan-shaped residual pattern signals that the constant-variance condition may be questionable. Also consider whether the observations can reasonably be treated as independent.

Natural logarithms are convenient, but the log base does not change the fitted slope when the same base is used for both variables. For example, \(\log_{10}(x)=\ln(x)/\ln(10)\) and \(\log_{10}(y)=\ln(y)/\ln(10)\). Both transformed coordinates are scaled by the same positive constant, so their centered cross-product and centered squared-predictor sum are each scaled by \(1/[\ln(10)]^2\). That common factor cancels in the slope ratio. The intercept’s numerical value depends on the chosen log base, but the slope does not.

Worked Example: Changing the Predictor's Units

Suppose the fitted model in the preceding example uses \(x\) measured in one unit and has slope \(0.6039\). Now express the same predictor values in units 100 times smaller, so that \(x'=100x\). Does the estimated exponent change?

On the log scale, \(u'_i=\ln(x'_i)=\ln(x_i)+\ln(100)=u_i+\ln(100)\). Adding the same constant to every predictor value shifts their mean by that constant but leaves every centered value unchanged: \(u'_i-\bar u'=u_i-\bar u\). Therefore the centered cross-product and the centered predictor sum of squares stay the same, so their ratio—and thus the fitted slope—remains \(0.6039\).

The intercept changes because the predictor's numerical scale changed. Specifically, \(\hat\beta'_0=\hat\beta_0-\hat\beta_1\ln(100)\), approximately \(2.2975-(0.6039)(4.6052)\approx-0.4836\). Changing units therefore changes the intercept's numerical value, not the estimated power exponent.

Common Mistakes and Why the Model Matters

A log-log model is useful because it converts a curved power-law pattern into a linear fitting problem while retaining a direct connection between the fitted slope and the exponent. It is not a license to apply OLS without checking the transformed data, nor does a fitted power law by itself establish a causal relationship.

Using only one logarithm. Logging only the response or only the predictor creates a different model. A power law becomes a straight line when both variables are logged.
Using original-scale values in the slope formula. OLS here is fitted to \(u_i=\ln(x_i)\) and \(v_i=\ln(y_i)\). Substituting \(x_i\) and \(y_i\) directly estimates a different line.
Dropping transformation factors from the sums. The numerator and denominator must be calculated from the same centered log values. A ratio may accidentally look plausible even when its component sums are incorrect.
Assuming the transformation guarantees a good model. Check the log-log scatterplot and residuals. Curvature or unequal spread on that scale still raises concerns about the linear regression model.
Treating the intercept as the exponent. In the log-log line, the slope estimates the exponent; the intercept is the log of the multiplicative constant under the stated natural-log convention.

Key takeaway: For positive predictor and response values, define \(u=\ln(x)\) and \(v=\ln(y)\), then fit the OLS line \(v=\hat\beta_0+\hat\beta_1u\). The fitted slope \(\hat\beta_1\) estimates the power-law exponent, provided the transformed-scale linear-regression model is appropriate.

Check Your Understanding

Answer each question using the definitions and procedures in this tutorial.

Starting from \(y=a x^b\), derive the equation relating \(\ln(y)\) to \(\ln(x)\). Which part of that line is the power exponent?
For a log-log regression, which transformed values are used in the OLS slope formula?
In the fitted model \(\widehat{\ln(y)}=1.2+0.7\ln(x)\), what is the estimated power exponent?
Explain why multiplying every predictor value by a fixed positive constant does not change the fitted slope.
Name two features of a log-scale residual plot that would make you question whether the linear-regression model is suitable.