Why Observed Price Differences Can Mislead
A price-demand model fitted to quote or renewal records describes patterns in the prices and outcomes that were observed. Those patterns need not equal the demand response that would occur if the same customers were offered different prices. Insurers may assign different prices to customers with different expected costs, prior demand, or other characteristics that also affect purchase. Then price and demand are associated partly because of those characteristics, not just because price changed.
The central task is to define the effect of interest and identify which pre-price variables create this confounding. Earlier material on elasticity distinguishes a demand response from its units and scale. Here the question is whether the price variation in observational data supports a causal interpretation at all.
For example, suppose a hypothetical insurer charges higher premiums to customers with higher expected claim costs. If those customers also have a greater need for coverage, their demand may differ even at the same premium. A raw comparison of purchase rates across premium levels mixes this difference in customer composition with the effect of price. The direction of the resulting bias is not automatic: it depends on how the confounder relates to both price and demand.
A Causal Diagram and the Adjustment Target
A causal diagram represents assumed causal relationships with arrows. Let \(P\) denote quoted price, \(Y\) denote demand (such as purchase), and \(R\) denote a risk or customer characteristic known before the price is set. If \(R\) influences both the price and demand, the diagram contains \(R \rightarrow P\) and \(R \rightarrow Y\), as well as the price effect \(P \rightarrow Y\). The open path \(P \leftarrow R \rightarrow Y\) creates a noncausal association between price and demand.
A set of variables blocks this backdoor path if conditioning on them removes the association transmitted along paths that enter price through an arrow pointing into price. For this simple diagram, adjusting for \(R\) blocks the path. In practice, several pre-price variables may be common causes: for example, risk tier, prior coverage, or distribution channel, if each genuinely affects both price assignment and demand. Their role must be justified by the pricing and demand process, not selected solely because they improve model fit.
These assumptions are substantive. A diagram makes the assumed relationships explicit, but it cannot prove that all important common causes were measured. If customers with different unobserved purchase propensity receive different prices, adjusting for recorded variables may leave residual confounding. Likewise, a fitted model cannot reliably compare prices in a risk group that was never offered one of them.
For a discrete price \(p\), let \(Y(p)\) be the outcome that would be observed under that price, and let \(C\) be a sufficient set of pre-price adjustment variables. Under exchangeability, positivity, and consistency, the mean potential outcome can be expressed as:
For continuous covariates, the sum is replaced by an integral. This formula standardizes the conditional demand at price \(p\) to one common covariate distribution. It prevents the price-specific averages from being compared using different customer mixes. The target distribution \(P(C=c)\) must be stated: it might be the portfolio being priced, or another defined population with adequate price overlap.
Worked Example: Composition Bias in Purchase Rates
Worked Example: Composition Bias in Purchase Rates
A hypothetical insurer observes two quoted premiums, $100 and $120, and classifies customers as lower-risk or higher-risk before pricing. The observed purchase rates are:
| Risk group | Purchase rate at $100 | Purchase rate at $120 | Share at $100 | Share at $120 |
|---|---|---|---|---|
| Lower-risk | 0.80 | 0.70 | 0.80 | 0.20 |
| Higher-risk | 0.50 | 0.40 | 0.20 | 0.80 |
Calculate the unadjusted difference in purchase rates, then standardize both prices to a target population that is 50% lower-risk and 50% higher-risk. Compare the implied change per dollar.
Unadjusted rate at $100. Weight each group’s rate by its observed share at that price:
Unadjusted rate at $120. The group shares change, so use the $120 shares:
The unadjusted difference is \(0.46-0.74=-0.28\) over a $20 premium increase. The implied unadjusted rate change is \(-0.28/20=-0.014\) per dollar.
Standardized rate at $100. Apply the same 50/50 target weights:
Standardized rate at $120:
The standardized difference is \(0.55-0.65=-0.10\), so the standardized change per dollar is \(-0.10/20=-0.005\). Within each risk group, purchase falls by 0.10 when price rises by $20, which also gives \(-0.005\) per dollar. The large unadjusted decline includes the shift toward higher-risk customers at the higher price.
Check. Both standardized rates use the same group weights, and their difference equals the average of the two within-group differences under equal weights: \(0.50(-0.10)+0.50(-0.10)=-0.10\). The common mistake is to interpret the raw \(-0.014\) as the price response while ignoring the change in composition.
Why Omitted Confounders Bias a Regression Slope
The same issue appears in regression. Suppose the conditional mean of demand is linear in price \(P\) and a confounder \(R\):
where the conditional error has mean zero given \(P\) and \(R\). If \(R\) is omitted and \(Y\) is regressed on \(P\) with an intercept, the population slope is:
This is not a general guarantee that a linear model is correct. It is the omitted-variable identity under the stated linear conditional-mean model and error assumptions. Its bias term combines the effect of \(R\) on demand with the degree to which \(R\) varies alongside price. If either \(\beta_R=0\) or \(\operatorname{Cov}(P,R)=0\), omitting \(R\) does not create this particular slope bias. Otherwise, the bias can increase, reduce, or reverse the apparent price response.
The sign is contextual. If higher-risk customers receive higher prices and also have greater underlying demand, then both \(\operatorname{Cov}(P,R)\) and \(\beta_R\) are positive. The omitted-variable term is positive, making a negative price slope look less negative than its conditional value. If the confounder affects demand in the opposite direction, the term may have the opposite sign.
Worked Example: Quantifying Omitted-Variable Bias
Worked Example: Quantifying Omitted-Variable Bias
In a hypothetical local demand analysis, \(P\) is premium measured in hundreds of dollars, \(R\) is a standardized pre-price customer score, and \(Y\) is purchase probability. Assume the conditional mean is \(Y=\alpha-0.08P+0.12R+\varepsilon\), with \(\operatorname{Cov}(P,\varepsilon)=0\). The observed data have \(\operatorname{Cov}(P,R)/\operatorname{Var}(P)=0.50\). At the evaluation point, \(P=10\) and fitted purchase probability is 0.60. Compare the omitted-variable and adjusted price slopes and their point elasticities.
Unadjusted slope. Apply the omitted-variable identity:
Adjusted slope. The model including \(R\) has conditional price coefficient \(\beta_P=-0.08\), per $100 of premium. The omitted confounder has moved the raw slope upward by \(0.06\) probability units per $100.
For a differentiable demand function \(q(P)\), point price elasticity is \(\epsilon=(dq/dP)(P/q)\). Since \(P\) here is in $100 units, use \(P=10\) in both calculations. The adjusted elasticity is:
The unadjusted elasticity is:
Rounded to two decimals, adjustment changes the estimated point elasticity from \(-0.33\) to \(-1.33\). These are local elasticities at the stated evaluation point, not percentage changes over an arbitrary large price interval. The key trap is to insert $1,000 as \(P\) while retaining a coefficient measured per $100; that would mismatch the units by a factor of 100.
Choose Adjustment Variables for the Right Effect
Adjustment is not the same as including every available variable. The correct set depends on the causal structure and on whether the target is a total or direct price effect. A total effect includes all causal pathways from price to demand. For instance, price may affect whether a customer seeks additional information, which then affects purchase. If that information-seeking variable occurs after the price is shown, it is a mediator. Controlling for it blocks part of the total price effect.
A collider is another reason not to adjust indiscriminately. In a structure \(P \rightarrow A \leftarrow U \rightarrow Y\), \(A\) is caused by both price and an unobserved factor \(U\) that also affects demand. Conditioning on \(A\) can create an association between \(P\) and \(U\), opening a noncausal path. A variable’s statistical association with price or demand alone does not establish that adjustment is appropriate.
Worked Example: Separating Confounders From a Mediator
A hypothetical insurer’s causal assumptions are summarized as follows: pre-price risk \(R\) and channel \(C\) both affect quoted premium \(P\) and purchase outcome \(Y\); premium also affects post-quote information seeking \(M\), which affects purchase. Assume a local linear structural model, with \(P\) measured in hundreds of dollars:
Assume no other confounding, adequate price overlap within \(R,C\), and that the equations describe the stated local range. Determine which variables should be adjusted for a total price effect, and compare that effect with the direct effect.
Identify the common causes. Both \(R\) and \(C\) precede pricing and affect both \(P\) and \(Y\). They are confounders in the stated diagram, so adjust for them to block the paths \(P \leftarrow R \rightarrow Y\) and \(P \leftarrow C \rightarrow Y\).
Calculate the total effect. A one-unit increase in \(P\), meaning a $100 increase, changes \(Y\) directly by \(-0.04\). It also changes \(M\) by \(-0.20\), which changes \(Y\) by \(0.10(-0.20)=-0.02\). Thus the total change is:
Compare with mediator adjustment. If \(M\) is held fixed, the coefficient on \(P\) is \(-0.04\), the direct effect in this model. Adjusting for \(M\) would therefore omit the mediated component \(-0.02\) and would not answer the total-effect question. For the total effect, adjust for \(R\) and \(C\), not \(M\). The result is \(-0.06\) probability units per $100, conditional on the stated model and assumptions.
Check. The total effect equals the direct effect plus the mediated effect: \(-0.04-0.02=-0.06\). The common error is to include every available predictor and then label the price coefficient a total effect, even when a post-price mediator has been controlled.
Interpretation and Common Exam Traps
An adjusted estimate is not automatically causal. It is a causal estimate only if the adjustment set and assumptions are credible for the specified population and price contrast. Before interpreting a price coefficient, state the outcome, price scale, target population, and whether the coefficient represents a conditional response or a standardized marginal response. In nonlinear models, such as logistic demand models, a conditional model coefficient and a population-standardized effect generally are not numerically identical; compute predicted outcomes under each price and average them over the target covariate distribution when a marginal effect is wanted.
- Ignoring different customer mixes: Raw rates at two prices may weight risk groups differently. Standardize to common weights or use an appropriate adjusted model.
- Choosing variables by prediction alone: A strong predictor is not necessarily a confounder. Use timing and causal relationships to justify adjustment.
- Adjusting for a post-price variable: This can remove part of the total effect or introduce bias, depending on its causal role.
- Assuming overlap: If a covariate group receives only one price, the data do not directly support a within-group comparison across prices.
- Overstating identification: Adjustment addresses measured confounding under assumptions; it does not demonstrate that unmeasured confounding is absent.
- Mixing units: A slope per dollar, per $100, or per percentage price change leads to different numerical calculations. Match the slope’s unit to the price used in an elasticity.
Check Your Understanding
For each problem, identify the relevant adjustment logic and check the final result.
- A hypothetical book of policies has purchase rates at premiums $90 and $110 as follows: lower-risk rates are 0.90 and 0.80; higher-risk rates are 0.60 and 0.50. Standardize both prices to a target group that is 60% lower-risk and 40% higher-risk. What are the standardized rates and the change per dollar? (Answer: 0.78 at $90, 0.68 at $110, and \(-0.005\) per dollar.)
- In a linear demand model, the price coefficient is \(-0.06\), the confounder coefficient is \(0.15\), and \(\operatorname{Cov}(P,R)/\operatorname{Var}(P)=0.20\). What slope results if the confounder is omitted? (Answer: \(-0.06+0.15(0.20)=-0.03\).)
- A risk group has no customers quoted the lower of two prices, although it has many customers quoted the higher price. Can ordinary covariate adjustment identify that group’s lower-price demand from these data? State the relevant condition. (Answer: No; positivity fails for that group and price contrast.)
- A pre-price score affects both assigned premium and purchase. A post-price service inquiry is affected by premium and affects purchase. For a total price effect, which variable is the confounder and should the inquiry automatically be adjusted for? (Answer: Adjust for the pre-price score; do not automatically adjust for the post-price inquiry because it is a mediator.)
- An adjusted linear demand slope is \(-0.04\) per $100, and fitted purchase probability is 0.50 at a premium of $800. Calculate the point elasticity, using the price in units matching the slope. (Answer: \((-0.04)(8/0.50)=-0.64\).)