Why the Observed Demand Sample May Mislead
A demand model can be correctly specified for the records in front of the analyst and still answer the wrong question. Quote records may include only customers who were approached, eligible, or willing to provide information. Renewal records may omit customers whose policies ended before the renewal offer or whose offers were not recorded. If the probability of appearing in the data is related to price or to customers’ potential demand, the observed sample can give a distorted price-response estimate.
This is sample selection: the outcome is observed only for a nonrandom subset of the population relevant to the analysis, or inclusion in that subset changes the relationship being estimated. The target might be demand among all eligible prospects, among customers actually quoted, or among policies reaching renewal. These are different populations and potentially different estimands. State the target before choosing a correction.
Selection is not automatically bias. If inclusion is independent of demand and price, or if it depends only on observed characteristics that are properly accounted for, the selected records may still identify the target relationship. The key question is how selection relates to the variables and population in the estimand.
Selection Is Not the Same as Confounding
Confounding concerns how price is assigned. For example, an insurer may offer higher prices to customers expected to have higher demand. Unless the analysis adjusts for the relevant factors or uses a credible source of price variation, a price-demand association may reflect those factors rather than a causal price response. The earlier material on confounding and instrumental variables addresses this problem.
Selection concerns who is observed. Even if price is assigned independently of customers’ potential demand in the target population, restricting analysis to records with a quote can create an association between price and demand if quote inclusion depends on both. Conversely, a complete sample can still have confounding if prices were assigned using demand-related information. Both problems can occur together; correcting one does not in general correct the other.
A useful diagnostic is to draw the data-generating sequence: who is eligible, who receives a quote, which price they receive, and for whom the outcome is recorded. Ask whether sample inclusion is caused by, or associated with, price and potential demand. Then compare the observed sample with the population to which the result is intended to apply.
Worked Example: Quote Selection Reverses the Price Pattern
A hypothetical insurer studies conversion among 2,000 eligible prospects, half in each of two customer types: higher-intent and lower-intent. Within each type, prices are assigned at random. At the lower price, conversion probabilities are 0.80 for higher-intent prospects and 0.20 for lower-intent prospects. At the higher price, the corresponding probabilities are 0.60 and 0.10. However, quote inclusion depends on both price and type: at the lower price, 20% of higher-intent and 80% of lower-intent prospects receive a recorded quote; at the higher price, the respective quote rates are 90% and 10%. Compare the target conversion rates with those in the quote sample. Treat conversion as Bernoulli with the stated probabilities; no interest discounting or mortality assumption is relevant.
Target population at the lower price. The two types are equally common, so the target conversion rate is \(0.50(0.80)+0.50(0.20)=0.50\).
Target population at the higher price. The rate is \(0.50(0.60)+0.50(0.10)=0.35\). Thus, in the eligible population the higher price is associated with a 0.15 lower conversion rate.
Quote sample at the lower price. Out of 1,000 prospects, 500 are higher-intent and 500 lower-intent. Expected quoted counts are \(500(0.20)=100\) and \(500(0.80)=400\). Expected conversions among them are \(100(0.80)=80\) and \(400(0.20)=80\). The selected-sample conversion rate is \((80+80)/(100+400)=0.32\).
Quote sample at the higher price. Expected quoted counts are \(500(0.90)=450\) higher-intent and \(500(0.10)=50\) lower-intent prospects. Expected conversions are \(450(0.60)=270\) and \(50(0.10)=5\). The selected-sample conversion rate is \((270+5)/(450+50)=0.55\).
The quote sample therefore shows conversion rising from 0.32 to 0.55 as price rises, despite a decline from 0.50 to 0.35 in the eligible population. The changing mix of types in the selected records overwhelms the within-type price pattern. The common mistake is to treat the quoted customers at each price as comparable simply because price was randomized within type. Quote selection still changes their composition. The calculation uses expected counts and rates; it is not a claim that a finite sample must equal those expectations.
What Selection Corrections Require
A correction is only as credible as the information and assumptions behind it. If selection is explainable by observed characteristics, inverse-probability weighting can reweight the selected records toward the target population. If selection depends on unobserved factors, a model may attempt to account for that dependence, but generally needs strong assumptions or design information. Neither approach makes an unidentified target identifiable by itself.
The reasoning is direct. A group that has inclusion probability \(\pi\) contributes only a fraction \(\pi\) of its members to the selected data on average. Multiplying each included member’s contribution by \(1/\pi\) offsets that underrepresentation. In expectation, for an outcome \(Y\), the weighted contribution conditional on \(P,X\) is
where the last equality requires \(Y\) to be independent of selection conditional on \(P,X\). If unobserved demand propensity still predicts inclusion after conditioning on those variables, that equality need not hold.
Positivity is also essential: every relevant target group must have a nonzero chance of being observed at the price levels being compared. If an eligibility rule excludes a group entirely, its demand cannot be learned from these records by assigning larger weights. Extremely small inclusion probabilities are technically positive but can produce highly variable weights and unstable estimates.
Define the population, outcome, and price comparison: for example, conversion among all eligible prospects under each price policy.
Use recorded design information or estimate \(P(S=1\mid P,X)\) from data that include selected and nonselected units.
Assess whether observed variables suffice to explain selection, whether inclusion probabilities are positive, and whether price assignment is separately addressed.
Apply inverse-inclusion weights, then check whether a few large weights dominate and whether weighted covariate distributions align with the target population.
Worked Example: Correcting Quote-Sample Rates by Weighting
In a hypothetical prospect population, 600 customers are type A and 400 are type B. At a lower price, the quote probabilities are 0.50 for type A and 0.25 for type B; conversion probabilities are 0.60 and 0.20. At a higher price, quote probabilities are 0.25 and 0.50; conversion probabilities are 0.40 and 0.10. Assume the stated conversion probabilities apply to the target population within each type, and quote inclusion is independent of conversion conditional on price and type. Outcomes are Bernoulli; no interest or mortality assumption is relevant. Calculate raw and weighted conversion rates at each price.
Lower price, raw quote sample. Expected quotes are \(600(0.50)=300\) type A and \(400(0.25)=100\) type B. Expected conversions are \(300(0.60)=180\) and \(100(0.20)=20\). The raw rate is \(200/400=0.50\).
The inclusion weights are \(1/0.50=2\) for type A and \(1/0.25=4\) for type B. The weighted converted count is \(180(2)+20(4)=360+80=440\). The weighted total is \(300(2)+100(4)=600+400=1{,}000\). The weighted rate is \(440/1{,}000=0.44\), matching the target calculation \(0.60(0.60)+0.40(0.20)=0.44\).
Higher price, raw quote sample. Expected quotes are \(600(0.25)=150\) type A and \(400(0.50)=200\) type B. Expected conversions are \(150(0.40)=60\) and \(200(0.10)=20\). The raw rate is \(80/350=0.2286\), to four decimal places.
The weights are \(1/0.25=4\) for type A and \(1/0.50=2\) for type B. The weighted converted count is \(60(4)+20(2)=240+40=280\); the weighted total is \(150(4)+200(2)=600+400=1{,}000\). The weighted rate is \(280/1{,}000=0.28\), also equal to \(0.60(0.40)+0.40(0.10)\). The weighted price contrast is \(0.28-0.44=-0.16\); the raw contrast is approximately \(0.2286-0.50=-0.2714\). Weighting corrects the type-mix distortion here because type captures selection-related differences and the assumptions hold. It would not correct unobserved selection or price confounding.
Selection Models and Design Information
When inclusion depends on unobserved factors, analysts sometimes use a selection model. One familiar approach is the Heckman model for a continuous outcome observed only when \(S=1\). It assumes a latent selection index and a joint distribution for the unobserved parts of the selection and outcome equations. It is not a universal correction for binary purchase outcomes; binary selection and binary demand generally require an appropriate joint binary model and its own identifying assumptions.
The correction term appears because observing \(Y\) requires \(u>-Z\gamma\). Under the joint-normal assumption, the conditional mean of the outcome error is \(E[\epsilon\mid u>-Z\gamma]=\rho\sigma_\epsilon E[u\mid u>-Z\gamma]\). For a standard normal \(u\), \(E[u\mid u>-t]=\phi(t)/\Phi(t)\), giving the stated inverse Mills ratio term. When \(\rho=0\), selection does not shift the conditional mean through correlated errors; when \(\rho\ne0\), omitting the term generally leaves a selected-sample mean that differs from the latent population mean.
Worked Example: Applying a Selection-Equation Adjustment
A hypothetical insurer records a continuous requested coverage amount, in thousands of dollars, only for customers who complete an application. A fitted selection model uses \(S=1\) when \(Z\gamma+u>0\), with \(u\sim N(0,1)\). The fitted outcome equation is \(Y=80+12x+\epsilon\), with \(\epsilon\sim N(0,10^2)\) and \(\operatorname{Corr}(u,\epsilon)=-0.8\). Thus \(\rho\sigma_\epsilon=-8\). For a customer with \(x=1\), compare the selected-sample conditional means at selection indices \(Z\gamma=0\) and \(Z\gamma=1\) with the outcome-model mean. Assume the joint normal model above; no interest discounting or mortality assumption is relevant.
Outcome-model mean. At \(x=1\), \(X\beta=80+12(1)=92\) thousand dollars.
At index \(0\), \(\phi(0)=0.3989\) and \(\Phi(0)=0.5000\), so \(\lambda(0)=0.3989/0.5000=0.7978\), approximately. The selected-sample conditional mean is \(92-8(0.7978)=85.6176\) thousand dollars. Applying the correction adds back \(8(0.7978)=6.3824\), yielding \(92.0000\) thousand dollars.
At index \(1\), \(\phi(1)=0.2420\) and \(\Phi(1)=0.8413\), so \(\lambda(1)=0.2420/0.8413\approx0.2876\). The selected-sample conditional mean is \(92-8(0.2876)=89.6992\) thousand dollars. Adding back the selection term gives \(89.6992+2.3008=92.0000\) thousand dollars.
The adjustment is larger at index zero because selection is less certain there and the selected observations are more strongly shifted by the correlated errors under the model. The common mistake is to interpret the selected conditional mean as the population outcome mean without checking whether the selection and outcome errors are correlated. These calculations illustrate the fitted model, not proof that joint normality or the selection equation is correct.
Practical Checks and Exam Traps
Selection-model identification is often fragile. An exclusion restriction—a variable that affects selection but has no direct effect on the outcome after controls—can help distinguish the selection equation from the outcome equation. For example, an administrative routing rule might affect whether a prospect receives a recorded quote. Its suitability depends on the actual process: if it also changes price, service, or customer mix in ways that affect demand, it is not a valid exclusion. Without credible design information, identification may rely heavily on functional-form and distributional assumptions.
- Do not confuse quote inclusion with purchase. A nonquoted prospect is not necessarily a nonbuyer; their purchase outcome under a quote may be unobserved.
- Do not treat renewal-only records as all policyholders. Policies that reach renewal may differ systematically from those that cancel or lapse earlier.
- Do not use weights without the denominator population. Estimating inclusion probabilities requires information about who was eligible or otherwise at risk of being selected, not just the selected records.
- Do not claim selection correction solves confounding. If price assignment is related to potential demand after adjustment, a selection weight alone does not identify a causal price effect.
- Do not extrapolate across zero-probability groups. If a subgroup could not receive a quote at a price, the data do not reveal its response there without additional assumptions or evidence.
The practical value of this diagnosis is substantial: a demand curve estimated from selected records can lead to poor price decisions if its population and selection mechanism are misunderstood. Report which population the estimate describes, how inclusion was measured, which assumptions support the correction, and how sensitive the result is to plausible changes in those assumptions. This topic is aligned with the CAS Exam 8 (Advanced Ratemaking) syllabus.
Check Your Understanding
For each problem, identify the relevant selection issue and calculate or state the requested result.
- A target population is 70% type X and 30% type Y. At a particular price, quote probabilities are 0.40 and 0.80, while conversion probabilities are 0.50 and 0.20. Find the raw selected-sample rate and the target rate, assuming selection is independent of conversion within type. (Answer: raw rate \(0.188/0.52=0.3615\); target rate \(0.41\).)
- A set of prospects has zero probability of receiving a quote under one price rule. Can inverse-probability weighting alone recover their conversion rate under that rule? (Answer: No; positivity fails, so their response is not identified from the selected data by weighting alone.)
- In a Heckman model, \(\rho=0\). What is the selection correction term in the conditional mean, and what does that imply under the model? (Answer: The term is zero; selection does not shift the outcome conditional mean through correlated selection and outcome errors under the model.)
- An insurer assigns higher prices to customers with higher predicted propensity to buy, and its dataset also includes only prospects who received quotes. Name the two distinct issues and one type of information needed to address each. (Answer: Price confounding requires sufficient adjustment or credible exogenous price variation; quote-sample selection requires information on the eligible population and inclusion mechanism, plus valid selection assumptions or design information.)
- A target population has equal numbers of groups A and B. At a lower price, conversion is 0.70 for A and 0.30 for B; at a higher price, it is 0.50 for A and 0.20 for B. If the target group mix is unchanged, calculate the target conversion rates and contrast. (Answer: lower-price rate \(0.50\), higher-price rate \(0.35\), contrast \(-0.15\).)