Why Demand Inference Must Respect Dependence
Demand data often contain repeated or grouped observations. A customer may receive several quotes, an agent may handle many customers using the same sales approach, or a time period may expose many observations to one market shock. If observations in the same group share unobserved influences, treating every row as independent can make standard errors too small. The coefficient estimates may still be useful, but their reported uncertainty can be misleading.
The practical decision is not simply whether a data set has repeated rows. It is which units can share unobserved shocks and which units can reasonably be treated as independent for the inference being made. Customer clustering is suited to repeated observations of the same customer; agent clustering is suited to shared agent-level influences; and time clustering is suited to common shocks within a time period. The choice follows the data-generating process and the question of interest.
For example, if a customer contributes multiple renewal quotes, the customer's unmeasured price sensitivity may affect every quote. If each customer contributes only one quote but an agent handles many customers using a common script, the agent may be the relevant dependence unit instead. A time cluster can be appropriate when observations in a month share a campaign, weather shock, or market condition. The rows are not automatically the right units for calculating uncertainty.
The Cluster-Robust Sandwich Estimator
Let \(\hat{\theta}\) be an estimate from a demand model defined by estimating equations. For observation \(i\), write its contribution to the estimating equation as \(\psi_i(\theta)\), and let cluster \(g\) contain the set of observations \(i\in g\). The fitted cluster score is the sum of the observation-level contributions within that cluster. In an ordinary least squares demand regression, that contribution is \(x_i e_i\), where \(x_i\) is the regressor vector and \(e_i\) is the fitted residual.
Here, \(G\) is the number of clusters, \(A\) is the negative derivative of the total estimating equation with respect to the parameters, evaluated at \(\hat{\theta}\), and \(A^{-\mathsf{T}}\) is the transpose of \(A^{-1}\). This is the cluster-robust sandwich covariance estimator. In a fitted linear regression with design matrix \(X\), \(A=X^\mathsf{T}X\), and \(U_g=X_g^\mathsf{T}e_g\), where \(X_g\) and \(e_g\) are the design matrix and residual vector for cluster \(g\).
A commonly used finite-sample adjustment multiplies this covariance estimator by \(c=\frac{G}{G-1}\frac{N-1}{N-p}\), where \(N\) is the number of observations and \(p\) is the number of estimated parameters, including the intercept. This adjustment is not a substitute for having enough independent clusters; with very few clusters, conventional cluster-robust inference can remain unreliable. Different software may use different corrections, so the reported method should be checked.
Worked Example: Calculating a Clustered Slope Standard Error
A hypothetical insurer fits a linear demand model with an intercept and log price. The fit uses \(N=12\) quotes in \(G=4\) independent agent clusters, and \(p=2\) parameters. The fitted bread matrix, \((X^\mathsf{T}X)^{-1}\), is diagonal with entries \(1/4\) for the intercept and \(1/2\) for the log-price coefficient. The four fitted cluster score vectors, in intercept-then-slope order, are \((1,2)\), \((-1,1)\), \((2,-1)\), and \((-2,-2)\). Calculate the finite-sample-corrected standard error of the log-price coefficient.
Method. Sum the outer products of the cluster scores to obtain the meat. Then apply the bread on both sides and the stated finite-sample correction.
The meat is \((1,2)^\mathsf{T}(1,2)+(-1,1)^\mathsf{T}(-1,1)+(2,-1)^\mathsf{T}(2,-1)+(-2,-2)^\mathsf{T}(-2,-2)\), giving \(\begin{pmatrix}10&3\\3&10\end{pmatrix}\). The uncorrected covariance is therefore \(\begin{pmatrix}1/4&0\\0&1/2\end{pmatrix} \begin{pmatrix}10&3\\3&10\end{pmatrix} \begin{pmatrix}1/4&0\\0&1/2\end{pmatrix} = \begin{pmatrix}0.625&0.375\\0.375&2.5\end{pmatrix}\).
The uncorrected slope variance is \(2.5\). The correction is \(c=\frac{4}{3}\frac{11}{10}=\frac{22}{15}=1.4667\) (rounded to four decimal places). Thus the corrected slope variance is \(2.5(22/15)=11/3=3.6667\), and the corrected standard error is \(\sqrt{11/3}=1.9149\) (rounded to four decimal places).
Check. The cluster scores sum to \((0,0)\), as expected for the fitted intercept-and-slope regression score equations. The standard error is in the same units as the log-price coefficient. Common mistake avoided: treating the 12 quotes as 12 independent clusters would use observation-level rather than agent-aggregated scores and would not allow within-agent dependence.
Choosing the Dependence Unit
Cluster at the level where the dependence arises, not at the level that happens to be easiest to identify in the data. In demand models, the most common candidates are customers, agents, and time periods. The table summarizes typical reasoning, not a rule that one cluster choice fits every data set.
| Candidate cluster | When it may be appropriate | Example shared influence |
|---|---|---|
| Customer | A customer contributes repeated quotes, purchases, or renewal observations. | Persistent, unmeasured price sensitivity for one customer. |
| Agent | An agent handles multiple customers who may be affected by a common agent-level influence. | A shared sales approach or negotiation practice. |
| Time period | Observations within a period may share a market or campaign shock. | A monthly competitor promotion affecting many quotes. |
An observation may belong to more than one meaningful dependence dimension. For example, customers may appear repeatedly and agents may also affect multiple customers. One-way clustering by customer does not automatically account for common agent shocks, and one-way clustering by agent does not automatically account for repeated-customer dependence. If dependence operates along multiple dimensions, an appropriate multiway method or a design-specific analysis may be needed. Do not claim that a single clustering choice handles every source of dependence.
Worked Example: Clustering When Agents Share a Sales Approach
A hypothetical insurer records 96 separate customer quotes. Each customer appears once, but 8 agents each handle 12 customers, and each agent uses a common sales approach. The fitted demand regression has \(p=3\) parameters. For inference about the price coefficient, identify the appropriate one-way cluster, calculate the finite-sample covariance multiplier, and specify the resampling unit for a cluster bootstrap.
Step 1: Choose the cluster. Customers appear once, so repeated observations of the same customer are not the stated concern. The shared sales approach can affect multiple customers handled by the same agent. Use agent clusters: \(G=8\), with \(N=96\) observations.
Step 2: Calculate the correction. \(c=\frac{G}{G-1}\frac{N-1}{N-p} =\frac{8}{7}\frac{95}{93} =\frac{760}{651}=1.1674\) (rounded to four decimal places). Standard errors are multiplied by \(\sqrt{c}=\sqrt{1.1674}=1.0805\) (rounded to four decimal places), relative to the uncorrected cluster-robust standard errors.
Step 3: Set up the bootstrap. Sample 8 agents with replacement and include all 12 quote records for every selected agent. Refit the model to that resampled data set. Resampling 96 individual quotes instead would break up the groups whose shared influence motivates the analysis.
Check. Each bootstrap sample contains 8 agent selections, with repeated selections represented by repeated copies of that agent's records. There are only 8 independent clusters, so the correction alone does not guarantee reliable large-sample inference; the small cluster count must be reported and considered. Common mistake avoided: clustering by customer merely because the response is a customer quote ignores the stated agent-level dependence.
Cluster Bootstrap for Curves and Elasticities
A cluster bootstrap estimates uncertainty by resampling entire clusters with replacement, refitting the demand model in each resample, and recalculating the quantity of interest. A bootstrap replicate must preserve the within-cluster records together. Depending on the model, the quantity recalculated might be a predicted demand probability at a specified price, a value on a response curve, or an elasticity derived from the fitted model. The earlier material on delta-method uncertainty for price elasticities provides a different route to uncertainty for a derived elasticity; the cluster bootstrap obtains its distribution by repeating the fitting and calculation steps under cluster resampling.
For \(G\) clusters, one bootstrap replicate draws \(G\) cluster labels with replacement from the observed labels. A cluster drawn twice contributes two copies of its full set of records. After refitting, the analyst records the prediction or elasticity. Repeating this procedure \(B\) times gives \(B\) estimates; their sample standard deviation is a bootstrap estimate of standard error. Percentile intervals can also be formed from the empirical quantiles, but their reliability depends on the number of independent clusters and the number of replicates. A small \(B\) is useful for showing arithmetic, not for a final analysis.
Worked Example: Bootstrapping a Response and an Elasticity
A hypothetical demand model is fitted to observations from 5 independent agent clusters. The original fitted model predicts demand of \(0.76\) at a price index of \(1.10\), and its fitted elasticity there is \(-1.20\). Five illustrative cluster-bootstrap refits produce the following results. The cluster selection counts are shown in the order A, B, C, D, E.
| Replicate | Cluster selection counts | Predicted demand at price index 1.10 | Elasticity at price index 1.10 |
|---|---|---|---|
| 1 | (2, 1, 1, 1, 0) | 0.78 | -1.05 |
| 2 | (0, 2, 1, 1, 1) | 0.75 | -1.30 |
| 3 | (1, 0, 2, 1, 1) | 0.80 | -0.95 |
| 4 | (1, 1, 0, 2, 1) | 0.72 | -1.45 |
| 5 | (1, 1, 1, 0, 2) | 0.76 | -1.25 |
Step 1: Verify the resampling unit. Each count vector sums to 5, so each replicate contains five cluster selections. A count of 2 means that the agent's complete record group is included twice; a count of 0 means it is omitted for that replicate.
Step 2: Estimate uncertainty in predicted demand. The five predictions have mean \((0.78+0.75+0.80+0.72+0.76)/5=3.81/5=0.762\). Their squared deviations from \(0.762\) sum to \(0.018^2+(-0.012)^2+0.038^2+(-0.042)^2+(-0.002)^2=0.00368\). The bootstrap sample variance is \(0.00368/(5-1)=0.00092\), so the bootstrap standard error is \(\sqrt{0.00092}=0.0303\) (rounded to four decimal places).
Step 3: Estimate uncertainty in elasticity. The five elasticities have mean \((-1.05-1.30-0.95-1.45-1.25)/5=-6.00/5=-1.20\). Their squared deviations from \(-1.20\) sum to \(0.15^2+(-0.10)^2+0.25^2+(-0.25)^2+(-0.05)^2=0.16\). Thus the bootstrap sample variance is \(0.16/4=0.04\), and the bootstrap standard error is \(\sqrt{0.04}=0.20\).
Check. The bootstrap average elasticity, \(-1.20\), matches the original fitted elasticity in this illustration; the bootstrap standard error is \(0.20\). The predicted-demand bootstrap standard error is \(0.0303\), in demand-probability units. Common mistake avoided: resampling individual rows would not preserve agent groups. In a practical analysis, use many more bootstrap replicates and enough independent clusters to support the inference; five replicates here serve only to make every calculation visible.
Practical Checks and Common Traps
Clustered standard errors and cluster bootstrapping address uncertainty under within-cluster dependence, but neither repairs an unsuitable model. A demand curve can still be misspecified, a price coefficient can still be confounded, and a fitted elasticity can still be sensitive to the chosen functional form. Treat the dependence adjustment as an inference choice, not as a substitute for the modeling and identification work.
- Do not confuse observations with independent units. A large row count can mask a small number of clusters. The number of independent clusters, not just the number of records, matters for uncertainty.
- Do not choose clusters by convenience. Use the level at which shared shocks or repeated measurements arise. If several dimensions create dependence, assess whether one-way clustering is adequate.
- Do not resample fragments of a cluster. In a cluster bootstrap, carry the whole cluster's records into each replicate, including all its repeated observations.
- Do not interpret a finite-sample correction as a cure for few clusters. With very few clusters, consider methods intended for small-cluster settings and be cautious about conventional standard errors and bootstrap intervals.
- Do not report only a band without its target. A pointwise interval for predicted demand at one price is not the same as an interval for an entire curve, nor is either automatically an interval for elasticity.
Check Your Understanding
For each problem, identify the dependence unit or calculate the requested clustered uncertainty quantity.
- A hypothetical insurer observes 180 customers, each with three renewal quotes in successive years. Assume persistent unobserved customer preferences can affect all three quotes, while customers are independent. Which unit should be used for one-way clustering, and how many clusters are there? (Answer: Cluster by customer; \(G=180\).)
- A model has \(N=50\) observations, \(G=5\) independent time-period clusters, and \(p=2\) estimated parameters. Calculate the stated finite-sample covariance multiplier \(c=\frac{G}{G-1}\frac{N-1}{N-p}\). (Answer: \(c=\frac{5}{4}\frac{49}{48}=\frac{245}{192}=1.2760\), rounded to four decimal places.)
- A fitted regression has three agent-cluster score vectors for its price coefficient of \(2,-1,-1\), and the corresponding scalar bread inverse is \(1/4\). Find the uncorrected cluster-robust variance and standard error for that coefficient. (Answer: Variance \(=(1/4)^2(2^2+(-1)^2+(-1)^2)=6/16=0.375\); standard error \(=\sqrt{0.375}=0.6124\), rounded to four decimal places.)
- A demand model uses 10 independent agent clusters. Each of \(B=200\) bootstrap replicates samples 10 agents with replacement, refits the model, and records predicted demand at a specified price. What is the bootstrap standard-error estimate, and what records must be resampled together? (Answer: The sample standard deviation of the 200 recorded predictions; all records belonging to each selected agent must be included together.)
- Each customer appears once, but several customers receive quotes from the same agent; the analyst believes agent practices and monthly market shocks both create dependence. Is one-way customer clustering sufficient, and what issue must be addressed? (Answer: No. Customer clustering does not account for the stated agent- or time-level dependence; assess a suitable multiway or design-specific method and the number of independent clusters in each relevant dimension.)