Why Survey Design Matters for Transport
Transporting a causal effect requires the target covariate distribution, not merely a sample that happens to contain target-population members. If target covariates come from a complex probability survey, respondents can have different inclusion probabilities, and observations may be clustered or stratified. Treating those respondents as a simple random sample can misrepresent the target distribution and understate or overstate uncertainty.
Two kinds of weights therefore enter the analysis for different reasons. Survey design weights represent target-population members who were more or less likely to be sampled. Transport weights reweight source-study participants so their covariate distribution resembles the target population. They are not interchangeable. The design weights help estimate the target distribution; transport weights connect that distribution to the source study.
Identification and Two Estimation Routes
The target causal effect is identified from source-study data and target covariates under substantive assumptions. For each treatment level \(a\), define \(m_a(x)=E(Y\mid A=a,X=x,S=1)\). In a source study with consistency, treatment positivity, and valid treatment exchangeability given \(X\), this conditional mean identifies the source conditional mean of the potential outcome \(Y^a\). Transport requires conditional exchangeability across study participation: \(E(Y^a\mid X=x,S=0)=E(Y^a\mid X=x,S=1)\). It also requires treatment positivity in the source study and participation overlap: covariate values in the target must be represented in the source study. Under these assumptions, the target mean under treatment \(a\) is
Here \(E_T\) averages over the target population. The target survey estimates this average with design weights. If the target sample has \(J\) respondents and \(d_j\) is respondent \(j\)'s survey weight, the direct-standardization estimator is
This route uses the survey weights to estimate the target covariate average of source-study outcome predictions. The prediction model must be fit using source-study data and must adequately represent the conditional means needed for standardization. Survey weighting does not repair a misspecified outcome model or a failure of causal identification.
A second route uses inverse odds of sampling weights. Estimate the relative density of target covariates to source-study covariates, \(f_T(x)/f_S(x)\), and weight source-study participants by that ratio. A practical approach fits a model distinguishing target-survey records from source-study records, using target records with their survey weights. The resulting odds are proportional to the density ratio, up to a constant that cancels when weights are normalized. If the source study also used unequal sampling, its design weights must be incorporated as appropriate to represent the source-study population.
The survey weights appear when estimating the target covariate distribution or fitting the target-versus-source sampling model. The transport weights then apply to source-study observations. Multiplying a target respondent's survey weight by a source participant's transport weight is not a meaningful operation: the weights attach to different records and serve different roles.
Worked Examples
Worked Example: Design-Weighted Standardization
Suppose a hypothetical source study provides estimated conditional mean outcomes under treatment and control for four covariate profiles. A target probability survey includes one respondent from each profile. The values below are hypothetical predictions and survey weights.
| Target profile | Survey weight \(d_j\) | \(\widehat m_1(X_j)\) | \(\widehat m_0(X_j)\) |
|---|---|---|---|
| 1 | 2 | 8 | 6 |
| 2 | 1 | 10 | 7 |
| 3 | 3 | 12 | 8 |
| 4 | 2 | 14 | 9 |
Assume the source outcome model identifies the conditional potential-outcome means, conditional transport exchangeability holds, and each target profile is represented in the source data. The weighted target total is \(2+1+3+2=8\). Thus,
and
The estimated target-population treatment effect is \(11.25-7.625=3.625\) outcome units. For example, if \(Y\) is a score, this is an estimated average increase of 3.625 score points in the target population. The result is a causal estimate only if the causal and transport assumptions hold; the arithmetic alone does not establish them.
Worked Example: Combining Survey Margins and Source Transport Weights
Consider a hypothetical target population divided into low- and high-risk profiles. A complex survey, analyzed with its design weights, estimates that 75% of the target population is low-risk and 25% is high-risk. In the source study, each profile accounts for half the participants. Suppose the source study supports the following conditional treatment and control means, with treatment exchangeability and adequate positivity within profiles.
| Profile | Target share | Source share | Ratio: target share / source share | Treatment mean | Control mean |
|---|---|---|---|---|---|
| Low-risk | 0.75 | 0.50 | 1.50 | 10 | 8 |
| High-risk | 0.25 | 0.50 | 0.50 | 16 | 11 |
The design-weighted survey shares give the covariate distribution to which the source data must be transported. The source-to-target ratios are \(0.75/0.50=1.50\) for low-risk participants and \(0.25/0.50=0.50\) for high-risk participants. Their source-share-weighted average is \(0.50(1.50)+0.50(0.50)=1\), so they already have mean one in this example.
Applying these ratios to the source conditional means gives the transported treatment mean
The transported control mean is
Therefore, the transported effect is \(11.50-8.75=2.75\) outcome units. The survey design weights entered when estimating the target shares; the resulting target-to-source ratios entered when reweighting source-study information. In a real analysis, estimated shares and ratios have sampling uncertainty, which must also be reflected in the interval estimate.
Worked Example: Design-Based Uncertainty for a Stratified Survey Mean
Suppose a hypothetical target survey selects a simple random sample without replacement within each of two strata. Each stratum contains \(N_h=4\) target units and samples \(n_h=2\), so the sampling fraction is \(f_h=n_h/N_h=0.5\). The measured quantity is a source-model prediction, and in each stratum the two sampled values are 4.9 and 5.1 in the first stratum, and 6.9 and 7.1 in the second. Each sample variance is \(s_h^2=0.0200\), since the sample mean is 5 or 7 and the deviations are \(-0.1\) and \(0.1\).
Under stratified simple random sampling without replacement, the estimated variance of the estimated population total is
For either stratum, the contribution is
The two strata therefore contribute \(0.0800+0.0800=0.160\) to the estimated variance of the total. The estimated population mean is \((4(5)+4(7))/8=6\). Dividing the total variance by \(N^2=8^2=64\) gives the estimated mean variance and standard error:
This calculation uses the stated stratified sampling design and its finite-population correction. It is not the variance formula for a general clustered survey. If the target survey instead samples primary sampling units, variance estimation must reflect those clusters and strata.
Carrying the Design Through Uncertainty Estimation
A standard error for a transported effect must account for more than the variability of outcomes in the source study. It can also reflect uncertainty in the target covariate distribution, estimation of the transport weights or outcome model, and the survey's sampling design. Ignoring the target design treats the observed survey respondents as if they were independent, equally likely draws, which may be inappropriate.
For a complex probability sample, use the variance method supplied or justified for that design. Common approaches include Taylor linearization and replicate weights such as jackknife, balanced repeated replication, or bootstrap replicate weights. The exact method and scaling factors depend on how the survey was designed and documented. If replicate weights are supplied, a robust procedure is to recompute the entire transport estimator for each replicate: refit any target-versus-source sampling model, update transport weights or outcome-model standardization as needed, and recalculate the final effect.
When the source study is also sampled or randomized, uncertainty from that source data must be included as well. One can use a joint resampling procedure that respects the source design and the target survey design, or combine variance components when independence and the estimator's structure justify doing so. Simply attaching a survey standard error to a point estimate while treating estimated source effects and transport weights as fixed will generally omit relevant uncertainty.
Specify the target population, treatment contrast, outcome scale, and covariates needed for transport.
Use the survey weights to estimate target covariate summaries or to construct the weighted target-versus-source sampling model. Retain strata, clusters, and replicate-weight information.
Use design-weighted target standardization or transport-weighted source outcomes, checking causal assumptions and covariate overlap.
Use an appropriate design-based variance method and refit estimated components within replicates or resamples when required.
Common Mistakes and Practical Interpretation
- Using survey weights as if they were transport weights. Survey weights represent target population members through the sampling design. Transport weights align source-study covariates with the target. They have different denominators and attach to different observations.
- Ignoring the survey design after calculating a weighted mean. The point estimate may use weights correctly, but an ordinary independent-observation standard error can still be wrong when the survey is clustered or stratified.
- Assuming weighted data guarantee causal validity. Design weights address unequal survey inclusion, not unmeasured confounding, unmeasured effect modification, or lack of overlap. Those require their own assumptions and diagnostics.
- Treating estimated weights as known constants. When a transport model or target covariate distribution is estimated, uncertainty in those estimates can contribute to uncertainty in the final effect.
A confidence interval should be interpreted as an interval produced by a procedure whose repeated-sampling behavior accounts for the relevant study and survey designs, subject to the modeling and identification assumptions. Its width is not determined by the nominal number of target respondents alone: unequal weights and clustering can reduce effective information, while stratification can improve precision for some estimands.
Check Your Understanding
Answer each question by identifying both the weighting role and the assumption or design feature involved.
- A target survey oversamples rural residents. Which weight accounts for this when estimating the target rural share, and why is that not itself a source-study transport weight?
- In direct standardization, what quantity is averaged over the weighted target survey to estimate the target mean under treatment \(a\)?
- A source study contains no participants with a covariate profile common in the target survey. Can a large survey weight resolve this support problem? Explain.
- Why might a standard error based on independent, equally weighted target respondents be inappropriate for a clustered probability survey?
- If replicate weights are provided, name two estimated components that may need to be recalculated in each replicate.