What a Pilot Can—and Cannot—Tell You
A pilot study can reveal whether a recruitment pathway is workable, how many cells a protocol tends to recover per donor, and how much a donor-level measurement varies. Those quantities affect the feasibility of a larger single-cell study. But a pilot is not a miniature version of the final study in every important sense: a small set of donors provides imprecise estimates, and the pilot may use a selected group or a protocol that changes during setup.
The unit of replication matters. For planning, recruitment is often summarized per person approached, while cell yield is summarized per donor or sample. Thousands of cells from one donor do not turn that donor into thousands of independent biological replicates. The earlier material on independent donor counts and cell allocation provides the relevant design context; here the focus is on what pilot observations can estimate and how uncertain those estimates are.
Before using any estimate, specify the target population, the denominator, the unit, and the process to which it applies. A consent rate among people already screened as eligible is not the same quantity as the yield of usable donors from everyone initially contacted. A cell yield measured after one laboratory protocol is not automatically applicable after a protocol change.
Recruitment Feasibility as a Proportion
Suppose each eligible person invited to participate either consents or does not. If the invited people are independent and have a common probability \(p\) of consenting under the recruitment process being evaluated, then the number who consent among \(n\) invitations has a binomial model. The observed proportion is \(\hat{p}=x/n\), where \(x\) is the number who consent. This model assumes the pilot denominator and outcome are defined consistently and that the pilot invitations are informative about the intended recruitment setting.
For small samples, a Wilson interval is often preferable to the simple Wald interval \(\hat{p}\pm1.96\sqrt{\hat{p}(1-\hat{p})/n}\), which can have poor coverage and can even extend beyond zero or one. For a 95% Wilson interval, let \(z=1.96\). Its center and half-width are:
Worked Example: Estimating a Consent Rate
In a hypothetical pilot, 12 of 18 eligible people invited to participate consent. The estimated consent probability is \(\hat{p}=12/18=0.6667\). Assume these invitations reasonably represent the intended recruitment process and that each person's consent outcome can be treated as independent with a common probability.
For a 95% Wilson interval, \(z=1.96\) and \(z^2=3.8416\). The denominator is \(1+3.8416/18=1.2134\). The center is \((0.6667+3.8416/36)/1.2134=0.6374\). The half-width is \(1.96\sqrt{(0.6667)(0.3333)/18+3.8416/(4\cdot18^2)}/1.2134\approx0.2000\).
The interval is approximately \(0.6374\pm0.2000\), or \([0.4374,0.8374]\). Under repeated sampling from the same recruitment process, this procedure produces intervals that cover the true consent probability about 95% of the time. It does not mean there is a 95% probability that this fixed, unknown probability lies in this particular interval.
If the planning target is 30 consenting participants, the pilot point estimate suggests inviting \(30/0.6667\approx45\) people. That figure is not a guarantee. Using the lower confidence limit as a cautious scenario gives \(30/0.4374\approx68.6\), or about 69 invitations. This is a planning illustration: it assumes the same eligibility and recruitment process will continue, and does not model any other source of loss.
Estimating Cell Yield per Donor
Let \(X_i\) be the number of usable cells recovered from donor \(i\) under a specified protocol. With \(n\) independent, representative donor-level observations, the sample mean \(\bar{x}\) estimates the mean yield \(\mu\), and the sample standard deviation \(s\) estimates the spread of yields across donors. A confidence interval for the mean can use a t distribution when donor yields are independent draws from an approximately normal population. This is a substantive assumption, especially when \(n\) is small: a single unusually high or low yield can strongly affect both \(\bar{x}\) and \(s\).
The interval quantifies uncertainty in the average yield, not the range expected for individual donors. A prediction interval for an individual donor is wider because it must account for both uncertainty in the mean and genuine donor-to-donor variation. For feasibility, it is often useful to examine the observed donor yields as well as the interval for their mean.
Worked Example: A Small Pilot’s Mean Cell Yield
A hypothetical pilot records usable-cell yields of 5,100, 6,545, 8,455, and 9,900 cells from four donors. These yields use the same processing protocol, and assume for this calculation that the donors are independent and representative and that the donor-level yield distribution is approximately normal.
The sample mean is \(\bar{x}=(5{,}100+6{,}545+8{,}455+9{,}900)/4=7{,}500\) cells per donor. The deviations from the mean are \(-2{,}400\), \(-955\), \(955\), and \(2{,}400\) cells. Thus the sum of squared deviations is \(2(2{,}400^2)+2(955^2)=13{,}344{,}050\) cells squared, and \(s=\sqrt{13{,}344{,}050/3}\approx2{,}109.03\) cells per donor.
For a 95% interval, \(df=4-1=3\) and \(t_{0.975,3}=3.182\). The standard error is \(SE=s/\sqrt{4}=2{,}109.03/2\approx1{,}054.52\) cells per donor. Using \(s\) rounded to 2,109, the margin is \(3.182(2{,}109)/2=3{,}355.419\) cells per donor. The interval is therefore about \(7{,}500\pm3{,}355.419\), or 4,145 to 10,855 cells per donor. Using the unrounded standard deviation gives the same endpoints to the nearest cell.
This interval is wide: four donors do not pin down the mean yield precisely. It describes uncertainty about the mean under the stated assumptions, not a promise that every donor will yield between 4,145 and 10,855 cells. The observed range, 5,100 to 9,900 cells, also warns that donor yields vary substantially.
Between-Donor Variability Is Especially Uncertain
For planning a comparison, the between-donor variance can matter as much as the mean. If \(s^2\) is the sample variance from \(n\) independent observations from a normal population with variance \(\sigma^2\), then \((n-1)s^2/\sigma^2\) has a chi-square distribution with \(n-1\) degrees of freedom. This exact result depends on normality and independence; technical replicates or many cells from one donor do not substitute for independent donors.
Let \(\chi^2_{q,df}\) denote the \(q\) quantile of the chi-square distribution with \(df\) degrees of freedom. Inverting the central 95% probability statement for this statistic gives the variance interval below. The lower endpoint uses the upper chi-square quantile because a larger denominator implies a smaller variance, and vice versa.
Taking square roots of both endpoints gives an interval for the population standard deviation \(\sigma\), in the original measurement units. The interval is often much wider than a single reported estimate suggests, particularly when there are few donors.
Worked Example: An Interval for Between-Donor Standard Deviation
Suppose a hypothetical pilot has six independent donor-level measurements on a log-expression contrast scale: 4, 4, 6, 6, 8, and 8 units. Assume these observations are representative and approximately normal. Their mean is 6 units. The deviations are \(-2,-2,0,0,2,2\), so the sum of squares is \(16\) units squared and \(s^2=16/(6-1)=3.20\) units squared.
For \(df=5\), use \(\chi^2_{0.025,5}\approx0.8312\) and \(\chi^2_{0.975,5}\approx12.8325\). The 95% interval for the variance is \([5(3.20)/12.8325,\;5(3.20)/0.8312]=[16/12.8325,\;16/0.8312]\approx[1.247,19.25]\) units squared.
Taking square roots gives a 95% interval for the standard deviation of approximately \([\sqrt{1.2468},\sqrt{19.2493}]=[1.117,4.387]\) units. The point estimate is \(s=\sqrt{3.20}\approx1.789\) units, but the interval shows that six donors leave substantial uncertainty about the true between-donor spread.
The justification for the variance interval is direct. Under the normal model, the statistic \(Q=(n-1)s^2/\sigma^2\) satisfies \(P(\chi^2_{0.025,df}\le Q\le\chi^2_{0.975,df})=0.95\). Substituting the expression for \(Q\), then taking reciprocals of positive quantities, reverses the inequalities involving the chi-square quantiles. Rearranging for \(\sigma^2\) yields \(P((n-1)s^2/\chi^2_{0.975,df}\le\sigma^2\le(n-1)s^2/\chi^2_{0.025,df})=0.95\). This establishes the stated interval when the normal, independent sampling model holds.
Diagnostics, Bias, and Choices for Planning
A pilot’s estimates are only useful to the extent that its observations represent the process being planned. Inspect the donor-level yields rather than only reporting a mean and standard deviation. A strong right tail, a single extreme value, or clusters corresponding to different processing runs can make a normal-theory interval unreliable or make one pooled estimate hard to interpret. With very few donors, plots and records can expose concerns but cannot establish that the distribution is normal or that the donors are representative.
For recruitment, check that the denominator includes everyone to whom the reported rate is meant to apply. If the pilot reports consent among eligible people but the planning question concerns all people initially contacted, those are different rates. Also compare recruitment settings: a highly motivated volunteer group, a single clinic, or a period with unusually intensive outreach may not represent the planned study.
A small pilot can be biased as well as imprecise. Donors may have been selected for expected sample quality; unsuccessful protocol attempts may be absent from the yield summary; or laboratory procedures may improve after early runs. These features can make observed recruitment or yield too optimistic for the main study. Conversely, an unusually difficult initial setup can make a pilot pessimistic. A confidence interval describes sampling uncertainty under a model; it does not correct selection bias, measurement changes, or a mismatch between pilot and target populations.
| Planning question | Useful pilot summary | Key caution |
|---|---|---|
| How many people may need to be invited? | Recruitment proportion with a binomial interval | Use the denominator that matches the recruitment stage being planned |
| What average cell yield is plausible? | Donor-level mean and an uncertainty interval | A mean interval is not an individual-donor prediction interval |
| How variable may donor outcomes be? | Donor-level variance or standard deviation with uncertainty | Normal-theory variance intervals are sensitive to nonnormality and small samples |
For a mean under a plausible normal model, the t interval is a transparent small-sample approach. A normal interval using a known population standard deviation is not appropriate when that standard deviation is unknown and estimated from only a few donors. A bootstrap can be useful when there are enough independent donors and a suitable resampling scheme, but resampling a handful of donors does not create new donor information; bootstrap intervals can be unstable or misleading in a tiny pilot. For recruitment proportions, Wilson or exact binomial intervals are preferable to relying on a simple Wald interval when counts are small.
In practice, treat pilot quantities as inputs with uncertainty, not fixed truths. Report the point estimate, its uncertainty, the raw donor-level observations or counts, and the assumptions. For planning, compare a central scenario with more cautious values—for example, a lower plausible recruitment probability, a lower plausible yield, or a larger plausible between-donor standard deviation. If those choices materially change feasibility, the pilot has identified an important uncertainty rather than a single definitive design answer.
Common Mistakes
- Treating cells as independent donors. Cells are observations nested within donors; cell count does not replace biological replication.
- Reporting a pilot point estimate as a guarantee. A mean yield or recruitment proportion is uncertain, especially when the pilot is small.
- Confusing a confidence interval for the mean with a range for individuals. The interval in the cell-yield example concerns the population mean, not the yield every donor will achieve.
- Assuming an interval fixes bias. A narrow interval from a selected or mismatched pilot can still be misleading for the intended study.
- Ignoring units and denominators. State whether a yield is cells per donor or cells per sample, and exactly which individuals form the recruitment-rate denominator.
Check Your Understanding
Answer each question, then compare with the solutions below.
- A hypothetical recruitment pilot observes 8 consents among 20 eligible people invited. What is the observed consent proportion, and what assumption is needed to apply a binomial model?
- Why does a confidence interval for mean cells per donor not describe the likely range of yields for individual donors?
- A pilot variance estimate is based on five independent, approximately normal donor measurements. What are the degrees of freedom for the chi-square variance interval?
- Name one way a pilot could yield a biased cell-yield estimate even if its confidence interval calculation is correct.
Solutions. (1) The observed proportion is \(8/20=0.40\), or 40%. A binomial model assumes independent outcomes with a common consent probability under the recruitment process being evaluated. (2) The mean interval quantifies uncertainty about the population average; individual donors also vary around that mean, so an individual prediction interval must include that variation and is wider. (3) The degrees of freedom are \(n-1=5-1=4\). (4) For example, selecting only donors expected to provide high-quality samples can make the observed yield unrepresentative. Correct interval arithmetic cannot remove that selection bias.