Pitching
Small Pilot Samples: Explain What the Numbers Cannot Prove
A small pilot can teach you what to test next without proving a market-wide conversion rate. Learn to explain the result and its limits.
What you will learn
- Report a pilot rate with its numerator and denominator.
- Explain sampling uncertainty without claiming a pilot is representative.
- Choose the next evidence needed for a decision.
A small pilot can reveal a valuable problem, a broken workflow or a promising behavior. It usually cannot establish a precise market-wide conversion rate. Show what happened, who was eligible and how much remains uncertain. A clear “3 of 10” is more useful than an unsupported “30% conversion.”
Start with the unit and observation window
Define whether the unit is a person, account, team or session. Then state who entered the pilot, the qualifying event, the time allowed to complete it and who could not be observed. Ten people at one customer are not automatically ten independent buying decisions.
Pilot evidence card
| Field | Fictional example |
|---|---|
| Unit | Ten independently recruited small-business accounts |
| Entry rule | Agreed to test the same reporting task |
| Outcome | Completed the task without founder intervention |
| Window | Seven days after access |
| Observed result | 3 completed; 7 did not complete |
| Recruitment limit | Convenience sample from the founder’s network |
| Open question | Would independently sourced buyers behave similarly? |
See how one account changes the headline
Fictional rate sensitivity
| Outcome | Calculation | Observed rate |
|---|---|---|
| 2 completions | 2 / 10 | 20% |
| 3 completions | 3 / 10 | 30% |
| 4 completions | 4 / 10 | 40% |
| 30 completions in a larger cohort | 30 / 100 | 30% |
With ten accounts, one result moves the rate by ten percentage points. With one hundred, it moves it by one point. A larger denominator can reduce sampling uncertainty, but it does not repair biased recruitment, a changed task or missing outcomes.
What a confidence interval can and cannot say
For an independent binomial model, a 95% Wilson interval around 3 of 10 is approximately 10.8% to 60.3%. For 30 of 100 it is approximately 21.9% to 39.6%. The NIST handbook explains proportion intervals and why a simple symmetric approximation can be misleading for small samples.
These are illustrative calculations under that model, not valid market error bars for a convenience pilot. The interval describes a method’s repeated-sampling coverage under assumptions; it is not a 95% probability that a fixed true rate sits inside this one interval. Recruitment and measurement limits still need a separate explanation.
Decide what the pilot actually taught you
Separate findings from inferences
| Evidence | Reasonable interpretation | Not established |
|---|---|---|
| Three independent accounts completed the task | The workflow is possible for these accounts | 30% of all target buyers will adopt |
| Seven accounts stopped at the same setup step | A specific friction deserves investigation | Removing the step will guarantee conversion |
| Users praised the prototype | The concept attracted interest | They will purchase or renew |
Write an investor-ready statement
A useful fictional statement is: “Three of ten pilot accounts completed the reporting task within seven days. Recruitment came through our network. We are testing the setup bottleneck and recruiting an independently sourced cohort before estimating broader adoption.” That combines momentum with an accountable next step.
Connect this result to pre-revenue traction, rather than presenting a pilot percentage as revenue or product-market fit.
Practice: choose the next test
If the question is whether the product can complete the task, investigate the failure cases. If it is whether buyers will pay, run a clearly defined purchase test. If it is repeat use, observe a mature cohort. More of the wrong data will not settle a different question.
Keep pilot evidence, investor questions and follow-up tasks together in Finta so the next update can show what changed. Use account-level summaries you are authorized to share, not identifiable customer data pasted into a public deck.