A pilot is worth running only if it can produce a number that would change your mind. A carpooling platform running a corporate pilot over two and a half months with several large employers reported downloads up sharply, a large majority of users finding a match, and most of those matches classified as high quality against a stated definition. That last part, the stated definition, is what made it a pilot rather than a demonstration.
Defining in advance what counts as success, and being specific about it, is the difference between evidence and a press release.
Define the success measure before you start
An excellent match defined as a deviation under a set distance or time is a testable claim. A pilot that reports enthusiasm has proven nothing, because enthusiasm was never going to be absent. Write down the threshold first, and be willing to conclude the pilot failed.
Pilot with organisations, not individuals
Running through employers gave the platform a concentrated pool of people travelling similar routes at similar times, which is exactly what a matching product needs to work. Any business whose product improves with density should find an existing group rather than recruiting individuals across a city.
Report the distribution, not only the average
Longest journey, shortest journey, average journey and the proportion with more than one match together describe the service far better than a single figure. Investors and partners read distributions because averages hide whether a product works for most users or works spectacularly for a few.
A pilot that works is a sales document
Structured results give the next corporate customer a reason to participate, which is why the pilot is worth doing even where you are confident of the outcome. Businesses selling to organisations should treat the first deployment as a case study being built, and agree upfront that the results can be published.
Know which rules apply before scaling
Shared transport arrangements sit near regulated passenger transport, and where money changes hands the position needs checking rather than assuming. Requirements are published on the government’s services portal, and establishing them during a pilot is far cheaper than after a rollout.
Frequently asked questions
What makes a pilot worth running?
That it can produce a number capable of changing your mind, with the success threshold defined before it starts.
Why pilot through organisations?
Because they supply a concentrated group with similar patterns, which is what any product improving with density needs to demonstrate value.
Why report the distribution rather than the average?
Averages hide whether a product works for most users or exceptionally for a few, and informed readers look for the spread.
Is a pilot worth doing if you already expect it to work?
Yes. Structured results become the sales document for the next customer, provided you agree upfront that they can be published.
What should be checked before scaling a transport product?
Whether the arrangement falls within regulated passenger transport, particularly where money changes hands, which is cheaper to establish during the pilot.
Further reading
Originally published in July 2017. Updated September 2026 to explain how to run a pilot that proves something, using the lead story from the original roundup.
