The most common pilot failure is not clinical—it is decision failure. A pilot runs, results look “promising,” and then it continues indefinitely because nobody agreed what would justify scaling, reshaping, or stopping. That creates two predictable harms: resources get trapped in perpetual pilot, and stakeholders lose trust that innovation produces decisions. In Pilot Evaluation & Learning Loops, the point is not learning for learning’s sake; it is learning that drives an explicit next step. And when you move toward New Service Models, decision discipline becomes non-negotiable because scaled services must be contracted, governed, and renewed based on defensible evidence.
Two oversight expectations typically shape stop/go/scale decisions. First, funders and boards expect explicit criteria and a documented rationale: what thresholds you set, what evidence you used, and why you proceeded or stopped. Second, they expect duty-of-care continuity during change: safety oversight, clear escalation rules, and a transition plan so service users are not stranded if the pilot ends or changes scope.
Define success and failure before the pilot “proves itself”
Stop/go/scale governance works best when you define it early—before you are attached to the pilot. The question is not “did we like it?” but “did it meet predefined criteria strongly enough to justify the next investment?” Criteria usually need to cover: delivery reliability (can it be done consistently), safety and risk (did risk increase or decrease), access and equity (who benefits and who is missed), and sustainability (workforce and partner viability). If criteria are invented after outcomes are known, decisions appear biased even when they are reasonable.
Use a stage-gate model that fits real service operations
A practical stage-gate model keeps decisions disciplined without slowing delivery. Many systems use three stages: Stage 1 (proof of delivery), Stage 2 (proof of effect), Stage 3 (proof of scale-readiness). Each stage has a short criteria set and a named decision forum with authority to proceed, proceed with conditions, reshape, or stop. “Proceed with conditions” is critical: it allows iteration while keeping accountability explicit. The key is that every review ends with a decision, not just discussion.
Operational Example 1: Monthly stage-gate decisions with explicit thresholds
What happens in day-to-day delivery
The pilot runs a monthly decision meeting with a fixed pack: performance on a small measure set (referral-to-first-contact, completion of follow-up for higher-risk cases, safety incidents and near-misses, documentation completeness), a summary of learning loop changes made that month, and operational capacity signals (staffing coverage, partner response times). Thresholds are written in advance and visible to staff. The outcome is recorded in a one-page decision log: decision category (proceed / proceed with conditions / reshape / stop), supporting evidence, named owners for conditions, and an agreed time window for reassessment.
Why the practice exists (failure mode it addresses)
Pilots often drift because “governance” becomes an informational forum with no clear authority. The failure mode is decision avoidance: nobody wants to stop, and nobody feels ready to scale, so the pilot persists by default. Explicit stage-gates exist to prevent that drift and to meet oversight expectations for accountable use of funds.
What goes wrong if it is absent
Without stage-gates, staff operate in permanent uncertainty. Referral partners receive shifting messages about eligibility and capacity. Leadership hesitates to invest in scale because evidence feels incomplete, yet also hesitates to stop because the pilot has goodwill. Eventually, decisions happen due to budget pressure rather than evidence, which is disruptive and undermines trust in future innovation. Operational fatigue increases because staff feel they are running a “not quite real” service indefinitely.
What observable outcome it produces
Stage-gates produce faster improvement cycles and defensible decisions. Evidence includes decision logs, criteria documents, and tracked completion of conditions (e.g., outreach redesign completed by a date, escalation pathway tightened, equity gap addressed). Commissioners can see whether the pilot is moving toward scale-readiness or whether persistent barriers justify reshaping or stopping. That clarity is often what unlocks future funding.
Operational Example 2: Scale-readiness playbooks that survive beyond pilot champions
What happens in day-to-day delivery
As the pilot approaches scale consideration, the team creates a scale-readiness playbook that captures the operational minimum: staffing model by expected volume, role boundaries, eligibility rules, referral and triage workflow, documentation templates, escalation pathways, partner interfaces, and supervision requirements. The pilot then tests transferability by onboarding a second team or micro-site using the playbook. Supervisors run structured observation and coaching (brief competency checklists, case reviews, documentation audits) to ensure the model is being delivered consistently and safely.
Why the practice exists (failure mode it addresses)
A pilot can look successful because a small, expert team carried it with tacit knowledge and informal relationships. The scale failure mode is “it worked there, but not elsewhere.” Playbooks and transfer tests exist to prevent scale based on heroics. Oversight bodies expect that scale-readiness includes replicability: the model works when delivered by competent teams who were not part of the original design.
What goes wrong if it is absent
If the model is not codified, early scaling creates variability immediately: staff interpret thresholds differently, documentation diverges, and partner agencies get inconsistent responses. Safety risk rises because escalation rules are unclear or applied inconsistently. Evaluation then shows “pilot worked, scale failed,” wasting investment and damaging trust. Teams also struggle to hold accountability because there is no reference standard for what “good delivery” looks like.
What observable outcome it produces
With a playbook and transfer test, the pilot can demonstrate stable performance across teams and reduced dependence on individual champions. Evidence includes the playbook, training and observation records, comparative metrics between the original and new team, and documented remediation where drift is found. Funders see that scalability is an operational property you tested, not an assumption.
Operational Example 3: Safe exit and transition planning when pilots stop or reshape
What happens in day-to-day delivery
The pilot maintains an exit protocol that treats stopping as a managed clinical/operational process, not a switch-off. If the pilot stops or changes scope, the team completes structured transitions: update plans, communicate clearly with participants and caregivers, hand off to standard services, and confirm follow-up where needed. The pilot tracks each transition against closure criteria (no unresolved escalations, risk issues handed to an accountable service, referrals completed, and documentation finalized). Partners receive a clear message on what is changing and where to refer going forward.
Why the practice exists (failure mode it addresses)
Stopping or reshaping can cause harm if participants lose support abruptly or if unresolved risks are left unmanaged. Oversight expectations include continuity and safeguarding: innovation should not compromise duty of care. Exit planning exists to prevent the failure mode where pilots are treated as optional extras rather than real care processes with real safety implications.
What goes wrong if it is absent
Without an exit protocol, people can “fall between” services, referral partners keep sending cases into a winding-down model, and staff close cases inconsistently. Operationally this drives complaints, safety incidents, and reputational damage that can overshadow any benefit the pilot delivered. It also harms innovation culture: leaders become risk-averse because the last pilot ended messily, even if the service design was sound.
What observable outcome it produces
A safe exit produces measurable continuity: high rates of documented handoffs, fewer post-exit urgent calls related to gaps in support, and clear closure documentation. Evidence includes transition logs, partner notifications, and post-transition checks for higher-risk participants. Counterintuitively, demonstrating you can stop safely increases willingness to fund pilots, because stakeholders trust that change is controlled.
Make the decision artifact as important as the dashboard
In practice, what survives audit, leadership turnover, and procurement scrutiny is the decision artifact: criteria, evidence summary, safety review, equity view, decision rationale, and next steps (scale plan or exit plan). It should be brief enough to read quickly but concrete enough to defend. If a pilot cannot produce that artifact, it is not scale-ready—no matter how “promising” the early data feels.
Perpetual pilots are not caution; they are missing governance. When stop/go/scale is explicit and documented, pilots become a disciplined pathway to better services and better contracts—not an endless experiment.