What should we measure during a pilot, and when do we expand or stop it?

A pilot should answer a business question, not prove that AI works. The measurement comes from the problem rather than from the technology. If people spend eight hours preparing a recurring report, measure preparation time before and after. If the issue is missed follow-ups, count missed follow-ups. Track adoption alongside the outcome, because a capable tool nobody uses produces nothing. Avoid counting prompts or documents generated, which measure activity rather than value.

Expanding is a higher bar than working. Scale changes the problem. More people means more training, more permissions, more support, and more ways to be wrong. Usage costs rise. A workflow that five enthusiastic volunteers loved can meet real resistance at five hundred users. Before expanding, go back to the business case and ask whether the outcome actually improved, whether users want it, whether the infrastructure holds at volume, whether governance is clear, and whether the economics still work. Scale on evidence, not on how impressive the demonstration was.

Stopping is the harder discipline. The most expensive management instinct is defending an initiative because money has already gone into it. The pilot exists specifically to generate evidence before a larger commitment, so when the evidence is unfavorable, use it. Maybe the problem was not valuable enough. Maybe adoption will not happen. Maybe a straightforward database application solves it better, which we have concluded more than once.

The failure is not ending an experiment. The failure is letting an experiment become a permanent system after its business case disappeared, because nobody wanted to say so.