Six months, a dozen client pilots, roughly half still running. Here is the pattern that emerged, for anyone about to start their own list.
What survived
The pilots that survived were the ones that made an existing task faster. The ones that died were trying to invent a new one. A tool that drafts a first-pass product description from a spec sheet: survived, because a person was already writing that description and the AI just moved faster than the blank page. A tool that “discovered” content ideas nobody had asked for: dead within a month, because nobody had a task it was replacing.
The maintenance nobody budgets for
Every AI feature needs someone to notice when the model’s output quietly gets worse — a provider changes a model version, a prompt that worked in March stops working in July. We now write that ownership into the scope explicitly, because “and someone will keep an eye on it” was never a real answer.
The question that kills bad ideas fastest
Before building anything, we ask what a human does today, by hand, that this would replace. If nobody can answer in one sentence, the feature is being built for the pitch deck, not for the work.
The pilots that survived were the ones that made an existing task faster. The ones that died were trying to invent a new one.