Most failed AI features share one trait: they started from the technology instead of the task. A better starting point is a short list of questions.
Start with the task, not the model
Pick one task a person does repeatedly. Write down what goes in, what comes out and how you would know the output is right. If you cannot define “right”, AI will not fix that. It will hide it.
Three tests for a good candidate
- The work is frequent. A one-off task rarely justifies building anything.
- Errors are recoverable. If a wrong answer is cheap to catch and correct, AI can assist safely. If not, keep a person in the loop.
- You have examples. Real inputs and known good outputs let you measure quality instead of guessing.
Prototype against your own data
A demo on clean sample text proves little. Run the prototype on a few hundred real, messy examples and score the results honestly. Decide in advance what score is good enough to continue.
Design the failure path first
Decide what happens when the system is unsure or unavailable. A review queue, a clear fallback and visible sources do more for trust than a slightly better model.
Keep it measurable after launch
Log inputs, outputs and corrections. They tell you where quality slips and where the next improvement is.