The AI features worth building — and the ones that aren't

Every roadmap review we sit in on now has at least one "add AI to this" line item. Some of them are genuinely good ideas. Most aren't — not because AI doesn't work, but because the workflow underneath it doesn't need it.

The test we actually use

Before proposing an AI feature, we ask one question: is the bottleneck a judgment call, or a lookup?

• If it's a lookup — finding a record, applying a fixed rule, formatting data — you don't need a model. You need better indexing, a form, or a plain automation. Reaching for an LLM here adds latency, cost, and a new failure mode for no real benefit.
• If it's a judgment call — triaging ambiguous support tickets, drafting a first-pass response, summarizing something genuinely unstructured — that's where a model earns its keep, because the alternative is a person doing that judgment call manually, every single time.

Ship it with a way to know it's working

The AI features that actually stick around share one habit: someone can answer "is this still doing a good job?" without guessing.

That usually means:

• A sample of outputs get reviewed on a schedule, not just at launch.
• There's a clear escalation path for the cases the model gets wrong.
• Someone owns the prompt/model choice the same way someone owns a piece of code — it gets updated when it stops working, not left to quietly degrade.

Skip that, and even a genuinely good AI feature turns into the thing nobody trusts six months later — which is a worse outcome than not building it at all.