How do you tell the difference?
Ask what the person doing the task is actually deciding. If they are transcribing, reformatting, looking something up, or applying a rule they could describe to you in a sentence, it is an integration or automation problem. If they are interpreting something ambiguous — what this email is asking for, which category this unlabelled line belongs in, whether these two records are the same entity — there is judgement in it, and that is where a model can help. The test is not how complicated the task feels. It is whether the rule can be written down.
The confusion is understandable, because both feel like “the computer should be doing this”. The difference only becomes expensive later: an integration you built five years ago still works, and a model you deployed five years ago is on its fourth version, with different behaviour and a different price.
What makes a task a poor fit, even when there is judgement in it?
Three things. If nobody could tell that the output was wrong, the error rate is unbounded and unmeasurable. If the cost of a wrong answer is regulatory or financial rather than merely annoying, the required accuracy is usually higher than the technology reliably delivers. And if the data the model would need does not exist yet, the real project is a data project wearing an AI hat. That is fine, but it should be budgeted as one.
What does “we’ll put a human in the loop” actually cost?
More than it sounds, and it is the assumption that quietly sinks the business case. A human reviewing every output is doing the original task plus a verification task, so the saving only appears if review is genuinely faster than doing the work. For short, structured outputs it often is not. Review also decays: people who check a system that is usually right stop checking carefully, which is precisely when the unusual error gets through.
| The request | What it usually is |
|---|---|
| “Use AI to pull data out of these reports” | Integration, if the reports are structured. Extraction, if they are not |
| “Use AI to forecast cash” | A data and process problem first; the model is the last 20% |
| “Use AI to answer routine emails” | Genuine fit — ambiguous input, checkable output, recoverable error |
| “Use AI to reconcile these accounts” | Rules for the 95%, judgement for the exceptions. Do not model the 95% |
So when is the answer yes?
When the input is genuinely unstructured or ambiguous, a person could check the output quickly, a wrong answer is recoverable, the data already exists, and the task happens often enough that the saving compounds. Those five together are less common than the current level of enthusiasm implies, and where they do hold, the case is usually obvious enough that nobody needs a strategy document to see it.