The demo is the easy part
Most organizations have already seen a convincing AI demonstration. Someone connected a model to a slice of company data, asked it a few questions, and the answers were good enough to make the room lean forward. The difficulty is rarely getting to that moment. The difficulty is everything that has to be true before the same capability can be trusted with a real customer, a real invoice, or a real clinical record. A demonstration is judged on whether it impresses. A production system is judged on what happens on its worst day, with its most unusual input, in front of the person least able to tolerate a mistake. Those are different problems, and they are not separated by a few weeks of tidying up.
Why capable pilots stall on the way to production
Pilots usually stall for reasons that have nothing to do with model quality. Nobody can say who owns the output when it is wrong. The data the prototype used was a hand-assembled extract that does not exist as a maintained feed. Access controls that were irrelevant in a sandbox become the central design question once real permissions apply. There is no agreed definition of what a good answer looks like, so there is no way to know whether a change has improved or degraded the system. Each of these is an operating problem rather than a technical one, and none of them are solved by a better prompt or a larger model. They are solved by deciding, early and explicitly, how the work will run once it is real.
Work backwards from the decision, not forward from the model
The most reliable way to scope AI work is to start at the end. Identify the decision or action the system will influence, then trace backwards through everything that decision depends on. Who currently makes it? What do they look at? What do they do when the information is incomplete or contradictory? What happens downstream if they get it wrong, and who finds out? That trace produces a far more useful specification than a list of desired features, because it surfaces the exception paths, the approval steps, and the accountability that a feature list quietly omits. It also tends to reveal that the highest-value opportunity is not the flashiest one. Very often it is a routine, high-volume decision that no one has thought about in years.
The people building it need to be close to the work
Requirements written by one group, interpreted by a second, and built by a third lose the operational detail that determines whether a system is usable. In AI work that loss is unusually costly, because the useful information sits in the exceptions: the case that always needs a supervisor, the field that is technically optional but never actually blank, the customer type that breaks the normal flow. These details rarely reach a written specification. They surface when an engineer sits with the people doing the work and watches what actually happens. That proximity is not a cultural preference. It is the mechanism by which the system ends up matching reality rather than matching the document that described it.
Evaluation is what separates a feature from a liability
Traditional software is largely deterministic: given the same input, it behaves the same way, and a test suite can pin that behavior down. Systems built on language models are not. The same question can produce different answers, and a change intended to fix one case can quietly degrade twenty others. The discipline that makes this manageable is evaluation: a maintained set of representative cases with agreed correct outcomes, run automatically whenever anything changes. It is the closest equivalent to a regression suite, and it plays the same role. Without it, a team is guessing about quality and will eventually discover a regression through a customer complaint rather than through a build. Evaluation sets are unglamorous to assemble and they are the difference between a capability you can improve and one you can only hope about.
Governance is infrastructure, not paperwork
Governance acquires a bad reputation when it arrives late, as a review gate applied to something already built. Treated as infrastructure, it looks different. Access control determines what the system can retrieve, which shapes the architecture. Audit logging determines what can be reconstructed afterwards, which shapes the data model. Human review determines where the system must stop and ask, which shapes the interface. Retention rules determine what may be stored and for how long. Decided at the start, these are design inputs that cost little. Retrofitted, they are expensive rewrites, and they tend to arrive at the least convenient moment, usually when a customer, auditor, or regulator asks a question that the system cannot answer about its own behavior.
What actually changes in how work is organized
An AI-native operating model changes the shape of delivery more than the job titles. Teams are organized around a workflow and its outcome rather than a technology component, so the same group owns the interface, the data path, the evaluation set, and the escalation route. Progress is demonstrated against real cases rather than described in status updates. Quality is a visible measurement rather than an assertion. Ownership persists past launch, because a system whose behavior depends on data and models needs someone accountable for how it behaves in month six, not only on release day. None of this requires reorganizing the company. It requires one team, working end to end, with the authority to change the workflow and not only the software.
Choosing the first workflow
The first workflow should be chosen for how much it will teach, not for how impressive it sounds. Four questions filter the candidates quickly. Does it happen often enough that improvement compounds? Is the current outcome measurable, so that a change can be evaluated honestly? Is the blast radius of a mistake understood and containable? Is there a named person who genuinely wants it fixed? A workflow that clears all four will produce a usable result and a great deal of institutional knowledge about data quality, permissions, and exception handling. A workflow chosen because it demonstrates well tends to produce a second demonstration.
Measuring what matters to the business
Model-level metrics are necessary for the team building the system and largely meaningless to the people funding it. The measures that carry an executive conversation are operational: how long the work takes now compared with before, how much of the volume completes without human intervention, how often the system escalates, how often it is overridden, and what the cost per completed unit of work is. Override rate in particular is worth watching closely, because it is the clearest signal of whether the people using the system actually trust it. A system with excellent internal scores and a high override rate is not working, whatever the dashboard says.
A realistic place to begin
The practical starting point is smaller than most roadmaps assume. Choose one workflow that meets the four tests. Map the decision, the systems it touches, and the exceptions. Agree what a good outcome looks like and write down enough real cases to evaluate it. Decide the data boundaries, the access model, and the point where a person must remain accountable. Then build the smallest version that runs end to end on real data, and improve it against the evaluation set. This produces something usable within a sensible horizon and, more importantly, produces the operating knowledge that makes the second and third workflows substantially faster.