AI demos are easy. AI in production is hard. The difference usually shows up six months after launch, when the model has decayed, cost has run away, user experience has degraded and accountability is unclear.
Production-grade AI requires that every model, prompt and data-product version is identifiable. When a production incident happens, you must be able to say: which model was running, at what version, with which prompt configuration and on which data. Most companies still operate on a "latest GPT version" philosophy, which means model behaviour changes with a provider update without anyone noticing.
A production model needs monitoring on three levels: technical (latency, errors, throughput), model-specific (accuracy, drift, bias indicators) and business (impact in euros, user experience, NPS effect). Classical APM tools cover level 1. Levels 2 and 3 require AI-specific tools or an LLMOps platform that understands model behaviour.
If a production model starts behaving badly, how is it rolled back to the previous version? Is there a controlled percentage of traffic on which a new version is tested before full rollout? In most companies the answer is "we do not know" or "manually, it takes hours".
The new LLMOps role is prompt management. A prompt is not a "file" or a "string" — it is a versioned artifact with tests, access rights and a change history. A prompt change in a use case is as big an event as a code change. A good platform offers a prompt registry, automatic tests (regression, tonality checks, hallucination checks) and A/B testing between prompt versions.
LLM costs run away easily. A mid-sized use case can cost 5,000 or 50,000 euros a month depending on how it is invoked, model choice and context window. Without per-agent and per-use-case cost limits the company never sees the real distribution. A good LLMOps platform exposes cost in real time and allows ceilings to be set before the bill explodes.