In enterprise LLM use cases three architectural options recur: retrieval-augmented generation (RAG), fine-tuning and autonomous agents. Most bad decisions arise from trying to choose between these for technology reasons rather than based on the requirements of the use case.
RAG fits use cases where information changes continuously and has to be current: product documentation, customer guidance, contracts, internal knowledge bases. The model is not trained on the data — it retrieves it at runtime. The advantages are obvious: low marginal cost per update, good traceability (the source can be cited), no need to own a model. Weaknesses come down to retrieval quality: if retrieval is bad, generation is bad.
Fine-tuning fits use cases where the model needs deep specialization: company-specific writing style, rare domain terms, very specific classification tasks or an output structure that prompting cannot reliably produce. The weaknesses are operational: it requires training data (often hundreds or thousands of examples), a retraining cycle, monitoring and governance decisions on what data may be used for training.
Agents are the right choice when a task requires multiple sequential decisions, external tool calls and adaptation to interim results. Examples: multi-step customer-request handling that reads order status, updates it and logs an event; or a research agent that iterates queries. Agents are also the most dangerous choice: a poorly designed agent can take irreversible external actions (send email, update systems, approve payments). A good agent platform requires guardrail configuration, HITL routing and an audit trail for every decision.
Put all four options side by side against the same criteria. Always start with the lightest one: a well-designed prompt solves a surprising share of use cases without any new infrastructure. Criterion | Prompting | RAG | Fine-tuning | Agent. Time to production | Days | 2–6 weeks | 4–12 weeks | 6–16 weeks. Data requirement | None | Documents and an index | Hundreds–thousands of examples | Tools and APIs. Fresh knowledge at runtime | No | Yes | No | Yes, via retrieval. Source traceability | Weak | Strong | Weak | Depends on design. Control of tone and structure | Moderate | Moderate | Strong | Inherited from base model. Multi-step reasoning and action | No | No | No | Yes. Maintenance load | Low | Medium (retrieval quality) | High (retraining cycle) | Highest (oversight and limits). Governance requirement | Prompt versioning | Access rights and sources | Training-data rights | Guardrails, HITL, audit trail. Typical use case | Summaries, drafts | Support centre, contract search | Classification, house tone | End-to-end process execution.
Total cost is not driven by model calls but by maintenance: quality control of the retrieval index for RAG, the retraining cycle for fine-tuning, and operational oversight for agents. Evaluate every option over three years of upkeep, not over a first-month pilot. The risk profile moves the same way: the more autonomy you grant, the more you need guardrails, human approval at critical steps and an auditable trail. That is a design decision, not a feature bolted on afterwards.