In enterprise LLM use cases three architectural options recur: retrieval-augmented generation (RAG), fine-tuning and autonomous agents. Most bad decisions arise from trying to choose between these for technology reasons rather than based on the requirements of the use case.
RAG fits use cases where information changes continuously and has to be current: product documentation, customer guidance, contracts, internal knowledge bases. The model is not trained on the data — it retrieves it at runtime. The advantages are obvious: low marginal cost per update, good traceability (the source can be cited), no need to own a model. Weaknesses come down to retrieval quality: if retrieval is bad, generation is bad.
Fine-tuning fits use cases where the model needs deep specialization: company-specific writing style, rare domain terms, very specific classification tasks or an output structure that prompting cannot reliably produce. The weaknesses are operational: it requires training data (often hundreds or thousands of examples), a retraining cycle, monitoring and governance decisions on what data may be used for training.
Agents are the right choice when a task requires multiple sequential decisions, external tool calls and adaptation to interim results. Examples: multi-step customer-request handling that reads order status, updates it and logs an event; or a research agent that iterates queries. Agents are also the most dangerous choice: a poorly designed agent can take irreversible external actions (send email, update systems, approve payments). A good agent platform requires guardrail configuration, HITL routing and an audit trail for every decision.
An enterprise-grade solution is often a hybrid. Example: a customer-service copilot uses RAG to retrieve from the knowledge base, a fine-tuned model to preserve company tone in responses and an agent runtime to execute actions in back-end systems. This is why LLM architecture cannot be chosen with a single decision. A use-case-specific analysis has to be done for each.
AI-Koutsi uses a four-part decision model per use case: (1) does the solution need fresh data at runtime? (RAG), (2) does it need specific style or structure? (fine-tuning), (3) does it need to take sequential decisions? (agent), (4) does it take irreversible external actions? (hard guardrail required). These four questions eliminate 95% of wrong architectural choices.