Enterprise AI platform: architecture, data, observability and costs

A first AI application can be launched within a few days. The second rarely reuses everything that has been built. The fifth reveals the duplicates: five authentication systems, three vector databases, prompts copied into the code, no inventory of sent data and invoices that cannot be linked to a usage.

It's not a model problem. It's a platform problem.

The role of an enterprise AI platform is to make common capabilities accessible without imposing a single technology. It accelerates the work of product teams, while giving security, data, finance and compliance managers a coherent control point. Its architecture must accept a reality: models change faster than business processes.

1. The figures that change the architecture decision

The market is growing at an unusual pace. According to Stanford's AI Index 2026, 88% of the organisations surveyed reported using AI in at least one function in 2025, and 70% used generative AI. However, agent use remained at a single-digit percentage. The gap between experimentation and operational autonomy recalls that a successful chatbot does not prove that a company knows how to operate an agent chain.

Industrial dependence is strong. More than 90% of the notable models in 2025 came from industry. The United States had 5,427 data centres, more than ten times the number reported in any other country according to the same report. The location of a supplier, the origin of its model and the actual compute location are therefore not deduced from a European logo or from a commercial interface.

Performance and prices are also moving. The AI Index 2025 measured a decrease of more than 280 times in the cost of inference for a performance level comparable to GPT‐3.5 between November 2022 and October 2024. MLCommons then observed, during MLPerf Inference v5.1, gains of up to 50% for the best systems compared to the release six months earlier. A platform too tightly coupled to one supplier turns improvements in the market into migration costs.

Quality is not progressing consistently. The Technical Performance chapter of the AI Index 2026 indicates that the best agents have increased from about 12% to 66% on OSWorld, a benchmark for computer use. It's a major leap. It is also a result that leaves about one in three cases unresolved in this benchmark. It would be imprudent to translate 66% on a benchmark into 66% of automated business processes.

Finally, incidents are increasing. Stanford had 362 AI-related incidents in 2025, compared to 233 in 2024. The architecture must therefore optimise something other than deployment speed: it reduces the blast radius when a component fails or is attacked.

2. A request's journey in eleven checks

The architecture becomes more legible when you follow a real request. Let's take an account manager who asks: "Compare this contract with our policy and prepare the points to negotiate.”

2.1. Identity and context

The identity gateway authenticates the user and retrieves their organisation, role, country and file. Identity is not only used to open the application. It determines what documents are accessible, what tools are allowed and the level of logging.

A demonstration account must never have the same connectors as a lawyer. An API key shared between multiple applications makes this separation almost impossible.

2.2. Use-case classification

A router identifies the use, sensitivity, language, complexity and possible presence of personal data. This step can combine rules and a small model. It shall decide whether the request is authorised, whether it requires confirmation and what budget may be consumed.

A request for internal reformulation is not risky. A contractual recommendation commits more. The path must not be the same.

2.3. Input protection

The system analyzes attachments, detects unexpected formats, removes obvious secrets and limits size. The recovered documents are treated as unreliable content. A phrase hidden in a PDF may attempt to divert instructions from the system.

The defence does not consist of "asking the model to be prudent." It separates instructions, data and permissions. Content does not get any rights by its mere presence in the context.

2.4. Access to data

The search layer applies permissions before retrieving content. It combines lexical and semantic search, metadata filters and reranking as needed. Each extract retains its source, version and effective date.

A vector base without rights management creates a horizontal leak: the user can receive a passage that they could not open in the original system. Permissions must follow the document up to the result.

2.5. Model selection

The router chooses a model according to four constraints: capacity, sensitivity, latency and cost. A small model can extract the clauses. A more powerful model can compare their effects. A model hosted in a dedicated environment may be required for the most sensitive data.

The selection is not final. The catalogue maintains a common interface and a register of capabilities. A new version passes first on phantom traffic, then gradually receives requests.

2.6. Context building

The system assembles the instructions, extracts, output diagram, useful history and tools. It removes the decorative context. More tokens do not automatically mean more quality; contradictory passages can disrupt the response and increase the cost.

Each prompt template has an ID and a version. A modification is a production setup, not an invisible retouch in a console.

2.7. Execute tools

If the task requires consulting a CRM, calculate an amount or create a task, it proposes a structured call. A policy layer checks the tool, parameters and authorization. High impact actions require human validation.

Reading is not writing. Preparing is not sending. The enforcement policy encodes these differences.

2.8. Output validation

The result is checked before display: valid schema, existing quotes, consistent amounts, no secret leakage, sufficient level of confidence. Some checks are deterministic. Others use a second model, without assuming that this automatic judge is infallible.

Where there is insufficient evidence, abstention is a useful exit. It's better than an elegant, invented answer.

2.9. Presentation and human action

The interface shows the sources, uncertainties and actions proposed. It facilitates correction. A "useful / useless" button without pattern produces little learning; a structured correction feeds a future evaluation dataset.

The account manager remains the decision-maker. If the system sends directly to the client, the level of requirement changes and control needs to be rethought.

2.10. Logging

The layer of traces records the necessary elements: version of the model, prompt, sources, tools, latency, cost, user's decision and possible error. Sensitive data are minimised, masked or separated. Journaling everything without retention policy creates a new risk.

2.11. Deferred evaluation

A sample enters a review queue. Errors are classified: retrieval, reasoning, tool, policy, interface or source data. This taxonomy turns incidents into precise work. Without it, the team tunes components blindly.

3. Reference architecture across eight layers

The previous route translates into eight independent but connected layers.

This separation does not mean eight products. It prevents an application from bundling everything into an impossible-to-audit block. A small organisation can deploy the base with a few services, provided that the logical boundaries are maintained.

4. Multi-model architecture: avoid two extremes

The first excess is dependent entirely on a supplier. It's simple at the start. The cost, limitations and policy of the supplier then become product constraints.

The second is to integrate ten models "to stay agnostic". The team then multiplies the tests, contracts, SDKs and behaviours to maintain. Total agnosticism can cost more than the dependency it seeks to avoid.

A reasonable strategy defines three ways: a primary model, a back-up solution, and a specialised model where economic evidence exists. Common interfaces cover basic functions. The specific capabilities of each supplier remain encapsulated, with an output test.

Routing is based on explicit rules. It may consider:

• the data class;
• the estimated complexity;
• the length of the context;
• maximum response time;
• the authorised cost;
• the availability of the supplier;
• the processing region;
• the score obtained on the use case.

The fallback must not silently degrade security. If the dedicated European model is unavailable, switching to a public API is not an acceptable continuity measure. Instead, the system can suspend processing and inform the user.

5. Observability: see the decision, not just the request

Conventional tools monitor error rate, latency and CPU. They remain necessary. The AI system adds semantic observability.

Each trace links demand to its sources, model, tools, outcome and final action. It answers five questions: what did the system receive, why did it choose this path, what evidence did it use, what did it produce, and what did the user do?

The indicators are divided into four groups.

Service. Availability, latency p50/p95/p99, errors, queues and fallback rates.

Quality. Accuracy per scenario, citation coverage, abstention rates, format violations, human corrections and regressions.

Risk. injection attempts, rejected tool calls, detected sensitive data, abnormal access and confirmed incidents.

Value. time saved, cycle delay, volume processed without rework, adoption and business result.

An overall average masks rare failures. We segment by language, client, document type, model version and criticality. A 2-point regression on the whole can hide a 20-point fall on German contracts.

6. SLO, error budget and deployment

A service level objective, or SLO, describes what the application must achieve. Example: 99.5% of authorised requests receive a response in less than fifteen seconds; 98% of citations point to an accessible passage; no financial action is executed without confirmation.

The error budget quantifies the permissible margin. If consumed, the team slows down the new features and focuses on reliability work. For semantic quality, the budget may cover critical errors observed in a controlled sample.

The versions follow a progressive path: offline tests, red team, phantom traffic, internal group, low percentage of production, then general rollout. Rollback includes model, prompt, search and rules. Returning only to the previous model is not enough if the documentary index has changed.

7. FinOps AI: a cost ledger

The invoice must be attributable to a product, team, customer and result. The general ledger records for each transaction: input and output tokens, search calls, storage, GPU time, external tools, retries and human review.

Three ratios are particularly useful.

Cost per task completed. It includes repeats and failures, not just successful call.

Cost per unit of value. For example, cost per qualified record or net time saved. This ratio makes it possible to compare the AI with a classical process improvement.

Quality marginal cost. How much does the increase from 90% to 94% switch on important cases cost? The best model can multiply the bill for a gain without business effect.

The levers are then targeted: cache for stable content, context summary, batch, smaller model, retry limit, more precise index or deletion of an automatic judgment step. The reduction must be tested on quality. A token saving that increases corrections is a false economy.

8. Security: limiting the range of action

Centralizing creates a checkpoint, but also a target. Least privilege applies to each connector. Secrets are short, rotating and attached to a service. Test environments do not access production data.

Agents require extra attention. NIST showed, in an evaluation published in 2025, that the success rate of abuse attacks increased from 57% with an attempt to 80% after 25 attempts in the study protocol. Blocking a malicious formulation is therefore not enough. Permissions, confirmations, ceilings and isolated environments must contain a possible success.

External content is unreliable. Outputs are unreliable too. Before being interpreted by a browser, base or shell, each output requires strict validation. A model never directly writes a free request in a critical system.

9. Sovereignty and residence: asking the right questions

The residence describes where data is stored or processed. Sovereignty describes who can decide, access, modify, interrupt or replace. Both overlap without being equivalent.

A useful mapping analysis: contracting company, hosting location, subcontractors, applicable law, origin of components, encryption keys, administration, telemetry and export capacity. It distinguishes the data at rest, in transit and present in the execution memory.

Sovereign hosting without internal skills can remain fragile. An open architecture without restoration exercise can remain captive. Control is demonstrated in concrete terms: to change the model, to cut a supplier, to restore an index and to revoke a connector.

10. Build or buy?

Buying an integrated base accelerates authentication, catalogue and observability. Building offers more control and adaptation. The right choice depends on the differentiator.

Generic capabilities are more readily purchased: model gateway, secret management, storage, logging and dashboards. The business logic is further constructed: taxonomy, decision rules, evaluations, user experience and specific connectors.

The contract must protect the assets. Prompts, tests, annotations, useful traces and configurations must be exportable. A solution that makes it easy to adopt but hard to exit transfers the risk to the future.

11. Deployment road map

Phase 1 — Inventory. Identify existing use cases, models, data, owners, contracts and costs. Eliminate shared keys and untracked prototypes.

Phase 2 — Standardize minimum. Set up identity, gateway, logging, prompt registry and data rules. Only one pilot case passes through the entire route.

Phase 3 — Measure. Build evaluation datasets, OLS and cost allocation. Compare two models on the same tasks.

Phase 4 — Industrialize. Add routing, fallback, progressive deployment, alerts and incident reviews. Open the base to several teams with templates.

Phase 5 — Exercise reversibility. Deliberately disconnect a supplier in a controlled environment. Restore data. Measure real migration time.

The base is not completed at any of these stages. It evolves as an internal product, with users, a roadmap, a manager and a level of service.

12. The day the supplier becomes unavailable

Disconnect the primary service. What's left? The user must know that. The team must know too.

The continuity scenario documents the detection, refusal of new actions, queue of pending requests, manual fallback, communication and restoration. The data already entrusted to the service are distinguished from calls that have never been completed. An incident of one hour and a final withdrawal do not demand the same plan, but both require an owner, a log, an understandable message for the user, a rule for partially executed actions, a reconciliation procedure after recovery and a delay beyond which the fallback mode itself becomes the new source of operational risk.

Exercise brings a truth rarely visible in a diagram: reversibility depends on people, contracts and formats as well as code. Time it. Fund it. Improve it. A plan that has never been played remains an elegant hypothesis. At the time of conclusion, the report must state precisely which functions have survived, what data have been recovered, which operations have requested intervention, what contractual commitments have proved useful, what skills were lacking and how much more time would be needed to transform this simple controlled exercise into a complete migration that would safeguard rights, history, evaluations and business continuity.

13. Frequently Asked Questions

13.1. Do you need a platform before the first AI project?

No. However, the pilot should not bypass identity, safety and measurement. The first case can build the minimum foundation; its components are then standardised when a second need confirms their usefulness.

13.2. Is a multi-model gateway enough?

No. It centralizes access and sometimes costs. It does not resolve documentary rights, evaluation, tool calls, or human control.

13.3. What do we need to log?

This makes it possible to diagnose, audit and improve, with a proportionate duration. Personal and secret data must be minimised or masked. An exploitable trace has an end-to-end identifier and component versions.

13.4. How can you prevent costs from spiralling?

Assign each expenditure, set budgets per task, limit retries and measure the cost of rework. Only then, optimise the model or context. Without value metrics, the drop in the bill can degrade usage.

13.5. Can we mix public cloud with private hosting?

Yes, if routing depends on the data class and if paths are tested. However, the hybrid architecture adds operations, evaluations and version management. It must meet a verifiable need.

14. What Logiks recommends

Design the platform around the path of a decision, not around the supplier of the moment. Centralise identity, policies, logging, evaluation and costs. Leave the models replaceable. Then limit autonomy to what the organisation can observe, interrupt and restore.

15. Main sources

• Stanford Institute for Human-Centered AI, AI Index Report 2026: https://hai.stanford.edu/ai-index/2026-ai-index-report
• Stanford HAI, Economy chapter: https://hai.stanford.edu/ai-index/2026-ai-index-report/economy
• Stanford HAI, Technical Performance Chapter: https://hai.stanford.edu/ai-index/2026-ai-index-report/technical-performance
• Stanford Institute for Human-Centered AI, AI Index Report 2025: https://hai.stanford.edu/ai-index/2025-ai-index-report
• MLCommons, MLPerf Inference v5.0 Results: https://mlcommons.org/2025/04/mlperf-inference-v5-0-results/
• MLCommons, MLPerf Inference v5.1 Results: https://mlcommons.org/2025/09/mlperf-inference-v5-1-results/
• NIST, Strengthening AI Agent Hijacking Evaluations: https://www.nist.gov/news-events/news/2025/01/technical-blog-strengthening-ai-agent-hijacking-evaluations
• OWASP, Top 10 for Agentic Applications: https://genai.owasp.org/2025/12/09/owasp-top-10-for-agentic-applications-the-benchmark-for-agentic-security-in-the-age-of-autonomous-ai/