Opens in a new tab
Contact Us
Get Started

Enterprise RAG Architecture: A Production Reference Built on the Retrieval Contract

Your RAG pilot passed every demo, and then someone asked whether it still answers from the policy document legal pulled last month. An enterprise RAG architecture prevents that failure. It is a production system of three paths (indexing, online retrieval, and generation) bound by a retrieval contract that every chunk carries, covering identity, version, permissions, provenance, and freshness. The Hammerspace data platform for AI helps keep that contract true at the data layer.

Why Is Production RAG a Data Correctness Problem, Not a Model Problem?

Production RAG usually fails because the retrieved data is stale, over-permissioned, duplicated, or untraceable, not because the model is weak. A production RAG architecture fixes this by treating the corpus as a governed, continuously changing dataset instead of a one-time snapshot.

A pilot answers from a frozen export. Someone copies a folder of files into a bucket, chunks it once, embeds it once, and the demo works. Nothing changes, nobody loses access, and nobody asks where an answer came from.

Production is different. The corpus changes constantly, it carries access controls from several identity domains, and it lives across file systems, object stores, sites, and clouds. The failure modes that follow are data failures:

  • Stale answers: the index reflects a version of a file that no longer exists.
  • Ghost answers: deleted content remains retrievable because the deletion never reached the index.
  • Permission leakage: a user retrieves a chunk from a file they lost access to at the source.
  • Duplicate and conflicting chunks: two staging copies of the same file produce two competing sets of vectors.
  • Untraceable answers: nobody can say which file, which version, or which pipeline run produced a cited passage.

A larger model does not fix any of these. Neither does a better prompt. What fixes them is making each chunk carry enough metadata to prove it is current, authorized, and traceable, and keeping that metadata true as the source changes.

That metadata set is the retrieval contract, and every later section refers back to it. The architecture defines where it is produced, maintenance keeps it current, security enforces it, and evaluation uses it to make results reproducible.

What Is a Retrieval Contract, and What Must Every Chunk Carry?

A retrieval contract is the fixed set of fields every chunk must carry so the system can prove where the chunk came from, which version it represents, who may see it, and whether it should still exist. It is the interface between the storage and data layer on one side and the vector index on the other.

The table below defines the contract field by field, with what each field holds and the failure mode it prevents.

FieldWhat it holdsWhy it exists and what it prevents
Deterministic chunk IDA stable identifier derived from source file identity, chunking policy version, and chunk position or content hashReprocessing a file replaces its chunks instead of duplicating them. Prevents duplicate and orphaned chunks.
Source versionThe file version, revision marker, or content hash at extraction timeThe index detects when a chunk no longer matches its source. Prevents stale answers.
Canonical namespace pathThe single authoritative path of the source file in the global namespace, never a staging-copy pathEvery citation and every re-read resolves to the same governed object. Prevents citations to ungoverned copies.
TimestampsSource modification time, extraction time, embedding time, and index write timeFreshness lag becomes measurable and events can be ordered. Prevents silent drift.
Model and policy versionsEmbedding model identifier and version, parser version, and chunking policy versionVectors from different models are not comparable, and targeted re-embedding requires knowing which chunks are affected. Prevents mixed embedding spaces.
ACLsThe principals and groups allowed to read the source, resolved at extraction, plus an ACL version or timestampThe retrieval layer filters on it at query time. Prevents permission leakage.
TagsClassification, sensitivity, business domain, retention class, and custom labelsDrives metadata filtering, retention, and placement. Prevents out-of-scope retrieval.
TombstoneA deletion marker with deletion time and reasonDeletions propagate reliably to replicas and caches. Prevents ghost answers.
ProvenanceLineage from chunk to source file version, extraction run, and transformations such as OCR or table parsingAnswers are auditable and citable. Prevents untraceable output.
Freshness targetThe maximum acceptable indexing lag for this class of source, inherited from policyStaleness becomes a measurable objective with a priority attached. Prevents treating all data as equally urgent.

Define the contract as a versioned schema, and reject any chunk at index write time that is missing a required field. That single rule catches pipeline drift before it reaches users.

Ownership matters here. The data layer is responsible for making the canonical path, version, timestamps, ACLs, and tags true at the source, while the index is responsible for storing those values faithfully and letting the retrieval layer act on them. When either side breaks its half, the contract tells you exactly which field went wrong.

What Does an Enterprise RAG Architecture Look Like Across Indexing, Retrieval, and Generation?

An enterprise RAG architecture has three distinct paths: an indexing path that turns source files into contract-bearing chunks, an online retrieval path that finds authorized chunks for a query, and a generation path that composes a grounded answer. Each path has its own access pattern and its own primary constraint.

Treating them as one pipeline is a common mistake. They scale differently, fail differently, and are usually owned by different people.

PathStagesStorage and access patternPrimary constraint
IndexingChange identification, extraction and parsing, chunking, embedding, and index upsert or tombstoneLarge, parallel reads of source files across silos and sites, bursty and throughput-bound, with writes going to the indexReading the correct version from the canonical location, at scale, within freshness targets
Online retrievalResolving the caller’s identity and group memberships, query rewriting, dense and sparse retrieval, fusion, metadata and ACL filtering, and rerankingLow-latency random reads against the vector index or hybrid search engine, not against source storageLatency and authorization correctness
GenerationContext assembly, prompt construction, LLM inference, citation, and output validationModel weights and KV cache on GPU-local memory and storage, plus occasional source fetches by canonical path for citation displayGPU utilization and groundedness

State this plainly in your design documents: a vector index or hybrid search engine is required, and it is maintained separately from source storage. Hammerspace is not a vector database or search engine and does not replace one. Embeddings and chunk metadata live in the index you choose.

Where the data layer matters most is the indexing path and the source-resolution step of generation. Indexing needs parallel reads across many systems, which is the problem Parallel NFS was built for. pNFS, introduced in NFSv4.1 (now specified in RFC 8881), separates metadata from data so clients read directly from storage in parallel instead of funneling through a single server.

How Do Staging Copies Break RAG, and How Do You Read Sources in Place?

Staging copies break the retrieval contract because each copy has its own path, its own timestamps, and its own permissions, so the index ends up pointing at data nobody governs. Reading sources in place through a standards-based global namespace keeps one canonical path, one version history, and one permission truth.

How the copy chain corrupts the contract

In a typical RAG pipeline, unstructured data is exported from file shares into an object bucket, copied into a scratch area for parsing, written as extracted text to another bucket, and only then embedded. Each hop damages a specific contract field:

  • Canonical namespace path: the index records the staging path, so citations point to a copy instead of the governed source.
  • Source version: the copy reflects the file at copy time. When the original changes, nothing tells the index.
  • ACLs: bucket policies do not carry NFSv4 or SMB ACLs faithfully. Permissions get flattened or reconstructed by hand.
  • Tombstones: deleting the original leaves the copy, and the vectors derived from it, in place.
  • Provenance: every extra hop adds lineage you have to track, and many pipelines stop tracking after the first one.
  • Governance: in regulated environments, every copy is another dataset to classify, secure, and audit.

Imagine a team that fixes one of these problems by writing a reconciliation job between the bucket and the file share. They now own a second synchronization system whose correctness nobody tests. The copy problem does not go away; it just gets a scheduler.

Reading in place through the Unified Data Plane

The alternative is to stop copying source files for ingestion. The Hammerspace Unified Data Plane provides standards-based read and write access to data across any storage, site, or cloud. Global Data Services assimilates metadata from existing file and object storage in place, so files appear in a single global namespace for AI and enterprise data without first being migrated.

Ingestion workers read through native Linux NFSv4.2 and pNFS, SMB, or S3. There is no proprietary client to install on embedding nodes and no vendor-specific ingestion connector to maintain. A standard Linux host mounts the namespace and reads the file at its canonical path, which is the path the contract records.

This is an open-standards argument, not only a convenience. Proprietary storage stacks tie your ingestion code to their client, their connector, and their upgrade cycle. Standard protocols keep the pipeline portable and avoid vendor lock-in. For more on how open standards protect enterprise infrastructure investments, see how open standards and abstraction layers protect enterprise infrastructure.

Hammerspace’s engineering heritage is relevant here. Hammerspace’s team includes Trond Myklebust, Tom Haynes, Brian Pawlowski, and CEO David Flynn, the engineers behind Linux NFS itself. Trond Myklebust maintains the Linux NFS client, and Tom Haynes is the editor of the NFSv4.2 specification published as RFC 7862. Your ingestion workers already have the NFS client in their Linux kernel, built in part by these engineers.

What reading in place does not change

Reading in place removes unmanaged source copies. It does not remove the index. Chunks, embeddings, and contract metadata are still written to a vector index or hybrid search engine, and query-time authorization still happens in the retrieval layer. The data layer’s job is to make sure the values that index receives are true.

To learn how the Unified Data Plane fits your own indexing path, contact Hammerspace or call +1 (650) 777-8728.

How Do You Maintain a RAG Index With Deterministic IDs, Versions, Tombstones, and Freshness SLOs?

RAG index maintenance should be incremental: identify changed files from source metadata, recompute only the affected chunks using deterministic IDs, record new versions, apply tombstones for deletions, and schedule the work by freshness target. Reserve full-corpus reprocessing for embedding model or chunking policy changes.

Incremental RAG ingestion depends entirely on the contract. Without deterministic IDs, every reprocess creates duplicates. Without source versions, you cannot tell what changed, and without tombstones, deletions silently fail.

The maintenance lifecycle

The table below lists each event, how it is detected, what the index does, and whether re-embedding is required.

EventDetectionIndex actionRe-embed?
New fileMetadata-aware change identification shows a new path in the namespaceExtract, chunk, embed, and insert with a complete contractYes, first embedding
Content modifiedThe current version or content hash differs from the contract’s source versionRe-chunk, upsert chunks under the same deterministic IDs, and tombstone chunk IDs that no longer existYes, but only for changed chunks when IDs include a content hash
Permission changeSource ACL metadata differs from the stored ACL versionUpdate ACL fields on every chunk from that fileNo
Tag or classification changeTag metadata differs from stored tagsUpdate tags, or tombstone the file’s chunks if the new classification excludes it from the corpusNo
Rename or move by a userThe canonical path changes while file identity persistsUpdate the path and provenance fieldsNo
Placement change by orchestrationNone needed, because the namespace path does not change when data moves between tiers or sitesNoneNo
DeletionThe file is absent from the namespaceWrite a tombstone, purge vectors once replicas confirm, and invalidate cached answers that cited the fileNo
Embedding model upgradeThe model version in policy changesBuild a shadow index, evaluate both, then cut overYes, full corpus
Parser or chunking policy changeThe policy version changesRe-chunk the affected scope under new deterministic IDs and tombstone the old onesYes, for the affected scope

Notice that five of these events require no embedding work at all. Pipelines that re-embed on every metadata change waste GPU time, and pipelines that ignore metadata changes leak data. The contract lets you do exactly the minimum.

Prioritizing freshness with file-granular SLOs

Not every source deserves the same freshness target. A directory of active policies matters more than a decade-old project archive. Set targets per source class based on business risk, and store them in the contract so lag is measured against the right objective.

Global Data Services supports this directly. File-granular Service-Level Objectives and a single set of policies govern placement, protection, and handling, and actionable metadata combines storage, usage, and POSIX metadata with custom tags. Your indexing scheduler reads those tags and objectives to order its work queue, so the highest-priority changes are processed first. For a deeper look at how metadata drives this, see how the impact of data orchestration for AI shapes pipeline priorities.

Data Orchestration then executes policy-driven actions in the background, even on live data. Changed files in a high-priority class are pre-staged near the embedding workers before the job runs, which is the practical core of keeping RAG data fresh.

How Do You Secure RAG With Permissions, Access Control, and Provenance?

Secure RAG by applying controls from ingestion through downstream agents, with RAG access control enforced at query time in the retrieval layer using the ACLs carried in the contract. This is permission-aware retrieval: the data layer supplies accurate source permissions, canonical locations, and provenance, and the retrieval layer makes the authorization decision.

The OWASP RAG Security Cheat Sheet, updated in September 2026 and published in the OWASP Cheat Sheet Series, is current security guidance that spans every stage of a RAG system, from ingestion through downstream agent integration. Use it alongside the stage-by-stage controls below. For governance framing in regulated environments, including federal agencies, the NIST AI Risk Management Framework and its Generative AI Profile (NIST AI 600-1) provide a common vocabulary for risk management, provenance, and accountability.

Controls by stage, with owners

  • Ingestion: verify that sources are authentic and read from canonical paths, capture ACLs and provenance, and screen content for embedded instructions. The data layer supplies metadata; the pipeline owns screening.
  • Embedding generation: pin model versions and isolate the embedding service. Treat embeddings as derived data with the same classification as their source, because vectors can leak source content.
  • Vector storage: encrypt the index, isolate tenants, and require ACL and tag fields on every write. The index owner is responsible.
  • Retrieval: resolve the caller’s identity and groups, then filter on contract ACLs before results leave the index. The retrieval layer owns query-time enforcement.
  • Response generation: separate system instructions from retrieved content and treat retrieved text as untrusted input.
  • Output validation: verify that each citation resolves to a live canonical path and current version, and apply data loss prevention checks.
  • Downstream agent integration: scope agent tools to the end user’s permissions, never a broad service identity.

To restate the boundary: Hammerspace does not perform query-time authorization and does not replace the vector index. Instead, it provides consistent source metadata, visibility into permissions, canonical locations, and orchestration, so the ACLs your retrieval layer enforces are correct. No architecture eliminates leakage risk entirely, but this one makes permission errors visible and testable.

Permission and deletion tests

Build these as automated tests that run on every index release:

  1. Permission test: for a fixed set of test principals with known access, run a standard query suite and assert that no returned chunk has an ACL excluding the caller. Then revoke a principal’s source access and assert the chunks disappear from their results within the source’s freshness target.
  2. Deletion test: plant canary documents containing unique tokens, confirm they are retrievable, delete them at the source, and assert that queries for those tokens return nothing within the freshness target. Check replicas, caches, and stored answer logs as well.

RAG data provenance ties this together for audits. In a regulated environment, tracing an answer to a chunk, a file version, an extraction run, and a model version is what gives governance and audit teams the evidence they need to approve a system. For more on governance at scale, see unstructured data governance at petabyte scale.

How Do You Improve and Evaluate Retrieval With Hybrid Search, Reranking, and Separate Metrics?

Dense vectors alone miss exact identifiers, acronyms, and rare terms that are common in enterprise content, so production systems combine dense and sparse retrieval, fuse the results, rerank the candidates, and filter on contract metadata. Evaluate retrieval and generation separately so you know which layer failed.

Why dense-only retrieval underperforms

Enterprise corpora are full of part numbers, contract IDs, ticket numbers, internal product codenames, and regulatory citations. Embedding models compress meaning, and compression blurs exact tokens. A query for a specific identifier often returns semantically similar but wrong documents.

Sparse retrieval, such as BM25, matches exact terms well and handles vocabulary the embedding model never learned. Dense retrieval handles paraphrase and intent. You need both.

A production retrieval sequence

  1. Resolve the caller’s identity and build ACL and tag filters from the contract fields.
  2. Run dense and sparse retrieval in parallel with those filters applied inside the index.
  3. Fuse the ranked lists, commonly with reciprocal rank fusion, which combines rankings without needing comparable scores.
  4. Rerank the fused candidates with a cross-encoder that scores each query and passage pair directly.
  5. Drop candidates whose source version is superseded or whose tombstone is set, as a final guard.

RAG hybrid search and reranking live entirely in the retrieval stack. The data layer’s contribution is upstream: accurate tags, ACLs, and versions make the filters in steps one and five trustworthy.

Two evaluation layers

Retrieval evaluation asks whether the right chunks came back. Build a golden query set labeled with relevant deterministic chunk IDs, then track recall at k, mean reciprocal rank, and nDCG. Because the IDs are deterministic, labels survive re-indexing, which is how you compare a shadow index against production during a model upgrade.

Generation evaluation asks whether the answer used those chunks faithfully. Measure groundedness against the retrieved context, citation validity (every cited canonical path and version exists and is live), answer relevance, and correct abstention when no authorized context exists.

For RAG evaluation and observability, log each query’s returned chunk IDs, source versions, scores, applied filters, and model versions. With contract fields in every trace, a bad answer becomes a reproducible incident instead of an anecdote.

How Do You Place RAG Data Near Embedding and Inference GPUs?

Use policy-driven Data Orchestration to move the working set, meaning changed files awaiting embedding and frequently retrieved corpora, to wherever GPUs run, on-premises or in the cloud, while the canonical source and its contract stay intact. Placement becomes a policy decision instead of a manual copy.

Embedding jobs are bursty. When a large set of files changes, the embedding cluster needs to read it fast, and it often sits in a different site or cloud than the source. The usual answer is another copy, which reintroduces every contract problem described earlier.

Orchestration handles it differently. Data Orchestration executes tiering, migration, and collaboration actions in the background across sites and clouds, under the same global namespace path. Because the file’s identity, path, and metadata do not change, the contract does not change either. Placement is a managed instance of the same governed file, not a new dataset.

For the GPUs themselves, Hammerspace Tier 0 turns the local NVMe already inside GPU servers into a shared, high-performance storage tier within the global namespace. Pre-staged corpora and embedding inputs sit next to the compute that needs them, which keeps GPUs fed instead of waiting on remote storage. The same approach supports RAG inference that fetches source passages for citation display.

The benefits are practical: embedding jobs bounded by compute rather than by data movement, fewer large cross-site transfers, and no unmanaged replicas for security teams to chase. Canonical sources remain where governance expects them.

How Do You Operate RAG in Production? Monitoring, Ownership, and a Readiness Checklist

Operate production RAG as three continuously managed workflows: source synchronization, index maintenance, and evaluation. Each has its own signals, owners, and failure alerts, and all three are measured against the retrieval contract.

Monitoring model

  • Data layer signals: change backlog per freshness class, lag against each freshness target, ACL synchronization lag, and files unreadable at their canonical path.
  • Index signals: chunk count reconciled against source file inventory, orphaned chunks without a live source, tombstone backlog, and mixed embedding model versions.
  • Retrieval signals: golden-set recall trends, empty-result rate, filter-drop rate, and permission test pass rate.
  • Generation signals: groundedness, citation validity, abstention rate, and deletion test pass rate.

Ownership boundaries

  • Data and storage teams own canonical sources, source metadata, ACL truth, placement policy, and freshness classes.
  • AI platform and MLOps teams own parsing, chunking, embeddings, the vector index, retrieval, query-time authorization, and evaluation.
  • Both jointly own the contract schema and its versioning.

Production-readiness checklist

  • Every chunk carries a deterministic ID, source version, and canonical namespace path.
  • The index rejects writes missing required contract fields.
  • Ingestion reads sources in place, with no unmanaged staging copies.
  • ACLs and tags sync on change without triggering re-embedding.
  • Deletions produce tombstones that reach replicas and caches.
  • Freshness targets are set per source class and monitored.
  • Model and policy versions are recorded, and upgrades use a shadow index.
  • Retrieval uses dense and sparse search, fusion, reranking, and contract filters.
  • Permission and deletion tests run on every index release.
  • Retrieval and generation are evaluated separately against a golden set.
  • Every answer traces back to a file version and pipeline run.

A durable enterprise RAG architecture starts with data you can prove is current, authorized, and traceable. To learn how the Unified Data Plane, Global Data Services, Data Orchestration, and Tier 0 support the retrieval contract across your storage, sites, and clouds, contact Hammerspace or call +1 (650) 777-8728.

Frequently Asked Questions

What infrastructure does a production enterprise RAG system need?

A production system needs a data layer that exposes source files at canonical paths and an ingestion pipeline for parsing, chunking, and embedding. It also needs a vector index or hybrid search engine, a retrieval service that handles identity resolution, filtering, and reranking, and GPU capacity for embedding and LLM inference. An evaluation and observability stack should log chunk IDs, source versions, and applied filters for every query. A versioned retrieval contract schema ties these components together, and a data platform like Hammerspace supports the data layer and placement while you run your own vector index and retrieval stack.

Does RAG require copying all your data into a vector database?

No. The vector database holds chunks, embeddings, and contract metadata, not full duplicates of your source files. Sources can stay where they already live and be read in place through a global namespace over standard protocols such as NFS, SMB, or S3. This avoids staging copies, which drift from the original in version, permissions, and deletion state. Citations then point back to the governed source file instead of an untracked duplicate.

How do you keep a RAG index in sync when source files change or are deleted?

Use metadata-driven change detection to identify new, modified, and deleted files, then process only what changed. Deterministic chunk IDs let the pipeline overwrite a file’s existing chunks instead of piling up duplicates, and a stored source version or content hash reveals when a chunk no longer matches its source. Deletions should produce tombstones that reach replicas and caches before vectors are purged. Schedule this work by freshness target so high-risk sources are refreshed ahead of low-priority archives.

How do you enforce document permissions in RAG so users only retrieve what they are allowed to see?

Capture each source file’s ACLs during extraction, store them on every chunk, and filter on them inside the index at query time after resolving the caller’s identity and group memberships. When permissions change at the source, update the chunk ACL fields directly without re-embedding. The data layer is responsible for accurate source permissions, while the retrieval layer makes the actual authorization decision. Confirm the setup with automated tests that revoke a test user’s access and check that the affected chunks disappear from their results within the freshness target.

What happens to a RAG index when you upgrade the embedding model?

Vectors produced by different embedding models are not comparable, so an upgrade requires re-embedding the full corpus rather than patching part of it. The safe approach is to build a shadow index with the new model, run the same golden query set against both indexes, and cut over only after the new one meets your quality bar. Recording the model version on every chunk lets you detect any leftover mixed embedding spaces. Deterministic chunk IDs keep relevance labels valid across both indexes, which makes the side-by-side comparison reliable.

How should you evaluate a RAG system in production?

Evaluate retrieval and generation separately so you can pinpoint which layer caused a bad answer. For retrieval, label a golden query set with relevant chunk IDs and track metrics such as recall at k, mean reciprocal rank, and nDCG. For generation, measure groundedness, citation validity, answer relevance, and correct abstention when no authorized context exists. Add permission and deletion tests to every index release, and log chunk IDs, versions, filters, and model versions per query so incidents can be reproduced.

Can RAG work across data stored on-premises and in multiple clouds?

Yes, as long as every source appears under one namespace with consistent paths, versions, and permissions regardless of where it physically sits. Hammerspace assimilates metadata from existing file and object storage in place, so on-premises and cloud data can be indexed without a migration project first. Data Orchestration can move working sets closer to embedding or inference GPUs in another site or cloud without changing the file’s path or metadata. This keeps the retrieval contract intact even as data placement changes.

Data Orchestration For Dummies

  • Unlock and monetize your data
  • Achieve a unified global data platform
  • Liberate from data silos
Free Download

Share

Make AI Anywhere, A Reality!

See how Hammerspace can unify all your data, accelerate your AI workloads, and deliver results faster.
Get Started

Related Blog Posts