How do you build a secure RAG system: document permissions, GDPR, and leak prevention?

Quick answer

A secure RAG system never shows a user more than that user could see without AI. In practice that means permissions from source systems carried into the index and checked inside every query (not after the fact), protection against leaks through answers, logs, and caches, minimizing personal data in line with GDPR, on-premise models where data cannot leave the company, and ongoing testing and monitoring.

Why does RAG change the rules of data security?

Because the model sees and summarizes everything retrieval returns. RAG (Retrieval-Augmented Generation) is an architecture where, before answering, the system retrieves matching chunks of company documents and passes them to the model as context. We explain how it works step by step in how RAG makes AI smarter.

A classic search engine shows a list of documents, and the access system decides whether the user can open them. RAG works differently: it merges the content of many chunks into one answer. If a clause from a legal contract or a salary table lands in the context, the model can summarize it for someone who would never be allowed to open that document.

The 2026 edition of the OWASP list of risks for LLM applications puts it briefly: in the retrieval layer, an access control failure makes the system indiscriminate about who gets to see what (OWASP GenAI Security Project, 2026). In agentic systems, where the model decides on its own what to search for and how many times, this matters even more. We cover those in our article on agentic RAG.

How does permission-aware (ACL-aware) retrieval work?

Every document and chunk in the index carries information about who may read it, and retrieval filters results by the user's identity inside the index query itself. An ACL (access control list) is the list of users or groups allowed to access a resource.

OWASP lists this as a baseline control for every deployment: authorize before retrieval, at document and chunk level, inside the index query, not in the application layer after results come back. A scope supplied by the client is "a suggestion, not a control", so identity and group membership are resolved on the server (OWASP GenAI Security Project, 2026).

Here is what that looks like in practice:

1. At ingestion you pull the list of groups with access from the source system (SharePoint, a file share, a DMS, a CRM) and store it with every chunk.
2. At query time the application resolves the user's identity from the login token, fetches their groups, and adds a filter to both the vector and the full-text query.
3. Only chunks that pass the filter reach the model. The model never decides about access.
4. When permissions change in the source, the index is updated. Access revoked at the source but removed from the index a week later means a week of exposure.

Search engines and vector databases ship mechanisms for this. Azure AI Search, for example, documents a security filter pattern where a field with group identifiers is filtered on every query, so documents without a matching group never come back (Microsoft Learn). Elasticsearch offers document and field level security (Elastic).

What should you watch out for?

• Chunks, not just documents. OWASP notes that a mostly public document can contain one confidential paragraph. For that kind of content, permissions belong at chunk level.
• One index for different trust levels. Web content, internal documents, and partner data should not share an index without hard isolation. For the most sensitive data, separate indexes beat labels in a shared one.
• Conclusions from combination. Individual chunks may be allowed while their combination is not (for example, a project list plus a list of people on sick leave). That is a data governance decision: which sources may be combined in one system.

How does data leak through RAG answers?

More often sideways than head-on. Missing permissions in the index are the most obvious channel, but not the only one.

The hidden instruction channel is indirect prompt injection: an attack where malicious instructions come from content the system reads rather than from the user. OWASP points out that an injection that lands in a RAG corpus or memory taints every later session that reads from it. That is why it pays to strip invisible characters and hidden text (such as white on white) at ingestion and to track the origin of every chunk. We go deeper on this threat in our article on securing AI agents and MCP servers.

The image channel is less obvious: if the chat interface automatically renders images from URLs the model generates, an attacker can push the model to write data into the URL. OWASP covers this under Improper Output Handling.

How should RAG handle personal data under GDPR?

Like any other system that processes personal data, with a few tasks specific to RAG. GDPR does not forbid AI knowledge bases, but it requires the scope of data, the purpose, and the safeguards to be thought through.

A report on privacy risks in LLMs, commissioned by the European Data Protection Board (EDPB) under its Support Pool of Experts program, lists three typical RAG risks: insecure logs and caches holding questions and retrieved documents, queries sent to third-party services, and exposure of personal data stored in the knowledge base. Among the safeguards it lists access control, anonymization or pseudonymization, access and change logs, and compliance with data transfer rules for outsourced RAG (EDPB, 2025).

In practice, the task list looks like this:

• Minimization (GDPR Article 5). Only documents needed for the purpose go into the index. Personal data that answers do not need (national ID numbers, contact details in email signatures, health data) is removed or pseudonymized at ingestion. Tools such as Microsoft Presidio can detect personal data in text (Presidio), but test them on your own data, because accuracy depends on language and format.
• Retention. Decide how long documents stay in the index and how long question logs and caches live.
• Right to erasure (Article 17). Erasing a person's data means deleting chunks and embeddings from the index, not only the file in the source. Build this into the ingestion pipeline.
• Impact assessment (Article 35). High-risk processing, such as large-scale processing of special categories of data, requires a DPIA. Prepare it with your data protection officer before launch.
• Processors (Article 28) and transfers. If the model or the database runs with a provider, you need a data processing agreement, knowledge of where processing happens, and an opt-out from training on your data (GDPR).

When should you run the model on-premise instead of a cloud API?

When documents or questions contain data that cannot leave the company: trade secrets, special categories of personal data, or records covered by professional secrecy. In that case the language model, the embedding model, and the reranker all run on your own infrastructure.

That is how we built our agentic knowledge base: it runs fully on-premise, on our own GPUs, with local models, so documents and questions never leave the company. We show what AI on your own infrastructure looks like on our AI infrastructure page.

How do you test and monitor RAG security?

As regularly as answer quality, because both degrade when the index, the model, or permissions change. RAG security is not a one-time review but a test suite that runs after every change.

Tests before launch and after every change:

• Permission tests. Questions from accounts in different departments about content they should not see. Expected result: a refusal or an answer without that data.
• Injection tests. Test documents with hidden instructions. You check whether the system follows them.
• Personal data leak tests. Questions that try to extract people's data from documents where it should not be visible.

Grounding and sources in every answer:

For the Misinformation risk, OWASP recommends grounding claims in current, authoritative sources and using groundedness checks rather than the model's confidence alone (OWASP GenAI Security Project, 2026). In our knowledge base, every claim in an answer cites a specific document fragment, and before it is sent the answer passes a grounding check against the sources and a hallucination detector. Citations help security too: you can see right away which document a statement came from, which makes it easier to spot a chunk that should never have been in the context.

Monitoring in production:

• a trace of every step (query, filters, retrieved chunks, answer) in an observability layer, with access control on the logs themselves,
• alerts on unusual patterns: many queries from one account, queries that return unusually many chunks, and chunks that match a very wide range of questions (OWASP describes this as a sign of index poisoning),
• a simple way to report bad answers and a path from the report back to the source.

Secure RAG checklist

A common mistake when building RAG is treating permissions as "phase two": the system is built on one shared index of every document, and access control is planned for later. Then the ingestion pipeline has to be rebuilt. It is cheaper to design permissions in from the first document.

Sources

• OWASP GenAI Security Project: OWASP GenAI LLM Top 10 2026
• EDPB: AI Privacy Risks and Mitigations, Large Language Models (2025)
• Regulation (EU) 2016/679 (GDPR)
• Microsoft Learn: Security filter pattern, Azure AI Search
• Elastic: Controlling access at the document and field level
• Microsoft Presidio: personal data detection and anonymization