A RAG demo usually runs over one shared folder that everyone in the room is allowed to read. Production is different. The HR policy folder, the board pack, a customer's contract and the engineering wiki end up in the same index, and each user may see a different slice of it. If retrieval ignores that, the SI assistant will happily quote a salary review to someone who could never have opened the file.
Per-user access control in RAG is mostly a retrieval problem, not a model problem. The rule is easy to state: a passage the user is not allowed to read must never reach the model on that user's behalf. The rest of this post is about making that rule hold through indexing, permission changes, caches and shared conversations.
Why filtering after generation fails
The tempting shortcut is to retrieve from everything, generate an answer, then check the cited sources against the user's permissions and hide anything they should not see. It fails for a simple reason: by then the model has already read the restricted text. It can paraphrase it, summarise it, or let it shape an answer without citing it, and no source check will catch that.
Prompt instructions such as "only use documents this user may access" fail for the same reason. The model does not know the user's permissions, cannot enforce them reliably, and can be talked out of them. OWASP's guidance for LLM applications makes the same point in general terms: authorisation belongs in deterministic code outside the model.
Once the model has read a passage, assume the user can get it out.
How do you add document-level access control to a RAG system?
Add document-level access control to a RAG system by filtering at retrieval time, so passages a user cannot read never reach the model on their behalf. At ingestion, copy each document's permissions from its source system onto every chunk as metadata: allowed users, groups, roles or tenant. At query time, resolve the real user's identity and group membership in application code from their session, then pass it as a filter inside the search, ideally applied during the search rather than after it, so users with narrow access still get good results. Never rely on prompt instructions or on checking citations after generation, because by then the model has already read the restricted text. Keep permissions in sync with change events plus periodic reconciliation, and fail closed when a document's access is uncertain. Then check the side channels: answer caches, precomputed summaries, shared conversations and logs. Finally, test with users of different roles and planted marker documents.
Filter at retrieval time, using the user's permissions
The control that works is a filter applied inside the search itself. Every query carries the identity of the real user, resolved by your application from their session and never from anything typed into the chat. The retriever only searches passages that user is allowed to read, and the model only ever sees what survives.
Most vector databases and search engines support metadata filters, but how the filter is applied matters. Post-filtering takes the nearest results first and then drops the ones the user cannot see, which can leave users with narrow access very few results, or none. Pre-filtering, or filtering during the index traversal, restricts the search to permitted passages from the start. For access control, prefer the second, and measure retrieval quality for your most restricted users, not only for admins. If you combine keyword and vector search, the same filter has to apply to both, a detail worth checking in any hybrid search and reranking setup.
Carry the ACLs into the index as metadata
For the filter to work, each chunk needs to know who may read it. That means copying permission data from the source system into the index at ingestion time.
- Store the allowed principals on every chunk: user IDs, group IDs, roles, tenant ID or a classification level, whichever model your source systems use.
- Inherit permissions from the parent document, and from its folder or space where the source works that way. A chunk never has looser access than its document.
- Resolve group membership at query time where you can, so adding someone to a group does not mean re-indexing every document that group can read.
- Handle deny rules and exceptions explicitly. If the source system supports them and your index cannot express them, the safe default is to leave the document out.
- Partition tenants physically or logically where customers' data must never mix, rather than relying on a single filter field.
This is real work. Mapping permissions from several source systems into one model is often the hardest part of an internal RAG project, and it is worth scoping before anyone starts tuning prompts.
Keep permission changes in sync
Permissions change all the time: people move teams, leave the company, join a deal room or lose access to a client. An index that copied ACLs once becomes wrong the moment the first of those changes lands.
Treat permission updates as their own sync, separate from content updates and at least as frequent. Subscribe to change events from the source systems where they exist, run a periodic full reconciliation to catch what the events miss, and revoke a departed user's access before anything else. When a document is deleted or restricted at source, remove or re-tag its chunks promptly. A stale index is one of the RAG anti-patterns we keep finding, and for restricted content it is a leak rather than a quality problem.
Decide in advance what happens when the sync falls behind. The safer choice is to fail closed: if you cannot confirm a document's current permissions, leave it out of the results.
Watch caches, summaries and shared history
Retrieval filtering is the main control, but restricted content can escape through side doors that teams add later for speed or convenience.
- Answer caches. A semantic cache that returns a stored answer for a similar question will serve one user's answer to another. Key caches by user or by permission set, or do not cache answers built from restricted content.
- Precomputed summaries. A summary of a folder, a project or a customer account carries the strictest permissions of everything in it. Index it with those permissions, or do not index it.
- Shared conversation history. If a colleague can open, fork or continue someone else's chat, they inherit text retrieved under the original user's permissions. Re-check access, or share the question without the retrieved context.
- Memory features. Long-term memory that stores facts from past conversations must stay scoped to the user who produced them.
- Logs and traces. Prompt logs contain the retrieved passages, so access to the logging tool is access to the documents. Restrict it, or keep sensitive content out of logs as described in LLM logging without personal data.
Test it with users of different roles
Access control in RAG fails quietly. Nothing errors; a user simply receives a paragraph they should not have. So it needs its own tests, alongside the retrieval quality checks in RAG evaluation before launch.
- Create test users for each role that matters: a standard employee, a manager, a member of a restricted group, an external guest, a user from a second tenant, and a user whose access was just revoked.
- Plant marker documents that only one role can read, each containing a unique string.
- Ask every test user questions that the marker documents would answer, directly and indirectly, and fail the run if a marker appears for anyone who should not see it.
- Check the retrieved passages as well as the answer, so you catch a leak even when the model happens not to repeat it.
- Revoke a permission at source, let the sync run, and confirm the document disappears from that user's results.
- Rerun the suite whenever the ingestion pipeline, the filter logic, the cache or a source system integration changes.
When you need this, and when you do not
Not every RAG system needs per-user access control. If every user may read every document, one shared index is simpler and correct. Our Realty One property-search copilot answers over public listings, so the hard parts there were grounding and citations rather than permissions. The moment the corpus includes internal documents, customer records or anything with a sharing setting, the filter has to exist from the first ingestion run.
Access control also shapes the bigger architecture choice. Retrieval can enforce permissions per query, but knowledge trained into model weights cannot be withheld from one user and shown to another, which is one more reason to keep private documents out of training data, as we argue in RAG vs fine-tuning.
If you are planning an internal assistant over documents with mixed permissions, our RAG development team designs ingestion, permission sync and retrieval filtering together, and Octalcode can review an existing pipeline for the leak paths above.