RAG and enterprise AI: governing proprietary generative AI

Short answer: Retrieval-Augmented Generation (RAG) lets a generative AI model answer from your own company documents rather than only from its training knowledge. That lowers fabrication and keeps sensitive knowledge inside your own sources, but it shifts the risk to access control, confidentiality and the provenance of answers. Good governance turns on source scoping, authorisation, logging and human oversight.

What RAG changes about the risk profile

In RAG the system retrieves relevant passages from a proprietary knowledge source and passes them to the model as context. The model therefore does not "know" your documents — it is handed them per query. The benefit: answers are traceable to a source, and the model need not be trained on your data, which reduces the risk of eroding trade secrets. But quality and safety now depend on which documents are retrievable and who may see them.

The biggest pitfall: authorisation

A RAG system that makes all company documents searchable without filtering can expose sensitive information to employees who normally have no access to it. The retrieval layer must respect existing access rights: a user must not see more through the chatbot than through the source system. This is the difference between a handy assistant and an uncontrolled data leak.

Confidentiality and the choice of model

Whether you use a hosted model or your own environment determines where your context data ends up. Choose an offering where retrieved passages are not used for training and not retained. If the source contains personal data, GDPR requirements apply: legal basis, processor agreement and clarity on the processing location.

Provenance, hallucination and human oversight

RAG reduces fabrication but does not eliminate it: the model can mis-summarise retrieved passages or ignore context. Always show the source alongside the answer so a user can verify. For decisions with legal effect or high risk, human oversight remains required — a RAG assistant is a tool, not a decision-maker.

Anchoring governance

Treat an internal RAG system like any other AI application: it belongs in your AI governance framework, with an owner, a risk assessment and logging. If you use a general model as the base, the GPAI regime may apply to the provider, while you as a user have your own obligations.

What to do

• Mirror access rights: the retrieval layer must never expose more than the source system allows.
• Choose a model/environment in which context data is not used for training or retained.
• Show sources with every answer and keep human oversight for weighty decisions.
• Curate the knowledge source: outdated or wrong documents lead to wrong answers.
• Log use and outcomes and give the system an owner within your governance.