RAG in ISMS

RAG stands for Retrieval-Augmented Generation RAG is a form of generative AI — it's an architecture/technique that combines a generative model (typically a large language model) with a retrieval step, rather than being a fundamentally different category of AI.

In our proof-of-concept project we use RAG to answer questions about information stored in our own security documentation! Our local AI will only retrieve what we have provided it with. 

At its core, RAG still uses an LLM to produce the final output — text generated token by token based on learned patterns, just like "plain" generative AI (e.g., a chatbot without retrieval).

Before generating a response, the system searches an external knowledge source (a document database, vector store, the web, etc.) for relevant information, and feeds that retrieved content into the model's context alongside the user's query. The model then generates its answer grounded in that retrieved material.

Why this matters
- reduces hallucination: the model can reference actual retrieved facts rather than relying purely on what it "remembers" from training.
- access to current/private data: it can pull in information the model was never trained on (recent events, internal company documents, etc.).
- traceability: outputs can often be traced back to specific source documents.

This video shows how RAG formulates an answer to "Can an administrator use a privileged account for normal office work?"