What architecture allows an LLM to cite its sources from a corpus of documents?

Retrieval Augmented Generation (RAG) architecture is specifically designed to allow large language models (LLMs) to enhance their responses by integrating information retrieved from external sources, such as a corpus of documents. This architecture combines the generative capabilities of a language model with a retrieval mechanism.

In practice, when a query is presented, RAG first retrieves relevant passages or documents from the corpus based on the input context. The generative model then uses this retrieved information to craft a response that is more informed and contextually relevant. This process enables the model to cite specific sources and incorporate accurate, up-to-date information from a designated set of documents, thus improving the reliability and credibility of its outputs.

This architecture is particularly useful in scenarios where factual accuracy is essential, as it helps mitigate the risks of hallucination commonly associated with traditional generative models that rely solely on their training data.