Retrieval-Augmented Fine-Tuning

Retrieval-augmented fine-tuning (RAFT) is a training approach that combines the benefits of fine-tuning and retrieval-augmented generation. Instead of choosing between teaching the model new knowledge (fine-tuning) or providing knowledge at query time (RAG), RAFT trains the model to be better at both retrieving and using retrieved information for a specific domain.

The motivation

Standard RAG has a limitation: the base model may not be optimally trained to use retrieved documents. It might ignore relevant retrieved information, over-rely on its parametric knowledge, or struggle to synthesize information from multiple retrieved passages. RAFT addresses this by fine-tuning the model specifically on the task of answering questions using retrieved context.

How RAFT works

The training data for RAFT includes questions, relevant documents (that contain the answer), distractor documents (that are related but do not contain the answer), and the correct answer with citations to the relevant document.

During training, the model learns to identify which retrieved documents are relevant, extract the correct information from those documents, ignore distractors, and generate answers that are faithful to the source material. Critically, some training examples include the relevant document and some do not, teaching the model to know when to rely on retrieved context versus its own knowledge.

RAFT vs standard approaches

• Standard RAG: Uses a base model with retrieved context. The model was not specifically trained to use retrieved documents effectively.
• Standard fine-tuning: Teaches the model new knowledge baked into its weights. No retrieval mechanism.
• RAFT: Fine-tunes the model to excel at using retrieved documents, combining the adaptability of RAG with the domain specialization of fine-tuning.

Practical benefits

RAFT produces models that are more accurate when working with domain-specific document collections. They make fewer hallucinations because they are trained to ground responses in retrieved evidence. They handle "distractor" documents better — real-world retrieval often returns partially relevant results, and RAFT models learn to navigate this noise.

When to use RAFT

RAFT is most valuable when you have a specific domain with a defined document collection, accuracy and faithfulness to sources is critical, and standard RAG produces too many hallucinations or misses important information from retrieved documents.