A language model only knows what it saw in training. It does not know your company policies, yesterday's news or a user's uploaded files. Retrieval-Augmented Generation (RAG) fixes that: before asking the model, you retrieve the relevant pieces of your own data and put them in the prompt.
The RAG pipeline
- Ingest: load documents (PDF, DOCX, HTML, text) and clean them.
- Chunk: split them into pieces of a useful size.
- Embed: turn each chunk into a vector.
- Store: save vectors and metadata in a vector database.
- Retrieve: embed the question and find the closest chunks.
- Generate: give the question and those chunks to the model and ask it to answer using only them.
RAG versus fine-tuning
- RAG adds knowledge at question time. Easy to update, can show sources, good for facts that change.
- Fine-tuning changes the model's behaviour or style using examples. Not a good way to teach it a body of facts.
- Many products use RAG first and consider fine-tuning only if prompts and retrieval are not enough.
Why it helps
- Fewer hallucinations, because the answer is grounded in supplied text.
- Answers can cite their sources.
- Private data stays in your control.
- You update knowledge by updating documents, not retraining.
Where it can go wrong
- The right passage was not retrieved, so the model has nothing to ground on.
- Chunks cut an idea in half.
- The model ignores the passages or mixes in its own memory.
The next lessons deal with each of these. For a resume product, RAG could retrieve guidance and examples for a job family before the model rewrites a section.