RAG interview questions
Retrieval-augmented generation (RAG) is the most common way companies connect large language models to their own documents. Interviews test whether you can build a RAG system that finds the right information and answers faithfully, and whether you can prove it works.
The questions below go from fundamentals to production problems, with what a strong answer should include.
What interviewers assess
- How a RAG pipeline works from documents to answer
- Retrieval quality: chunking, embeddings, hybrid search and re-ranking
- Evaluation: measuring retrieval and answer quality separately
- Production concerns: freshness, access control, cost and latency
Fundamentals questions
What is RAG and why use it instead of only prompting or fine-tuning?
What a good answer covers: Retrieve relevant documents at query time and give them to the model as context. It keeps knowledge current, lets answers cite sources and avoids retraining. Fine-tuning suits style and format more than facts.
Walk me through a basic RAG pipeline.
What a good answer covers: Ingest and clean documents, split them into chunks, create embeddings, store them in a vector index; at query time embed the question, retrieve the top chunks, build the prompt and generate an answer with citations.
What are embeddings, and how is similarity measured?
What a good answer covers: Vectors that represent meaning; similar texts are close together, usually measured with cosine similarity or dot product. The same embedding model must be used for documents and queries.
How do you choose a chunk size?
What a good answer covers: Small chunks are precise but lose context; large chunks keep context but add noise and cost. Split along document structure (headings, paragraphs), use some overlap, and tune with an evaluation set.
What does a vector database do?
What a good answer covers: Stores embeddings and finds the nearest ones quickly using approximate nearest-neighbour indexes (such as HNSW), often with metadata filtering. Mention that Postgres with pgvector is enough for many projects.
Improving retrieval questions
What is hybrid search and when does it help?
What a good answer covers: Combining keyword search (BM25) with vector search. Keyword search catches exact terms such as product codes, names and error messages that embeddings can miss.
What is re-ranking?
What a good answer covers: Retrieve a larger set quickly, then score each candidate against the question with a more accurate model (a cross-encoder) and keep the best few. It usually improves precision noticeably.
Users ask questions in different words from the documents. How do you handle that?
What a good answer covers: Query rewriting or expansion, hypothetical document embeddings (HyDE), hybrid search, and adding context such as titles or summaries to chunks before embedding.
How do you use metadata in retrieval?
What a good answer covers: Filter by date, product, language, department or user permissions before or during search; boost recent documents; show the source in citations.
When might long-context models replace RAG?
What a good answer covers: For small, fixed document sets, putting everything in the context can be simpler. RAG still wins on large or changing collections, cost, latency, access control and citing sources.
Evaluation and production questions
How do you evaluate a RAG system?
What a good answer covers: Separately measure retrieval (did the right chunks come back: recall and precision at k) and generation (is the answer faithful to the sources, relevant and complete), using a test set of real questions with known answers.
The system gives confident wrong answers. How do you find the cause?
What a good answer covers: Check whether the right content was retrieved. If not, it's a retrieval problem (chunking, search, missing documents); if it was, it's a generation problem (prompt, model, conflicting sources). Fix the right layer.
How do you reduce hallucinations in RAG answers?
What a good answer covers: Instruct the model to answer only from the provided sources, require citations, allow "I don't know", improve retrieval quality, and check faithfulness automatically on a sample.
How do you make sure users only see documents they are allowed to see?
What a good answer covers: Store permissions with each chunk and filter at retrieval time based on the user; never rely on the prompt to hide content.
How do you keep the index up to date as documents change?
What a good answer covers: Incremental ingestion triggered by changes, document IDs and versions for updates and deletes, and re-embedding when you change the embedding model.
Common mistakes
- Evaluating only the final answer, so you can't tell whether retrieval or generation failed.
- Using default chunking without testing alternatives.
- Relying only on vector search for content full of names, codes or numbers.
- Ignoring document permissions in a company setting.
How to prepare
- Build a small RAG app over documents you know well and write 20 test questions with answers.
- Measure retrieval and answer quality, then try one improvement (hybrid search or re-ranking) and measure again.
- Be ready to draw the pipeline and explain each choice.
- Prepare a story about a failure case you found and how you fixed it.
Reading questions only goes so far: the real test is answering them aloud, under follow-up questions. Career Scan builds a mock interview from your own resume and the job, then shows exactly which answers need work.
Frequently asked questions
Which vector database should I learn for interviews?
Any one is fine. The concepts (embeddings, indexes, filtering, hybrid search) transfer. pgvector, Pinecone, Weaviate, Qdrant and Chroma are common choices.
Is RAG still relevant with very long context windows?
Yes. Long context helps, but RAG remains cheaper, faster and easier to control for large or frequently changing document collections.
What is agentic RAG?
An agent decides when and how to search, can run several searches, refine queries and combine sources, rather than doing one fixed retrieval step.