RAG Interview Questions for 2 Years Experience

RAG combines a retrieval system with an LLM: retrieve relevant documents from a knowledge base, then pass them as context to the LLM to generate answers. It's better than fine-tuning for: (1) Up-to-date knowledge — add new docs without retraining, (2) Source attribution — you can cite which document answered the question, (3) Cost — no expensive GPU training, (4) Interpretability — you can debug what was retrieved. Fine-tuning is better for style, tone, and domain-specific reasoning patterns.

Chunking splits documents into smaller pieces for embedding and retrieval. Common strategies: (1) Fixed-size with overlap (chunk_size=512, overlap=50) — simple but ignores structure, (2) Recursive character splitting — splits on paragraphs → sentences → words, (3) Semantic chunking — uses embedding similarity to split at topic boundaries, (4) Document-aware (markdown headers, code blocks). Rule of thumb: chunk size should match the granularity of your queries. For Q&A, 256–512 tokens works well. For summarization, 1000–2000 tokens.

Sparse retrieval (BM25, TF-IDF) uses keyword matching — fast, interpretable, good for exact term matches. Dense retrieval uses neural embeddings — captures semantic similarity, better for paraphrase matching. Hybrid search combines both: dense retrieval for semantic similarity + sparse for keyword precision. In production, hybrid search with reciprocal rank fusion typically outperforms either approach alone.

Vector databases store high-dimensional embeddings and support fast approximate nearest neighbor (ANN) search. Comparison: Pinecone — managed cloud, easiest setup, best for production, paid. Chroma — local/self-hosted, great for development and small apps, free. Weaviate — open-source, supports hybrid search natively, self-hosted. Qdrant — open-source with cloud option, very fast, good filtering support. For startups: start with Chroma locally, deploy Pinecone in production.