RAG interview questions, answered the way you’d say them.

Use test questions whose relevant passages you know. Recall@k is the share of a question’s relevant passages that appear in the top k results, averaged over questions; MRR averages 1 divided by the rank of the first relevant result, scoring 0 when none is found. Recall@k says whether the evidence reached the model at all; MRR says how high it ranked.

def recall_at_k(results, relevant, k):
    return sum(len(set(r[:k]) & rel) / len(rel) for r, rel in zip(results, relevant)) / len(results)
def mrr(results, relevant):
    rr = lambda r, rel: next((1 / i for i, d in enumerate(r, 1) if d in rel), 0.0)
    return sum(rr(r, rel) for r, rel in zip(results, relevant)) / len(results)
results = [["d1", "d4", "d2"], ["d5", "d6", "d3"], ["d7", "d8", "d9"]]
relevant = [{"d1"}, {"d3"}, {"d0"}]
print(recall_at_k(results, relevant, 3), mrr(results, relevant))   # 0.666... 0.444...

Some write-ups call the hit rate “recall@k”; the ir-measures library names that Success@k, so say which one you mean.

ir-measures documentation: Measures

Practise it with Prism

Usually, yes. Long inputs cost tokens and time on every call, and models use them unevenly: on the RULER benchmark almost all models dropped sharply as context grew despite near-perfect needle-in-a-haystack scores, and only half of those claiming 32K tokens held up at 32K. Retrieval also enforces access control and keeps answers current. For a small, stable collection, though, putting it all in the prompt with caching can be simpler.

Also mention “lost in the middle”: models use information at the start or end of a long input better than information in the middle.

Hsieh et al., RULER: What's the Real Context Size of Your Long-Context Language Models? (2024)Liu et al., Lost in the Middle: How Language Models Use Long Contexts (2023)Anthropic Engineering, Introducing Contextual Retrieval (2024)

Practise it with Prism

Retrieval-augmented generation pulls the passages that best match a question into the prompt, which lets the model answer from your own documents and cite them. The 2020 paper that coined the name joined a generator to a neural retriever, which searched a dense vector index. The big advantage over retraining is that new knowledge only means updating the index, while the model stays as it is.

Lewis et al. 2020: Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks

The full answer, in the guidePractise it with Prism

Add the missing context to each chunk before indexing. The simplest step is to prepend the document title and section headings. Anthropic’s contextual retrieval goes further: a model writes a short note situating each chunk within its document, prepended before both embedding and BM25 indexing, which in their tests cut top-20 retrieval failures from 5.7% to 2.9%, and to 1.9% with re-ranking.

For small collections, under about 200,000 tokens, Anthropic suggests skipping retrieval and putting the whole knowledge base in the prompt with prompt caching.

Anthropic Engineering, Introducing Contextual Retrieval (2024)

Try it in RAG chunkingPractise it with Prism

For each query term, BM25 multiplies the term’s rarity across the collection (inverse document frequency) by a term-frequency score that saturates, so the tenth mention adds less than the first, and it normalises for document length. Elasticsearch’s defaults are k1 = 1.2 for saturation and b = 0.75 for length normalisation. On the BEIR benchmark, BM25 proved a robust baseline across domains while dense retrievers often generalised worse, which is why hybrid search keeps both.

Know each side’s blind spot: BM25 misses synonyms and paraphrases, while embeddings can miss exact codes such as an IFSC or a policy number.

Elasticsearch reference: Similarity settings (BM25)Thakur et al., BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models (2021)

Try it in RAG chunkingPractise it with Prism

Give each document 1 / (k + rank) from every list it appears in and sort by the total, with k = 60 as Cormack and colleagues used. Only ranks are needed, so it sidesteps the fact that BM25 scores and cosine similarities sit on different scales. A document ranked well by both systems rises to the top, and k limits how much a single first place from one outlier system can dominate.

def rrf(rankings, k=60):
    scores = {}
    for ranking in rankings:                              # doc IDs, best first
        for rank, doc in enumerate(ranking, start=1):
            scores[doc] = scores.get(doc, 0) + 1 / (k + rank)
    return sorted(scores, key=scores.get, reverse=True)
bm25 = ["circular_12", "faq_3", "policy_7"]
dense = ["policy_7", "circular_12", "blog_9"]
print(rrf([bm25, dense]))      # ['circular_12', 'policy_7', 'faq_3', 'blog_9']

If asked why not average the raw scores: they aren’t comparable across systems without normalisation, which RRF avoids by using ranks.

Cormack, Clarke and Büttcher, Reciprocal Rank Fusion outperforms Condorcet and individual Rank Learning Methods (SIGIR 2009)

Practise it with Prism

A cross-encoder reads the query and a document together, so attention spans both and it judges relevance more accurately, but it must run once per pair at query time. A bi-encoder embeds documents in advance, so search is a fast vector lookup. Sentence-BERT’s authors put numbers on the gap: finding the most similar pair among 10,000 sentences took about 65 hours with BERT reading pairs, and about 5 seconds with their bi-encoder. So retrieve cheaply, then re-rank about the top 100.

Name the trade-off: the re-ranker can only reorder what the first stage found, so first-stage recall sets the ceiling.

Reimers and Gurevych, Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks (2019)Sentence Transformers: Retrieve & Re-Rank

Practise it with Prism

Rewrite when the raw query makes a poor search string: a follow-up such as “what about its fees?” needs the earlier turn folded in, and a short question may look nothing like the documents. HyDE has a model write a hypothetical answer, embeds it and searches with that vector; the invented text may contain false details, but only real documents near it are returned. Each rewrite adds a model call, so measure the gain on your own questions.

Never show the hypothetical document to the user: it’s a search key, not an answer.

Gao, Ma, Lin and Callan, Precise Zero-Shot Dense Retrieval without Relevance Labels (HyDE, 2022)

Practise it with Prism

It has seven stages. I ingest PDFs, tables and scans as clean text with metadata, because a parsing mistake spoils every step after it, then chunk with some overlap and index a vector per chunk, tagged with date and document type. Retrieval finds the top matches, often mixing vector and keyword search, and a re-ranker keeps the best few. The model answers from them with citations or admits the documents don’t cover it, and I score a fixed question set after every change.

The full answer, in the guidePractise it with Prism

It’s a trade-off: small chunks match precisely but drop surrounding context, while big ones preserve context but dilute relevance and spend more tokens. I split along the document’s structure first, by headings, sections and paragraphs, let neighbouring chunks overlap a little, so a sentence split between two chunks isn’t lost, and settle the size by measuring retrieval on my own questions. LangChain’s RecursiveCharacterTextSplitter works this way, trying to keep paragraphs intact, then sentences, then words.

Mind the units: in RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=150), both numbers are characters, not tokens.

LangChain: Text splitters

The full answer, in the guideTry it in RAG chunkingPractise it with Prism

Exact, or flat, search scores the query against every stored vector, so recall is perfect but it gets slow as the collection grows. HNSW is a graph-based approximate index that is far faster at the cost of a little recall, though FAISS points out that it can’t remove vectors. To get cosine similarity from FAISS, I normalise every vector to unit length and use inner product.

Start with an exact index and switch to HNSW once the collection gets large.

FAISS wiki: Faiss indexes

The full answer, in the guidePractise it with Prism

Embeddings capture meaning but can miss exact strings, like a product code, a section number or a circular ID, and keyword search such as BM25 catches those. Hybrid search does both searches and combines the result lists, often by reciprocal rank fusion. Then a cross-encoder re-ranker takes each candidate together with the question and rescores the top few.

Sentence Transformers: Quickstart

The full answer, in the guideTry it in RAG chunkingPractise it with Prism

I measure retrieval and generation separately, using a golden set of real questions whose answers and sources are known. For retrieval, I check whether the right passage made the top k and how high it ranked. For answers, I check that they’re backed by the retrieved text, which is faithfulness, and that they actually answer the question; in Ragas these are context precision and context recall for retrieval, and faithfulness and response relevancy for answers.

Ragas: List of available metrics

The full answer, in the guidePractise it with Prism

It should say it couldn’t find the answer rather than guess. I’d tell it to use nothing but the retrieved context and to reply with something like “I couldn’t find this in the documents”, and set a retrieval-score threshold under which it declines to answer. I’d also add unanswerable questions, such as next year’s figures, to the test set, so refusals are measured as well.

The full answer, in the guidePractise it with Prism

It stores embeddings with IDs and metadata, finds the ones closest to a query quickly, and adds what a similarity-search library such as FAISS leaves to you: inserts and deletes, metadata filters, persistence and access control; pgvector, for example, brings this to PostgreSQL. Its indexes are approximate: HNSW searches a layered proximity graph, fast and accurate but memory-hungry; IVF clusters the vectors, often with k-means, and searches only the nearest clusters; product quantisation compresses vectors into short codes. Each has a speed–recall knob, such as efSearch or nprobe.

FAISS notes that its HNSW index can’t remove vectors, which matters if documents are deleted often.

FAISS: READMEFAISS wiki: Faiss indexespgvector: READMEMalkov and Yashunin, Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphs (2016)

Practise it with Prism

The simplest chunker cuts fixed-size windows; a sliding window makes neighbouring chunks overlap, so a sentence cut at one boundary survives whole in the next chunk. Production systems usually go further. Recursive splitting, like LangChain’s RecursiveCharacterTextSplitter, keeps paragraphs whole, then sentences, then words; document-aware splitting follows structure, such as Markdown headers, HTML tags or code functions; and semantic splitting cuts where the topic shifts between neighbouring sentences. Count sizes in tokens if your limit is in tokens.

def chunk_words(text, size=200, overlap=40):        # a sliding window over words
    assert 0 <= overlap < size
    words, step = text.split(), size - overlap      # each window starts size - overlap words later
    return [" ".join(words[i:i + size])
            for i in range(0, max(len(words) - overlap, 1), step)] if words else []
print(chunk_words("a b c d e f g h i j", size=4, overlap=1))
# ['a b c d', 'd e f g', 'g h i j']

Store each chunk’s source, position and headings as metadata, so answers can cite where they came from.

LangChain: Text splitters

Try it in RAG chunkingPractise it with Prism

Split the answer into claims and check each against the retrieved context; faithfulness is the share of claims the context supports, which is how Ragas defines it. The check can be an LLM judge or a trained classifier such as HHEM. Verify citations too: the ALCE benchmark scores whether cited passages support each statement, and found even the best models lacked complete citation support half the time on one dataset.

Faithful isn’t the same as correct: an answer can faithfully repeat an outdated document, so score retrieval and answer correctness as well.

Ragas documentation: FaithfulnessGao et al., Enabling Large Language Models to Generate Text with Citations (2023)

Practise it with Prism

Trace each bad answer to a stage, using the seven failure points Barnett and colleagues reported: the content isn’t in the index, the right chunk ranked below the top k, it was retrieved but dropped before the prompt, it reached the prompt but wasn’t used, or the answer had the wrong format, specificity or completeness. Log the query, retrieved chunks and final prompt for every answer, and score retrieval and generation separately on a fixed question set.

Their headline lesson: a RAG system can only be fully validated in operation, so plan monitoring from the first release.

Barnett et al., Seven Failure Points When Engineering a Retrieval Augmented Generation System (2024)

Practise it with Prism

Isolation has to be enforced in retrieval, never left to the prompt. Every chunk carries a tenant ID and every query filters on it, with the tenant read on the server from the signed-in user’s identity; the alternative is a separate index for each tenant. Then I’d test with questions that try to cross tenants, and isolate caches and logs in the same way.

The full answer, in the guidePractise it with Prism

Collect query–passage pairs from your domain: search logs, FAQ pairs, or questions written by an LLM and checked by people. Train with in-batch negatives, where each query must score its own passage above every other passage in the batch, as Sentence Transformers’ MultipleNegativesRankingLoss and DPR do, and add mined hard negatives: passages that look relevant but aren’t. Triplet loss, which pushes an anchor closer to its positive than to a negative by a margin, is the older option. Compare recall@k with the base model on held-out queries.

With in-batch negatives, bigger batches mean more negatives per query, which is why cached variants exist to fit larger batches in memory.

Sentence Transformers: LossesKarpukhin et al., Dense Passage Retrieval for Open-Domain Question Answering (2020)

Practise it with Prism

Time each stage first: embedding the query, retrieval, re-ranking, the model’s prefill and its generation. Then cut where the time goes. Cache embeddings and answers for repeated queries; tune the approximate index’s speed–recall setting, such as HNSW’s efSearch; re-rank fewer candidates, or with a smaller model; and send fewer, tighter chunks, since a longer prompt raises time to first token. Put fixed instructions first so prompt caching can reuse them, stream tokens, run independent steps in parallel, and send simple questions to a smaller model.

Report the 95th or 99th percentile, not the average: the occasional slow re-rank or cache miss is what users notice.

NVIDIA NIM: LLM inference benchmarking metricsClaude API docs: Prompt cachingFAISS wiki: Faiss indexes

Practise it with Prism

Treat retrieved text as untrusted data, and never let it trigger a tool without checks. Accept documents only from verified sources and record where each came from, so a poisoned one can be traced and removed; PoisonedRAG forced chosen answers 90% of the time by adding five crafted texts per target question to a database of millions. Enforce access permissions inside retrieval, mask personal data before indexing and in logs, and keep tamper-proof retrieval logs. For staleness, store effective dates and versions, and delete superseded chunks when re-indexing.

OWASP’s 2025 list treats this as its own risk, vector and embedding weaknesses, and recommends permission-aware vector stores.

OWASP Top 10 for LLM Applications 2025: LLM08 Vector and Embedding WeaknessesZou et al., PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Models (2024)

Practise it with Prism

For multi-hop questions, retrieve in steps: split the question into sub-questions, or let each reasoning step choose the next search, as IRCoT does, which improved retrieval by up to 21 points. Keep tables structured, as Markdown or rows the model can query, instead of flattening them into prose. For questions that span many documents, GraphRAG builds an entity graph with community summaries. And route by difficulty: Adaptive-RAG trains a small classifier to pick no retrieval, one step or several for each query.

Score each route on its own question set, since one average can hide that multi-hop questions still fail.

Trivedi et al., Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions (IRCoT, 2022)Jeong et al., Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexity (2024)Edge et al., From Local to Global: A Graph RAG Approach to Query-Focused Summarization (2024)

Practise it with Prism

Don’t make the model read figures out of text chunks or do arithmetic in its head. Keep prices and fundamentals in a database and let the model call tools, such as parameterised SQL or a market-data API, with code doing every calculation; PAL showed that handing the computation to a Python interpreter avoids the arithmetic slips language models make. Retrieve filings and news as text only for context, filtered by ticker and date. Show each figure with its source and timestamp, and test with numeric questions scored by exact match.

Use strict tool definitions, so every call matches its schema, and log each query for audit.

Gao et al., PAL: Program-aided Language Models (2022)Claude API docs: Tool use with Claude

Practise it with Prism

Know the costs: Claude, for example, reads each PDF page as extracted text plus an image, typically 1,500 to 3,000 text tokens a page, so dense files can fill the context before the page limit. Split large documents and send only the relevant pages, found by retrieval over the extracted text; put the PDF before the question; and cache the document with prompt caching when you ask it several questions. Ask for quotes or page citations so answers can be checked, and use batch processing for high volumes.

Run independent sections in parallel and merge the answers, rather than one long request that risks timing out.

Claude API docs: PDF supportClaude API docs: Prompt caching

Practise it with Prism

When answers depend on relationships spread across many documents, or the question is about the whole corpus, such as “what are the main themes in these reports?”, which top-k chunk retrieval can’t assemble. GraphRAG uses an LLM to extract entities and relationships into a knowledge graph, clusters it into communities with the Leiden algorithm, and pre-writes community summaries for global questions. Its authors found large gains in comprehensiveness and diversity on such questions over corpora of about a million tokens. For simple lookups, plain RAG is cheaper and enough.

Indexing calls the LLM over every chunk of the corpus, so estimate that cost before you choose GraphRAG.

Edge et al., From Local to Global: A Graph RAG Approach to Query-Focused Summarization (2024)Microsoft GraphRAG documentation

Practise it with Prism