RAG Improvements & Prompt Engineering Techniques for deep-seek coder RAG

Context: deep-seek coder local LLM augmented with a files library python, C, C++ documentation, theory on foundations algorithms in computer science, and top 100 most popular wikipedia articles.

Retrieval

1. Query Expansion

eg. "Expand query {query} into 5 versions using related/synonyms technical terminology from {domain} domain"

Pros:

• This allows the model to capture broader context, since the model is a coder it can leverage its training terminology to enhance the query
• Good for code-heavy queries (files/functions)
• Using this retrival prompt engineering technque would enbale the model to search a greater breadth since the model will rely on function names and conventions to find more context across texts from different authors (who might not describe things the same way but will liely use similar libraries or functions Cons:
• The model may retrieve contextually irrelevant information because using the same libraries and functions does not necessarily mean that the author is describng the same concept:
• Additionally, there is a lot of synonymous terminology in CS, for example "tree traversal" vs "graph traversal" - withou sematic grounding the model will conflate them

2. Hypothetical document embeddings (HyDE)

eg. "Write a hypothetical paragraph answering {query}, use this to find relevant documents"

Pros:

• This enables the model to use the depth of its own training data to frame the strucured of how it will search the docs, allowing the model to capture details that it could have missed
• Best for conceptual queries Cons:
• It's most effective when the model's training data closely matches the documents' domain and style, in this case there is some tangible variance of domains and source types (Python/C/C++ docs with Wikipedia and CS theory)
• HyDE is better for narrower domain scope

Generation

1. Chain of Thought Reasoning

eg. "Based on retrieved context: {retrieved documents}, answer query {query} step by step, first identifying key facts then reason through while expanding on each part of the solution/answer."

Pros:

• Using this technique allows the user to evaluate each part of the answer and evaluate how the model arrived there
• Leverages the deep-seek coder logic/reasoning capabilties
• Provides a specific framework for the model to use, decreasing ambiguity and increaing the likelihood of a structured answer
• Helps user understand the "why" behind answers Cons:
• CoT doesn't guarantee the model stays grounded in retrieved context, so the model may drift into general/out-of-domain knowledge

2. Extractive Answering

eg. "Extract the most relevant passages from retrieved documents {documents}"

Pros:

• Minimizes likelihood that the model will hallucinate as it generates the answers, since the answer will be extract(s) directly from the source
• Very clear where the model got its information Cons:
• Does not leverage the generative (summarization/explanation) capabilities of the model
• Extrated passages may not be sufficient in stand-alone context for them to be useful for the user

Combining Strategies: Retrieval:

• Filtering step where the model self-rates the relevance of each expanded query before retrieval Generation:
• Interleaved retrieval-then-reasoning, where the model explicitly cites or references the documents at each reasoning step
• Extractive summarization (identify key passages, then synthesize them into a coherent summary)

Retrieval Quality Metrics

Metric	Definition	Why It's Useful
Recall@K	% of relevant documents in top-K results	Tells you if your retrieval is missing critical information that could answer the query—no generation technique can compensate for documents you never retrieved.
Precision@K	% of top-K results that are relevant	Indicates the signal-to-noise ratio of your retrieval; high precision means CoT and other generation methods have cleaner, more focused context to work with, reducing hallucination opportunities.
MRR (Mean Reciprocal Rank)	Average rank of first relevant result	Measures user friction—if the first relevant answer is at rank 8, users may abandon the system before finding it, even if the document exists.
Domain Coverage	% of retrieved docs from each source (code docs vs. Wikipedia vs. theory)	Exposes whether your retrieval strategy is skewing toward one domain, which helps explain systematic failures like "too much general knowledge" or "missing code examples."
Retrieval Confidence Score	Model's own rating (0-10) of "does top-K answer this query?"	Provides a learned signal to automatically route queries to the right generation strategy (extractive vs. CoT) without requiring manual tuning of thresholds.

Incorporating a Retrieval-Generation Feedback Loop

1. USER QUERY
2. RETRIEVE DOCUMENTS (Query Expansion, HyDE, or standard embedding search) Output: Top-K results + retrieval metadata
3. COMPUTE RETRIEVAL CONFIDENCE SCORE (0-10): Overlap between query & retrieved docs, Precision@K of top results, etc
4. EXTRACTIVE SUMMARY, INTERLEAVED CoT & EXTRACTIVE, RE-RETRIEVE
5. GENERATE ANSWER
6. MEASURE ANSWER QUALITY: factual accuracy, query coverage, user feedback
7. UPDATE RETRIEVAL METRICS & THRESHOLDS

eg. Query: "Why are red-black trees better than AVL trees?

• Retrieve: Docs on red-black trees, AVL trees, balancing tradeoffs (some Wikipedia, some theory).
• Confidence Score: 5/10 (medium—docs exist but don't directly compare; synthesis needed).
• Strategy: Interleaved CoT + Citations.
• Answer: "Red-black trees are better in some cases because: (1) faster insertions [cite C++ theory doc], (2) looser balancing [cite Wikipedia], leading to fewer rebalances [cite algorithm foundation]. However, AVL trees offer tighter bounds [cite]; the choice depends on your workload."
• Feedback: User gives mixed feedback ("good explanation, but missing cache behavior")→ note gap; consider adding performance docs to knowledge base.