Any explanation on how RAG works? #6

Activity

1. MrDoe commented on Jun 11, 2026

   

   MrDoe

   on Jun 11, 2026

   Owner

   More actions

   I get why this feels confusing — I had the same “who talks to whom?” moment when I first built a RAG pipeline. The simplest way I think about it: embeddings and the vector DB are the search engine, the LLM is the writer. You create vector representations of your documents (indexing) and store them in a DB like LanceDB. At query time you turn the user question into an embedding, use that to fetch the most relevant passages, then hand those passages (as plain text) to the LLM. The LLM never needs the vectors — it only needs the retrieved text to compose an answer.

   That means an LLM doesn’t have to “use the same” embedding provider to be part of RAG. What does matter is that the index and the query use compatible embeddings so the DB returns the right passages. If you index with EmbeddingGemma and also compute query embeddings with EmbeddingGemma (same model, same preprocessing), retrieval quality will be much better. If you mix very different embedding models, nearest‑neighbor search degrades and you’ll get less relevant context to feed the LLM.
2. MrDoe commented on Jun 11, 2026

   

   MrDoe

   on Jun 11, 2026

   Last edited by MrDoe

   Owner

   More actions

   To answer the second question: This is a typical RAG pipeline:

   Prerequisite - Index all files

   1. Split the files in chunks
   2. Use embedding LLM to calculate the vector of each chunk
   3. Store each chunk with vector and clear text in vector database.

   Semantic Search

   1. User prompt is sent to the embedding LLM.
      -> Embedding vector is generated.
   2. Find nearest neighbors of this vector in vector space to get the most relevant chunks.
   3. Pass the chunks (as text) to the LLM.
3. MrDoe

   closed this as completedon Jun 11, 2026
4. MrDoe commented on Jun 11, 2026

   

   MrDoe

   on Jun 11, 2026

   Owner

   More actions

   I've set up "Discussions" for questions in the repo now.
5. bilogic commented on Jun 12, 2026

   

   bilogic

   on Jun 12, 2026

   Last edited by bilogic

   Author

   More actions

   Thanks! Can this issue be converted to a discusssion, I have seen it happen on other repos.

   For clarity, I would like to name these:

   • embeddings provider, seems you can use embeddinggemma locally

   • vector database, I think it is LanceDB

   • LLM, seems like you can pick any from opencode, let's use big pickle from opencode

     Prerequisite - Index all files

     ...
     2. Use embedding LLM to calculate the vector of each chunk
     ...

     Semantic Search

     1. User prompt is sent to the embedding LLM.
        ...

   1. I see a new term embedding LLM in Prerequisite and Semantic Search
   2. Do they both refer to embeddinggemma or one of the LLM in opencode, e.g. big pickle?
   3. Are you able to point me to the code where the User prompt is sent to opencode in return for an embedding vector?
6. Repository owner locked and limited conversation to collaborators on Jun 12, 2026
7. MrDoe

   converted this issue into a discussion #8 on Jun 12, 2026