Thanks (to both you and the parent) for sharing these details. So is it fair to say the following:
1. Fine-tuning bakes the knowledge into the model, but getting the "source" of an answer to a specific question becomes cagey and it is unclear if the answer is accurate or just a hallucination.
2. Therefore vector databases, which can provide context to the LLM before it answers, can solve this "citation" problem, BUT:
3. We then have limits because of the context window of the LLM to begin with.
Is that a fair understanding, or have I totally gotten this incorrect?
Edit: Or, are you saying that you both fine-tune AND also use a vector database which stores the embeddings of the dataset used to fine-tune the model?
1. Fine-tuning bakes the knowledge into the model, but getting the "source" of an answer to a specific question becomes cagey and it is unclear if the answer is accurate or just a hallucination.
2. Therefore vector databases, which can provide context to the LLM before it answers, can solve this "citation" problem, BUT:
3. We then have limits because of the context window of the LLM to begin with.
Is that a fair understanding, or have I totally gotten this incorrect?
Edit: Or, are you saying that you both fine-tune AND also use a vector database which stores the embeddings of the dataset used to fine-tune the model?