That sounds about right! To the first point, I'd add that indexing/retrieval augmented generation could be another alternative that falls between in-context learning and finetuning. It's especially useful if the info is too large to fit it into the context.
Regarding your second point, I like the KL divergence idea from PPO, but including…
That sounds about right! To the first point, I'd add that indexing/retrieval augmented generation could be another alternative that falls between in-context learning and finetuning. It's especially useful if the info is too large to fit it into the context.
Regarding your second point, I like the KL divergence idea from PPO, but including a small percentage of the original data is probably the safer bet. In https://arxiv.org/abs/2403.08763 they found it was quite effective.
Regarding RAG, my knowledge is pretty limited so I am guessing the simpler form would be to load the new data, chunk it to fit in the context window, and store it in a vector DB. Seems like a pretty efficient way to get performance bumps in some cases.
But then it seems like we have to pay much attention to database?
That sounds about right! To the first point, I'd add that indexing/retrieval augmented generation could be another alternative that falls between in-context learning and finetuning. It's especially useful if the info is too large to fit it into the context.
Regarding your second point, I like the KL divergence idea from PPO, but including a small percentage of the original data is probably the safer bet. In https://arxiv.org/abs/2403.08763 they found it was quite effective.
Regarding RAG, my knowledge is pretty limited so I am guessing the simpler form would be to load the new data, chunk it to fit in the context window, and store it in a vector DB. Seems like a pretty efficient way to get performance bumps in some cases.
But then it seems like we have to pay much attention to database?
Actually, for most practical use cases you don't even need a database but can store it in memory. I have an example here: https://github.com/rasbt/MachineLearning-QandAI-book/blob/main/supplementary/q18-using-llms/03_retrieval-augmented-generation/retrieval-augmented-generation.ipynb
Thank you for sharing this 🙌🏻
These notebooks look really helpful for starters. I hope I can make some time to play with them soon!