Combining RAG with Continued Pretraining of LLMs
This story is from 2026-09-15. It is preserved in the archive; the latest stories are on the live feed.
For leaning purposes, I ran an experiment where I used Unsloth to train a model on a new domain using continued pretraining. Then to make it more flexible, I added a RAG step to inject dynamic data to augment the stable training data. The specific example is training qwen 3.5 4B on a fictional subw…
Read the full story at r/artificial ↗
Timeline · 3 reports
- 2026-09-17 13:18 · r/ArtificialInteligence
Combining RAG and Continued Pretraining of QWEN 3.5 4B for Better Results - 2026-09-16 02:27 · r/reinforcementlearning
Combining RAG and Continued Pretraining - 2026-09-15 17:13 · r/artificial
Combining RAG with Continued Pretraining of LLMs