Train your own Reasoning model like DeepSeek-R1 locally (5GB VRAM min.)
Hey guys! This is my first post on here & you might know me from an open-source fine-tuning project called Unsloth! I just wanted to announce that we made a new update today so you can now train your own reasoning model like R1 on your own local device! 5gb VRAM works with Qwen2.5-1.5B.
- R1 was trained with an algorithm called GRPO, and we enhanced the entire process, making it use 90% less VRAM + 10x longer context lengths.
- We're not trying to replicate the entire R1 model as that's unlikely (unless you're super rich). We're trying to recreate R1's chain-of-thought/reasoning/thinking process
- We want a model to learn by itself without providing any reasons to how it derives answers. GRPO allows the model to figure out the reason autonomously. This is called the "aha" moment.
- GRPO can improve accuracy for tasks in medicine, law, math, coding + more.
- You can transform Llama 3.1 (8B), Phi-4 (14B) or any open model into a reasoning model. You'll need a minimum of 7GB of VRAM to do it!
- In a test example below, even after just one hour of GRPO training on Phi-4, the new model developed a clear thinking process and produced correct answers, unlike the original model.
Highly recommend you to read our really informative blog + guide on this: https://unsloth.ai/blog/grpo
To train locally, install Unsloth by following the blog's instructions & installation instructions are here.
I also know some of you guys don't have GPUs, but worry not, as you can do it for free on Google Colab/Kaggle using their free 15GB GPUs they provide.
We created a notebook + guide so you can train GRPO with Phi-4 (14B) for free on Colab: https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Phi_4_(14B)-GRPO.ipynb-GRPO.ipynb)
Thank you for reading! :)