# Context we want to add qlora (lora + 4 bits int quant) to our codebase. The goal is to reduce memory usage and the cost of finetuning without degrading quality. [paper](https://arxiv.org/abs/2305.14314) [blog post](https://huggingface.co/blog/4bit-transformers-bitsandbytes) ## Steps - [x] activate 4bits in Peft. Should be as easy as turning on a flag - [x] run the modal on one GPU for one epoch and look at memory consumption compared to 8 bits training - [x] run a full training (3 epochs) with the same parameters as 8bits training and compare result to see if we don't degrade quality
Context
we want to add qlora (lora + 4 bits int quant) to our codebase.
The goal is to reduce memory usage and the cost of finetuning without degrading quality.
paper
blog post
Steps