QLoRA Fine-Tuning Framework

A config-driven, production-ready framework for fine-tuning any causal language model with QLoRA (4-bit quantization + LoRA adapters). Built on Hugging Face transformers, peft, trl, and bitsandbytes.

Use it for price prediction, sentiment classification, instruction following, chat tuning, or any supervised text generation task.

Features

• Generic: YAML config for model, data, LoRA, training, hub, and WandB settings
• Multiple data formats: prompt_completion, instruction, chat, and raw text
• Hugging Face Hub & local JSONL datasets
• Lite preset: quick-run overrides for Colab / smaller GPUs
• Hub push: optional automatic push to a private Hugging Face repo
• WandB: optional experiment tracking
• CLI: qlora-finetune run, validate-config, print-config
• uv: fast, reproducible dependency management

Requirements

• Python 3.11+
• NVIDIA GPU with CUDA (recommended: ≥16 GB VRAM for 3B models in 4-bit)
• uv installed

Quick Start

# Install dependencies
uv sync

# Copy and edit environment variables
cp .env.example .env
# Set HF_TOKEN (required) and WANDB_API_KEY (if using WandB)

# Validate a config
uv run qlora-finetune validate-config examples/price_prediction/config.yaml

# Run fine-tuning (from project root)
uv run qlora-finetune run examples/price_prediction/config.yaml

Project Structure

qlora-finetuning-framework/
├── pyproject.toml          # uv / hatch project definition
├── src/qlora_finetune/
│   ├── config.py           # Pydantic models + YAML loading
│   ├── data.py             # Dataset loading + text formatting
│   ├── model.py            # Quantized model + LoRA setup
│   ├── trainer.py          # SFTTrainer pipeline
│   ├── tracking.py         # HF + WandB auth
│   └── cli.py              # Typer CLI
├── examples/
│   ├── price_prediction/   # Product price prediction
│   ├── sentiment_analysis/ # Review sentiment classification
│   ├── instruction_following/ # Alpaca-style instruction tuning
│   └── local_jsonl/        # Fine-tune from a local JSONL file
└── tests/

Configuration

Every run is driven by a YAML file. Key sections:

Section	Purpose
project	Name, run name, output directory, seed
model	Base model ID, 4-bit/8-bit quantization
data	Dataset source, splits, field names, format
lora	Rank, alpha, dropout, target modules
training	Epochs, batch size, LR, logging, eval
hub	Push to Hugging Face Hub
wandb	Weights & Biases tracking
lite	Preset overrides for fast / small-GPU runs

Data formats

Format	Description	Typical fields
prompt_completion	Concatenate prompt + completion	prompt, completion
instruction	Alpaca-style blocks or chat template	instruction, input, output
chat	Multi-turn messages	messages (list of {role, content})
text	Pre-formatted text column	text

Lite preset

Enable lite.enabled: true to apply smaller defaults (fewer LoRA modules, smaller val set, etc.) without rewriting the full config. The price prediction example uses this mode.

Examples

1. Price prediction (Hugging Face dataset)

Fine-tune Llama 3.2 3B on arneesh/items_prompts_lite (replace this with yours)

# Edit examples/price_prediction/config.yaml — set hub.user to your HF username
uv run qlora-finetune run examples/price_prediction/config.yaml

2. Sentiment analysis (local JSONL)

Small demo dataset for review sentiment classification.

uv run qlora-finetune run examples/sentiment_analysis/config.yaml

3. Instruction following (Alpaca dataset)

General assistant-style tuning on yahma/alpaca-cleaned.

uv run qlora-finetune run examples/instruction_following/config.yaml

4. Local JSONL

Bring your own data as JSONL with instruction, input, and output fields.

uv run qlora-finetune run examples/local_jsonl/config.yaml

Creating Your Own Task

1. Prepare data as Hugging Face dataset or JSONL.
2. Copy an example config.yaml and adjust:

• model.base_model — any causal LM on Hugging Face
• data — source, fields, and format
• lora.target_modules — match your model architecture
• training — batch size and sequence length for your GPU

3. Run:

uv run qlora-finetune validate-config your/config.yaml
uv run qlora-finetune run your/config.yaml

Minimal config template

project:
  name: my-task

model:
  base_model: meta-llama/Llama-3.2-3B
  quant_4bit: true

data:
  source: huggingface
  dataset_name: your-org/your-dataset
  format: prompt_completion
  prompt_field: prompt
  completion_field: completion

hub:
  push_to_hub: false

wandb:
  enabled: false

Inference (after training)

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
import torch

base_model = "meta-llama/Llama-3.2-3B"
adapter_path = "price-prediction-2025-01-01_12.00.00-lite"  # your output_dir

bnb_config = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4")

tokenizer = AutoTokenizer.from_pretrained(base_model)
model = AutoModelForCausalLM.from_pretrained(
    base_model,
    quantization_config=bnb_config,
    device_map="auto",
)
model = PeftModel.from_pretrained(model, adapter_path)

prompt = "Title: Wireless Headphones\nCategory: Electronics\n"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=32)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

CLI Reference

qlora-finetune run CONFIG.yaml          # Start training
qlora-finetune validate-config CONFIG.yaml  # Check config without GPU
qlora-finetune print-config CONFIG.yaml     # Print resolved YAML

Environment Variables

Variable	Required	Description
HF_TOKEN	Yes	Hugging Face token (gated models + hub push)
WANDB_API_KEY	If WandB enabled	Weights & Biases API key

Development

uv sync --extra dev
uv run pytest
uv run ruff check src tests