unsloth-fine-tuning

Unsloth - Fast LLM Fine-Tuning

Fine-tune LLMs 2-5x faster with 50-80% less VRAM. Supports SFT, RL (GRPO), vision, TTS, and 300+ models with zero accuracy loss.

When to Use Unsloth

Use Unsloth when:

• Fine-tuning on a single GPU with LoRA/QLoRA (consumer or datacenter)
• Training reasoning models with GRPO, Dr. GRPO, DAPO, BNPO, or GSPO
• Fine-tuning vision models (Qwen3-VL, Gemma 3, Llama 3.2 Vision)
• Fine-tuning TTS models (Orpheus, Sesame-CSM, Whisper)
• Exporting to GGUF for Ollama, llama.cpp, or LM Studio
• Need padding-free training and uncontaminated packing (automatic)
• Using FP8 precision for additional memory savings on Ampere+ GPUs

Don't use Unsloth when:

• Multi-node distributed training at scale (Unsloth DDP works but is single-node)
• Apple Silicon / MLX (not yet supported)
• Full fine-tuning of 70B+ models (use DeepSpeed + Transformers)
• Custom architectures not supported by transformers
• Cloud-managed training without GPU access (use Tinker instead)

Unsloth vs Alternatives:

Quick Reference

Installation

# Recommended (pip)
pip install unsloth

# With vLLM (for GRPO fast inference)
pip install uv && uv pip install unsloth vllm

# Docker (all dependencies pre-installed)
docker run -d -e JUPYTER_PASSWORD="mypassword" \
  -p 8888:8888 --gpus all -v $(pwd)/work:/workspace/work \
  unsloth/unsloth

Requirements: Linux or Windows (WSL), NVIDIA GPU with CUDA Capability 7.0+ (V100, T4, RTX 20-50, A100, H100, L40). AMD and Intel GPUs also supported. Python 3.10-3.13.

Workflow 1: SFT (Supervised Fine-Tuning)

Use this for standard instruction tuning, chat fine-tuning, or domain adaptation.

Checklist

• Prepare dataset in ShareGPT, ChatML, or Alpaca format
• Choose base vs instruct model (see Model Selection below)
• Select QLoRA (4-bit) or LoRA (16-bit) based on VRAM
• Set hyperparameters (rank, alpha, LR, epochs)
• Run training with SFTTrainer
• Save and deploy (LoRA adapter, merged 16-bit, or GGUF)

Implementation

from unsloth import FastLanguageModel
from trl import SFTTrainer, SFTConfig
from datasets import load_dataset

# Step 1: Load model (QLoRA 4-bit)
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="unsloth/Qwen3-8B-bnb-4bit",  # or any HF model
    max_seq_length=2048,
    load_in_4bit=True,   # False for LoRA 16-bit
)

# Step 2: Add LoRA adapters
model = FastLanguageModel.get_peft_model(
    model,
    r=16,                              # Rank: 8-128 (16-32 recommended)
    lora_alpha=16,                     # Alpha: equal to r or 2*r
    lora_dropout=0,                    # 0 is default, use 0.05-0.1 for regularization
    target_modules=["q_proj", "k_proj", "v_proj", "o_proj",
                    "gate_proj", "up_proj", "down_proj"],
    use_gradient_checkpointing="unsloth",  # 30% less VRAM
    use_rslora=False,                  # True for rank-stabilized LoRA
)

# Step 3: Prepare dataset
dataset = load_dataset("philschmid/dolly-15k-oai-style", split="train")

# Step 4: Train
trainer = SFTTrainer(
    model=model,
    tokenizer=tokenizer,
    train_dataset=dataset,
    args=SFTConfig(
        output_dir="./sft-output",
        per_device_train_batch_size=2,
        gradient_accumulation_steps=4,   # Effective batch = 2*4 = 8
        num_train_epochs=3,
        learning_rate=2e-4,
        fp16=True,                       # or bf16=True
        logging_steps=10,
        optim="adamw_8bit",
        max_seq_length=2048,
        packing=True,                    # Uncontaminated packing (2-5x faster)
    ),
)
trainer.train()

# Step 5: Save
model.save_pretrained("lora_adapter")          # LoRA only (~6MB)
tokenizer.save_pretrained("lora_adapter")

Data Formats

Use get_chat_template(tokenizer, chat_template="chatml") to apply templates. Use standardize_sharegpt(dataset) for ShareGPT-formatted data with non-standard keys.

Training on Completions Only

Mask user inputs so loss is only computed on assistant responses:

from unsloth.chat_templates import train_on_responses_only
trainer = train_on_responses_only(
    trainer,
    instruction_part="<|start_header_id|>user<|end_header_id|>\n\n",      # Llama 3.x
    response_part="<|start_header_id|>assistant<|end_header_id|>\n\n",
)
# For Gemma: instruction_part="<start_of_turn>user\n", response_part="<start_of_turn>model\n"

Sources: docs/datasets.md, docs/chat-templates.md, docs/lora-hyperparameters.md

Workflow 2: RL Training (GRPO)

Use this for training reasoning models with reward functions — math, code, format compliance, verifiable tasks.

Checklist

• Define reward function(s) returning float scores
• Choose model and enable vLLM fast inference
• Enable Unsloth Standby for memory-efficient RL
• Configure GRPOConfig with num_generations, epsilon, loss_type
• Monitor reward curves and KL divergence
• Save and export model

Implementation

import os
os.environ["UNSLOTH_VLLM_STANDBY"] = "1"  # Memory-efficient RL

from unsloth import FastLanguageModel
import torch
import re

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="unsloth/Qwen3-8B",
    max_seq_length=2048,
    load_in_4bit=True,          # False for LoRA 16-bit
    fast_inference=True,         # Enable vLLM for fast generation
    max_lora_rank=32,
    gpu_memory_utilization=0.9,  # Reduce if OOM
)

model = FastLanguageModel.get_peft_model(
    model, r=32, lora_alpha=64,
    target_modules=["q_proj", "k_proj", "v_proj", "o_proj",
                    "gate_proj", "up_proj", "down_proj"],
    use_gradient_checkpointing="unsloth",
)

# Define reward functions
def correctness_reward(completions, answer, **kwargs):
    scores = []
    for completion in completions:
        match = re.search(r"<answer>(.*?)</answer>", completion, re.DOTALL)
        extracted = match.group(1).strip() if match else ""
        scores.append(1.0 if extracted == answer else 0.0)
    return scores

def format_reward(completions, **kwargs):
    pattern = r"<reasoning>.*?</reasoning>\s*<answer>.*?</answer>"
    return [1.0 if re.search(pattern, c, re.DOTALL) else 0.0 for c in completions]

# Train
from trl import GRPOConfig, GRPOTrainer

training_args = GRPOConfig(
    output_dir="./grpo-output",
    per_device_train_batch_size=2,
    gradient_accumulation_steps=4,
    learning_rate=5e-6,
    num_generations=8,          # Rollouts per prompt
    max_completion_length=512,
    max_prompt_length=512,
    max_steps=250,
    temperature=1.0,
    # RL algorithm variants
    loss_type="dapo",           # "grpo", "dr_grpo", "dapo", "bnpo"
    epsilon=0.2,
    epsilon_high=0.28,          # DAPO upper clipping
    scale_rewards="none",       # Dr. GRPO: no reward scaling
    optim="adamw_8bit",
    report_to="none",
)

trainer = GRPOTrainer(
    model=model,
    processing_class=tokenizer,
    args=training_args,
    train_dataset=dataset,
    reward_funcs=[correctness_reward, format_reward],
)
trainer.train()

# Save
model.save_lora("grpo_saved_lora")

RL Algorithm Variants

Unsloth Standby (Memory-Efficient RL)

Set os.environ["UNSLOTH_VLLM_STANDBY"] = "1" before imports. This shares vLLM's weight space with training and repurposes KV cache memory during training — saving up to 60% VRAM. On H100 80GB: 16GB shared weights + 64GB multi-purpose space.

Sources: docs/tutorial-grpo.md, docs/advanced-rl.md, docs/memory-efficient-rl.md

Workflow 3: Vision Fine-Tuning

Use this for training vision-language models on image+text tasks.

Implementation

from unsloth import FastVisionModel
from trl import SFTTrainer, SFTConfig
from unsloth.trainer import UnslothVisionDataCollator

model, tokenizer = FastVisionModel.from_pretrained(
    "unsloth/Qwen2.5-VL-7B-Instruct-bnb-4bit",
    max_seq_length=2048,
    load_in_4bit=True,
)

model = FastVisionModel.get_peft_model(
    model,
    finetune_vision_layers=True,       # Toggle vision encoder training
    finetune_language_layers=True,
    finetune_attention_modules=True,
    finetune_mlp_modules=True,
    r=16, lora_alpha=16,
    target_modules="all-linear",
    use_gradient_checkpointing="unsloth",
)

# Dataset format: user content has text + image
def convert_to_conversation(sample):
    return {"messages": [
        {"role": "user", "content": [
            {"type": "text", "text": "Describe this image."},
            {"type": "image", "image": sample["image"]}]},
        {"role": "assistant", "content": [
            {"type": "text", "text": sample["caption"]}]},
    ]}

dataset = [convert_to_conversation(s) for s in raw_dataset]  # Use list, not .map()

trainer = SFTTrainer(
    model=model, tokenizer=tokenizer,
    data_collator=UnslothVisionDataCollator(model, tokenizer),
    train_dataset=dataset,
    args=SFTConfig(output_dir="./vision-output", max_seq_length=2048,
                   per_device_train_batch_size=1, gradient_accumulation_steps=4),
)
trainer.train()

Vision RL (GRPO with Images)

For VLM RL with vLLM, set fast_inference=True but finetune_vision_layers=False (vLLM limitation). Enable Standby for memory savings.

Supported Vision Models

Sources: docs/vision-fine-tuning.md, docs/vision-rl.md

Workflow 4: TTS Fine-Tuning

Use this for voice cloning, style adaptation, or speech-to-text fine-tuning.

from unsloth import FastModel
from datasets import load_dataset, Audio

model, tokenizer = FastModel.from_pretrained(
    "unsloth/orpheus-3b-0.1-ft",
    max_seq_length=2048,
    load_in_4bit=False,  # 16-bit recommended for TTS
)

dataset = load_dataset("MrDragonFox/Elise", split="train")
dataset = dataset.cast_column("audio", Audio(sampling_rate=24000))  # 24kHz required

Orpheus supports emotional tags: <laugh>, <sigh>, <cough>, <gasp>, <yawn>, etc.

TTS Models

Sources: docs/tts-fine-tuning.md

Workflow 5: Colab Fine-Tuning (Remote GPU)

Use this to run any Unsloth workflow on a Google Colab GPU directly from openscience — no local GPU required.

Setup

1. Generate the bridge notebook: colab_notebook workflow=bridge
2. Upload to Google Colab, select GPU runtime, run all cells
3. Copy the WebSocket URL → colab_connect connection_url="wss://..."

Run Training Remotely

colab_finetune workflow=sft model="unsloth/Qwen3-4B-unsloth-bnb-4bit" dataset="mlabonne/FineTome-100k"

All SFT/GRPO/DPO/vision/TTS workflows work identically on Colab. The plugin handles:

• Unsloth installation on the Colab VM
• Model loading, LoRA setup, dataset preparation
• Training execution with streaming output
• Model saving and optional HuggingFace Hub push

GPU Recommendations

Key Differences from Local Training

• Files are ephemeral — save to HuggingFace Hub with push_to_hub parameter
• Session may disconnect — use keep-alive cell in bridge notebook
• Package installation happens on each new session

See the colab-finetuning skill for detailed Colab-specific guidance.

Model Selection

Instruct vs Base Model

Model Name Conventions

VRAM Requirements

Common OOM fix: reduce per_device_train_batch_size to 1 or 2.

Sources: docs/model-selection.md, docs/requirements.md

Key Hyperparameters

Batch Size Equivalence

Unsloth's gradient accumulation fix makes all configurations equivalent:

Effective Batch Size = per_device_train_batch_size × gradient_accumulation_steps
# batch_size=2, accum=4 ≡ batch_size=1, accum=8 ≡ batch_size=8, accum=1

Sources: docs/lora-hyperparameters.md

Saving and Deployment

Save Methods

# LoRA adapter only (~6MB)
model.save_pretrained("lora_adapter")

# Merged 16-bit (for vLLM deployment)
model.save_pretrained_merged("model_16bit", tokenizer, save_method="merged_16bit")

# GGUF (for Ollama, llama.cpp, LM Studio)
model.save_pretrained_gguf("model_gguf", tokenizer, quantization_method="q4_k_m")

# Push to Hugging Face Hub
model.push_to_hub_merged("username/model", tokenizer, save_method="merged_16bit", token="...")
model.push_to_hub_gguf("username/model", tokenizer, quantization_method="q4_k_m", token="...")

GGUF Quantization Options

Deployment Targets

Inference with Unsloth (2x faster)

from unsloth import FastLanguageModel
model, tokenizer = FastLanguageModel.from_pretrained("lora_adapter", max_seq_length=2048, load_in_4bit=True)
FastLanguageModel.for_inference(model)  # Enable 2x faster inference

inputs = tokenizer("What is machine learning?", return_tensors="pt").to("cuda")
output = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(output[0], skip_special_tokens=True))

Sources: docs/saving-to-gguf.md, docs/saving-to-ollama.md, docs/vllm-guide.md, docs/inference.md

Common Issues

Sources: docs/troubleshooting-faq.md, docs/troubleshooting-inference.md

Best Practices

1. Start with QLoRA 4-bit (load_in_4bit=True) — fits most models on consumer GPUs with minimal accuracy loss
2. Use unsloth-bnb-4bit model variants for higher accuracy than standard 4-bit quants
3. Set use_gradient_checkpointing="unsloth" — 30% less VRAM than standard gradient checkpointing
4. Use target_modules="all-linear" for best quality, or specify attention+MLP modules
5. Start with rank 16-32, increase only if quality is insufficient
6. Set lora_alpha = r or 2*r — higher alpha increases effective learning rate
7. Enable packing (packing=True in SFTConfig) for 2-5x faster training with proper attention masking
8. Use train_on_responses_only to avoid training on user prompts
9. For RL, enable Standby (UNSLOTH_VLLM_STANDBY=1) and fast_inference=True
10. Use DAPO loss (loss_type="dapo") as the default RL algorithm — most stable
11. Always use the same chat template for training and inference to avoid gibberish output
12. Consider FP8 (load_in_fp8=True) on Ampere+ GPUs for 60% less VRAM with ~equal accuracy
13. Split dataset into train/test and enable eval_strategy="steps" for monitoring
14. Save adapters frequently — they're tiny (~6MB) and easy to rollback

References

Core Training

• docs/overview.md — Features and capabilities overview
• docs/datasets.md — Dataset formatting and preparation
• docs/chat-templates.md — Chat template configuration
• docs/lora-hyperparameters.md — LoRA parameter tuning guide
• docs/kernels-packing.md — Custom kernels and packing optimizations

Reinforcement Learning

• docs/tutorial-grpo.md — Step-by-step GRPO tutorial
• docs/advanced-rl.md — Advanced RL parameters and batching
• docs/memory-efficient-rl.md — Standby and memory optimizations
• docs/fp8-rl.md — FP8 precision RL training
• docs/fp16-vs-bf16.md — Precision comparison for RL stability
• docs/reward-hacking.md — Counter-measures for reward hacking

Specialized Models

• docs/vision-fine-tuning.md — Vision-language model training
• docs/vision-rl.md — VLM RL with GRPO/GSPO
• docs/tts-fine-tuning.md — Text-to-speech fine-tuning
• docs/tool-calling.md — Tool/function calling training

Deployment & Inference

• docs/saving-to-gguf.md — GGUF export and quantization
• docs/saving-to-ollama.md — Ollama integration
• docs/vllm-guide.md — vLLM deployment
• docs/inference.md — 2x faster native inference
• docs/lora-hot-swapping.md — Runtime adapter switching

Infrastructure

• docs/requirements.md — System and VRAM requirements
• docs/model-selection.md — Choosing the right model
• docs/models.md — Full model catalog (300+ models)
• docs/multi-gpu-ddp.md — Multi-GPU distributed training
• docs/installation-pip.md — pip installation guide
• docs/installation-docker.md — Docker setup

Known Conflicts

• Do not install alongside flash-attention in the same environment. Unsloth bundles xformers which may conflict with flash-attn on attention kernels. Use separate environments.

Resources

• GitHub: https://github.com/unslothai/unsloth
• Documentation: https://docs.unsloth.ai
• Hugging Face: https://huggingface.co/unsloth
• Docker Hub: https://hub.docker.com/r/unsloth/unsloth
• Notebooks: https://github.com/unslothai/notebooks
• Discord: https://discord.gg/unsloth