AI Infra Interviews logo

Fine-tune and serve Qwen3-0.6B with Unsloth

Train a QLoRA adapter, compare held-out labels, reload the saved files and serve the same Qwen3 adapter through a local vLLM API.

Free accountGuided GPU project

AI Infra Interviews editorial · Updated

How does QLoRA fine-tuning work with Unsloth?

QLoRA keeps the quantized base model frozen and updates small added LoRA matrices. This guide applies it to a pinned Qwen3-0.6B checkpoint, supervises completion tokens, and carries the saved adapter into a local vLLM service using the same base revision. Dataset and artifact checks have CPU evidence. CUDA training, adapter serving and GPU memory fit still require a hardware run.

Read the primary sources
What you build
A saved adapter, base-versus-adapter predictions and a local API comparison using that adapter.
Environment
Linux · Python 3.12 · NVIDIA GPU; serving path targets an L4 (24 GB)
Plan your session
2–3 hours of guided work plus setup and GPU runs. A planning estimate, not measured runtime.
Before you begin
Complete the dataset guide; keep its prepared directory. Have a CUDA-capable NVIDIA driver and a separate Python environment.
GPU run pending. Tokenizer and completion-loss masks executed on CPU. Dataset/resume identity, adapter hash checks and HTTP comparison client exercised with local fixtures. CUDA training, adapter loading in vLLM and L4 memory fit remain unexecuted.
  1. 01Frozen base

    Load the original checkpoint in 4-bit form. Keep its base weights fixed.

  2. 02Trainable adapter

    Gradients update the added LoRA matrices. Completion labels choose which tokens contribute loss.

  3. 03Same adapter over HTTP

    Reload the saved adapter, then serve it with the same base revision. Inspect answer changes when the runtime changes.

Follow the changing state through the project. Each boundary has a check in the walkthrough.

Continue with a free account

Sign in to read the complete walkthrough and download its training scripts. The technical overview, prerequisites and verification notes above are public.

Sign in free

Start with Prepare a JSONL dataset for small-model fine-tuning (Free to read).

Primary sources

Checked . Source review and execution checks are described separately above.