AI Infra Interviews logo

Fine-tune and serve Qwen3-0.6B with Unsloth

Train a QLoRA adapter, compare held-out labels, reload the saved files and serve the same Qwen3 adapter through a local vLLM API.

Free accountGuided GPU project

AI Infra Interviews editorial · Updated 2026-09-20

What you build
A saved adapter, base-versus-adapter predictions and a local API comparison using that adapter.
Environment
Linux · Python 3.12 · NVIDIA GPU; serving path targets an L4 (24 GB)
Plan your session
2–3 hours of guided work plus setup and GPU runs. A planning estimate, not measured runtime.
Before you begin
Complete the dataset guide; keep its prepared directory. Have a CUDA-capable NVIDIA driver and a separate Python environment.
GPU run pending. Tokenizer and completion-loss masks executed on CPU. Dataset/resume identity, adapter hash checks and HTTP comparison client exercised with local fixtures. CUDA training, adapter loading in vLLM and L4 memory fit remain unexecuted.
  1. 01Frozen base

    Load the original checkpoint in 4-bit form. Keep its base weights fixed.

  2. 02Trainable adapter

    Gradients update the added LoRA matrices. Completion labels choose which tokens contribute loss.

  3. 03Same adapter over HTTP

    Reload the saved adapter, then serve it with the same base revision. Inspect answer changes when the runtime changes.

Follow the changing state through the project. Each boundary has a check in the walkthrough.

Continue with a free account

Sign in to read the complete walkthrough and download its training scripts. The prerequisites, project outline and verification notes above are public.

Sign in free

Start now with the free CPU training project.

Primary sources

Checked 2026-09-20. Source review and execution checks are described separately above.