Hands-on AI model guides
Train a small language model (SLM) on your CPU, fine-tune with Unsloth, or host a model with vLLM. Follow projects for streaming inference, serving performance and private networking. Choose a path, follow its commands and check the result at each stage.
Choose your learning path
Build a tiny language model
A CPU foundation project. Learn the training loop and keep your own resumable checkpoint. This checkpoint is separate from the pretrained Qwen models below.
- Train, save and resumeFree to read
- Start with
- Python and the included synthetic text
- Keep for the next step
- tiny.pt and generated text
Fine-tune and serve with Unsloth
Follow the incident-label task from grouped data to a served adapter. Optional candidate selection needs a fresh holdout for quality claims; export a selected adapter as qwen-adapter/ before serving. If the base wins, keep it as the baseline.
- 1. Prepare the datasetFree to read
- Start with
- Grouped source incidents
- Keep for the next step
- prepared/ and manifest.json
- 2. Train and reloadFree account
- Start with
- prepared/ and Qwen3-0.6B
- Keep for the next step
- qwen-adapter/ and local predictions
- Start with
- The preserved dataset and two independent adapter runs
- Keep for the next step
- selection.json, test report and selected-adapter export
- 3. Serve your adapterFree account
- Start with
- qwen-adapter/ from training or the selected export
- Keep for the next step
- incident-labeler on localhost:8001
- 4. Check the servicePremium
- Start with
- That endpoint and the same task cases
- Keep for the next step
- Per-request evidence and a release check
Deploy on a named GPU
Start with an existing model, without training. This path uses Qwen3.5-4B on an NVIDIA L4 and its own request set.
- 1. Deploy Qwen3.5-4BFree to read
- Start with
- L4 host, Docker and pinned model files
- Keep for the next step
- hands-on-qwen on localhost:8000
- 2. Check the servicePremium
- Start with
- That endpoint and reviewed task cases
- Keep for the next step
- Per-request evidence and a release check
Operate and tune an inference service
Begin with CPU streaming and HTTPS fixtures, then connect the client to the named-GPU deployment for a controlled serving experiment. Fixture results verify the tools; actual model performance requires the GPU runs.
- 1. Build a streaming clientFree to read
- Start with
- Python and the synthetic HTTP fixture
- Keep for the next step
- A completion-aware client and request evidence
- 2. Configure private accessFree to read
- Start with
- A model endpoint or the separate local proxy fixture
- Keep for the next step
- Verified HTTPS, a route allowlist and streaming checks
- 3. Compare serving settingsPremium
- Start with
- The streaming client, frozen cases and a working GPU service
- Keep for the next step
- Paired experiment records and acceptable-answer throughput
Build and defend an inference service
Premium coursework with downloadable scripts and evidence to submit. Start with device observations, verify request telemetry, compare a serving setting, measure speculative decoding, then test routing and defend a benchmark. Tracing and speculation use separate experiments on a pinned small model; the optional kernel project uses a separate workload and environment. Bring your own hardware; CPU fixtures do not measure GPU speed.
- Start with
- A Linux GPU host and request records
- Keep for the next step
- Device samples and an incident worksheet
- Start with
- The shared streaming client and the project’s pinned local GPU stack
- Keep for the next step
- Connected traces, checked metrics and an outage acceptance record
- Start with
- The streaming client, fixed cases and a compatible GPU
- Keep for the next step
- Three matched A/B blocks
- Start with
- The pinned small-model engine and frozen copy/transform cases
- Keep for the next step
- Paired GPU journals, acceptance counters and correctness checks
- 3. Test prefix routingPremium
- Start with
- CPU HTTP fixtures; two GPU replicas for cache measurements
- Keep for the next step
- A routing policy and explicit residency limits
- Start with
- A separate CUDA environment and RMSNorm inputs
- Keep for the next step
- Numerical checks and four-baseline GPU timings
- 4. Defend the benchmarkPremium
- Start with
- Matched serving records and declared cost assumptions
- Keep for the next step
- A quality-gated comparison and decision report
Prove a GPU kernel change
Begin with CPU models for work mapping and output ownership, then qualify an actual Triton kernel against eager and compiled baselines. CPU predictions and GPU measurements remain separate evidence.
- 1. Model work and ownershipPremium
- Start with
- Python and synthetic shapes/counts
- Keep for the next step
- Tested offsets, wave plans and mapping-evidence.json
- Start with
- A compatible NVIDIA GPU and the project’s separate environment
- Keep for the next step
- Correctness checks, raw timings, PTX and native artifacts
20 guides · Use a path above or open a standalone project below. Reading access and compute requirements are listed separately; cloud GPU charges are not included.
Train a tiny language model from scratch with PyTorch
Build a 470,528-parameter causal language model on a CPU. Train it, inspect validation loss, save a checkpoint and resume the same run.
You finish with: A trained byte-level model, a resumable checkpoint and sampled text.
CPU · Python 3.12 · no GPU required
60–90 minutes of guided workVerification: CPU workflow executed.
Prepare a JSONL dataset for small-model fine-tuning
Validate conversation records, reject conflicting duplicates, split source groups and save a data manifest before spending GPU time.
You finish with: Train, validation and test JSONL files with disjoint source groups and recorded hashes.
CPU · Python 3.12 · standard library only
45–60 minutes of guided workVerification: CPU workflow executed.
Fine-tune and serve Qwen3-0.6B with Unsloth
Train a QLoRA adapter, compare held-out labels, reload the saved files and serve the same Qwen3 adapter through a local vLLM API.
You finish with: A saved adapter, base-versus-adapter predictions and a local API comparison using that adapter.
Linux · Python 3.12 · NVIDIA GPU; serving path targets an L4 (24 GB)
2–3 hours of guided work plus setup and GPU runsVerification: GPU run pending.
Deploy Qwen3.5-4B with vLLM on an NVIDIA L4
Start a text-only Qwen3.5 API with pinned model files, a recorded container digest, bounded context, local access and a checked response.
You finish with: A locally bound chat endpoint and a release record you can reproduce.
Linux x86-64 · one NVIDIA L4 (24 GB) · Docker
60–90 minutes plus downloadsVerification: GPU run pending.
Benchmark a vLLM service and build a release gate
Run repeatable HTTP checks, retain failed requests, measure end-to-end latency and require correct, complete answers within a declared deadline.
You finish with: Per-request JSON evidence and a release command that fails when the declared threshold is missed.
Python 3.12 client · a running local vLLM endpoint
60–90 minutes plus measured runsVerification: Client tested · GPU run pending.
Stream vLLM responses with Python and detect incomplete answers
Build a text-stream client, measure first visible output, preserve partial failures and test SSE parsing before connecting to a model.
You finish with: A bounded streaming client, executable HTTP fixtures and per-request JSON evidence.
CPU · Python 3.12 · optional existing model endpoint
90–120 minutes, plus an optional model runVerification: HTTP fixtures and small-model streaming executed.
Secure a model endpoint with SSH tunnels and HTTPS
Configure private model access, verified TLS and an Nginx streaming proxy. Test authentication, route limits, buffering and upstream failures.
You finish with: An isolated HTTPS proxy, a tested request path and a private SSH forwarding procedure.
CPU · Linux · Python 3.12 · Nginx with TLS · OpenSSL
2–3 hours for the local lab; remote setup variesVerification: Local TLS and proxy executed.
Tune vLLM serving parameters with controlled experiments
Compare token budgets or sequence limits with matched workloads, all-attempt goodput and repeated paired runs. Includes a small-model RTX 5070 branch.
You finish with: Recorded serving configurations, raw request evidence and a comparison that rejects mismatched experiments.
Python 3.12 client · an existing single-GPU vLLM host
2–4 hours plus measured GPU runsVerification: Small-model GPU branch executed · L4 branch pending.
Evaluate fine-tuned adapters and freeze a model selection
Compare Unsloth training budgets against the base model, inspect paired regressions, bind predictions to artifacts and keep test data outside selection.
You finish with: A frozen validation choice, recorded prediction identities and a separate held-out test report.
CPU evaluation · existing Unsloth NVIDIA environment for model runs
2–3 hours plus two training and prediction runsVerification: CPU selection checks executed · GPU run pending.
Train your own decision model with Unsloth and Qwen3.5
Turn Qwen3.5-0.8B into a decision model that returns calibrated probabilities instead of text. GPU-executed on an RTX 5070: 36% to 69% on 2,000 held-out decisions in 4.3 minutes.
You finish with: A saved decision model (LoRA adapters, joint schema head, calibration temperatures) with a reload check and per-workflow scores.
One NVIDIA GPU, 8 GB or more · tested on RTX 5070 12 GB · Python 3.12
45–60 minutes, including a 5-minute training runVerification: GPU workflow executed.
Observe a GPU service and distinguish missing signals from healthy hardware
Capture real device samples, expose local metrics and correlate hardware activity with request failures. Build a dashboard that preserves missing data.
You finish with: Device evidence, a local metrics exporter, dashboard configuration and an incident worksheet.
Linux · NVIDIA GPU for device capture · CPU fixture checks
90–120 minutes plus service measurementsVerification: Device capture executed · dashboard deployment pending.
Build a prefix-aware routing proxy and test its failure paths
Compare round-robin, prefix affinity and a load-aware policy. Stream responses through a bounded local proxy and account for busy or failed replicas.
You finish with: A tested local routing proxy, policy experiment records and a cache-residency limitation analysis.
CPU · Python 3.12 · two compatible model replicas for cache measurements
2–4 hours plus optional two-replica GPU runsVerification: HTTP routing executed · GPU cache experiment pending.
Write a fused RMSNorm kernel in Triton and benchmark fair baselines
Implement masked RMSNorm, check three dtypes and compare eager PyTorch, torch.compile, native RMSNorm and a custom GPU kernel.
You finish with: A forward-only kernel, numerical checks, repeated timing records and an optional profiler trace.
Linux · CUDA NVIDIA GPU · executed on RTX 5070 12 GB
2–4 hours plus CUDA environment setupVerification: GPU kernel and four baselines executed.
Defend an inference benchmark with quality gates and cost evidence
Turn paired request records into an auditable cost, latency and useful-throughput comparison. Reject confounded runs and candidates that miss the quality rule.
You finish with: A reproducible comparison, an eligible Pareto frontier and a written decision with a reversal condition.
CPU · Python 3.12 · real serving records required for performance conclusions
2–3 hours after collecting paired runsVerification: Report checks executed · local GPU evidence available.
Trace inference requests with OpenTelemetry, Jaeger and Grafana
Build a local vLLM observability lab, follow native engine spans, verify request metrics, and test sampling and collector outages against an independent journal.
You finish with: A checked trace and metrics pipeline, four request journals, and an acceptance record that distinguishes missing telemetry from failed inference.
Linux · Python 3.12 · NVIDIA GPU · Docker Compose
2–3 hours plus environment setupVerification: Local GPU and monitoring stack executed.
Speculative decoding: measure when it helps
Benchmark vLLM prompt-lookup speculation against ordinary decoding. Check acceptance, output correctness, latency and goodput with matched GPU runs.
You finish with: Paired GPU request records and a decision report that separates accepted drafts, complete output and correct answers.
Linux · Python 3.12 · One NVIDIA GPU
2–3 hours plus environment setupVerification: Paired GPU experiment executed.
Model GPU work mapping and parallel scans
Run CPU models for GEMM scheduling, tensor offsets, prefix scans and softmax state merging. Inspect intermediate values and test the assumptions before writing a GPU kernel.
You finish with: Executable work-mapping models, a synthetic evidence file and tested ownership calculations.
Python 3.10+ · CPU only
60–90 minutesVerification: 13 CPU tests executed.
Qualify and benchmark a Triton kernel
Test an FP32 residual-plus-ReLU kernel against adversarial cases, eager PyTorch and a compiled baseline. Retain GPU timings, source hashes, PTX and native artifacts.
You finish with: A correctness-qualified experiment with raw matched timings, compiled artifacts and a scoped implementation decision.
Linux · Python 3.12 · NVIDIA GPU
2–3 hours plus environment setupVerification: GPU qualification and timing executed.
Run Muse Spark Open locally with llama.cpp: hardware, quants and setup
Run Meta's 816B Muse Spark Open on one machine. Pick a GGUF quant from real file sizes, start llama.cpp, set reasoning effort and verify the local API.
You finish with: A local OpenAI-compatible Muse Spark Open endpoint, a quant chosen from your memory budget and a checked streamed reply.
About 230 GiB+ of RAM plus VRAM, or Apple unified memory · llama.cpp or Unsloth · planner runs on any CPU
30–45 minutes of setup plus a 176–506 GB downloadVerification: Planner and client CPU-executed; model run pending.
Post-train an LLM from scratch in PyTorch: SFT, DPO, PPO and GRPO
Implement the full LLM post-training pipeline in pure PyTorch on a CPU. Pretrain a tiny Transformer, then compare SFT, a reward model, DPO, PPO and GRPO with a verifiable reward.
You finish with: Six checkpoints with lineage, per-stage metrics and a five-seed comparison of SFT, DPO, PPO and GRPO.
CPU · Python 3.12 · PyTorch 2.6.0 · no GPU required
60–90 minutes of guided work; every stage runs in about 15 secondsVerification: CPU workflow executed across five seeds.