Runtime recipe
2 × GB300 GPUs
GPUs within an NVLink system; this is not a count of whole racks.
FP8 weights
vLLM Flash-Next guide
Minimum validated FP8 configuration; four GPUs are the recommended full-tray layout.
Qwen / MoE
Qwen3.8-Flash-Next
The memory placement is part of the architecture
New to model sizes or GPU memory? Start with weights, parameters and quantization →
Flash-Next is compelling when you can exploit its small active backbone without pretending the rest of the model disappears. Its n-gram tables make host memory and transfer behavior part of the deployment design.
180Breported total parameters
6Bactive parameters per token
262,144configured token positions
180B sum: 125B backbone + 51B n-gram embeddings + 4B MTP. The 6B active count applies to the backbone.
Evaluate it for
Multimodal services that can test GPU versus host placement and newer hybrid kernels.
Choose another path when
A runtime that treats every tensor as an ordinary all-GPU transformer weight or lacks the required offload path.
Release context
Why this release matters
The reported package separates a 125B backbone, 51B n-gram tables and 4B MTP component. Moving the tables off the accelerator is a deliberate placement strategy; it is not ordinary KV-cache offload.
Date corroborated by the FlagOS team’s account of its same-day integration; architecture is sourced to Qwen’s pinned card.
Deployment starting points
Which hardware, how many, at what precision?
Start with an exact artifact and an engine that supports it. A configuration below is evidence of a documented path; its device count does not promise a particular throughput or concurrency.
Runtime recipe
4 × GB300 GPUs
GPUs within an NVLink system; this is not a count of whole racks.
FP8 weights
vLLM Flash-Next guide
Recommended full-tray FP8 configuration. Keep scheduler and memory-utilization limits from the guide.
Runtime recipe
8 × H200 SXM
141 GB per GPU
FP8 weights
vLLM Flash-Next guide
Use tensor/expert parallelism TEP8 and the Triton MoE backend on Hopper.
Runtime recipe
4 × H100 80 GB SXM
80 GB per GPU
FP8 weights
vLLM Flash-Next guide
Requires n-gram/PLE tables in host memory. Reserve at least 51 GB host RAM plus buffers; follow the model-specific offload path.
Runtime recipe
4 × MI355X
288 GB per GPU
FP8 weights
vLLM Flash-Next guide
Use the documented ROCm/AITER configuration. Support depends on the model-specific kernels.
360 GB of tensor data in the pinned index
This is the index’s reported tensor payload, in decimal GB. It excludes file headers and runtime memory. Mixed precision, conversion and host/device placement determine how much GPU memory the loaded model needs.
Inspect the exact index metadata →Capacity screen: would the weights fit?
This estimates inference memory for a hypothetical uniform precision. It is useful for rejecting an allocation that is too small. It does not establish a working deployment or estimate training memory.
A uniform-precision GPU count would hide this model’s component layout. 180B sum: 125B backbone + 51B n-gram embeddings + 4B MTP. The 6B active count applies to the backbone. Use the documented artifact and placement path above; no GPU count is inferred here.
Architecture in practice
What the serving engine has to do
Shared tables and request cache have different lifetimes.
Model tables do not multiply with every request.
The lookup and host/device placement are model-specific.
Mechanism schematic based on the pinned model card and configuration. It explains a design principle; it is not a full implementation graph.
Inspect the full checkpoint specifications
| Exact checkpoint | Qwen/Qwen3.8-Flash-Next |
|---|---|
| Text model type | qwen4_exp_text |
| Layers | 48 |
| Hidden width | 2,560 |
| Attention / KV heads | 24 / 2 |
| Head dimension | 256 |
| Routed / selected experts | 512 / 10 |
| Configured positions | 262,144 |
| Documented extension | 1,000,000 tokens; additional configuration and evaluation required |
| Inputs → output | text, image, video → text |
| License metadata | qwen-community-1.0 · custom terms |
36 linear attention layers; 12 full attention layers. These counts describe the configured pattern.
Open weights do not imply unrestricted use. Read the applicable license terms linked from the pinned card.
A useful comparison
What else belongs on the shortlist?
The 27B model is the simpler deployment baseline; DeepSeek V4.1 offers a different conditional-memory design.
14 Aug 2026
Qwen3.8-27B
A current dense baseline before you scale out
27B reported total · text + image + video
10 Sept 2026
DeepSeek-V4.1-Flash
The September model that changes the deployment diagram
552B reported total · text + image
These are editorial comparison candidates. We have not run a matched quality or serving benchmark, so this is not a ranking.
Prepare to explain it
The interview lesson
Draw separate boxes for expert weights, n-gram tables and request state. Explain which grows with concurrency and which is shared across requests.
Attention layouts · Quantization · Courses and worked examples
For a deployment evaluation, record exact weights, precision, engine, device count, interconnect, prompt/output lengths and concurrency. Report task success, errors, TTFT, TPOT and useful throughput together.
Sources and citation
Our assessment is editorial judgment based on the linked architecture and deployment evidence. Model facts are publisher-reported or attributed to runtime maintainers; no independent GPU benchmark was run.
AI Infra Interviews, “Qwen3.8-Flash-Next: release, architecture and deployment”, checked 2026-09-13. Preserve this date and the exact checkpoint when citing.