Runtime recipe
1 × H100 80 GB SXM
80 GB per GPU
FP8 weights
vLLM ≥0.17 model guide
Single-GPU recipe; start with its context and scheduler settings. FP8 weights and cache dtype are separate choices.
Qwen / MoE
Qwen3.5-35B-A3B
The easiest MoE memory lesson to make concrete
New to model sizes or GPU memory? Start with weights, parameters and quantization →
This is a useful entry point for serving a multimodal MoE: 35B total, 3B active, and documented single-GPU FP8 options. It makes a good first experiment before the 122B and 397B variants.
35Breported total parameters
3Bactive parameters per token
262,144configured token positions
Publisher-reported, rounded model count; not an exact tensor census. Multimodal components and packaging can change the artifact size.
Evaluate it for
Learning MoE deployment and testing a comparatively modest multimodal service.
Choose another path when
Copying the hosted Flash service’s limits into the open checkpoint or assuming every quantization has equal runtime support.
Release context
Why this release matters
The family combines gated delta networks with full-attention layers. Both the expert store and the two types of attention state matter when you raise context or concurrency.
Deployment starting points
Which hardware, how many, at what precision?
Start with an exact artifact and an engine that supports it. A configuration below is evidence of a documented path; its device count does not promise a particular throughput or concurrency.
Runtime recipe
2 × H100 80 GB SXM
80 GB per GPU
BF16
vLLM ≥0.17 model guide
Two H100 GPUs are listed for BF16. Tensor parallelism distributes weights and adds inter-device communication.
71.9 GB of tensor data in the pinned index
This is the index’s reported tensor payload, in decimal GB. It excludes file headers and runtime memory. Mixed precision, conversion and host/device placement determine how much GPU memory the loaded model needs.
Inspect the exact index metadata →Capacity screen: would the weights fit?
This estimates inference memory for a hypothetical uniform precision. It is useful for rejecting an allocation that is too small. It does not establish a working deployment or estimate training memory.
| Representation | Approximate weight floor | What remains to budget |
|---|---|---|
| 16-bit uniform | 70 GB | Cache, modality state, workspace, quantization metadata and uneven sharding |
| 8-bit uniform | 35 GB | Cache, modality state, workspace, quantization metadata and uneven sharding |
| 4-bit uniform | 17.5 GB | Cache, modality state, workspace, quantization metadata and uneven sharding |
Enter the total across the whole model replica, not per GPU: KV or recurrent cache for all concurrent requests, image/audio state and execution workspace. There is no universal default. We additionally leave 15% of each GPU unused.
70 GB for hypothetical weights
Enter a request-and-buffer budget to calculate the aggregate capacity requirement.
A count of one does not prove the engine supports this GPU. A count above one does not prove the model shards evenly or communicates efficiently. Publisher-reported, rounded model count; not an exact tensor census. Multimodal components and packaging can change the artifact size.
Architecture in practice
What the serving engine has to do
Two kinds of layers retain two kinds of state.
Hybrid attention does not mean zero request state.
A full-attention formula cannot describe every layer.
Mechanism schematic based on the pinned model card and configuration. It explains a design principle; it is not a full implementation graph.
Inspect the full checkpoint specifications
| Exact checkpoint | Qwen/Qwen3.5-35B-A3B |
|---|---|
| Text model type | qwen3_5_moe_text |
| Layers | 40 |
| Hidden width | 2,048 |
| Attention / KV heads | 16 / 2 |
| Head dimension | 256 |
| Routed / selected experts | 256 / 8 |
| Configured positions | 262,144 |
| Documented extension | 1,010,000 tokens; additional configuration and evaluation required |
| Inputs → output | text, image, video → text |
| License metadata | apache-2.0 |
30 linear attention layers; 10 full attention layers. These counts describe the configured pattern.
Open weights do not imply unrestricted use. Read the applicable license terms linked from the pinned card.
A useful comparison
What else belongs on the shortlist?
The dense 27B exposes the sparsity tradeoff; 3.6 provides the direct upgrade comparison.
24 Feb 2026
Qwen3.5-27B
Dense compute alongside a sparse sibling
27B reported total · text + image + video
16 Apr 2026
Qwen3.6-35B-A3B
A smaller expert store for coding-agent experiments
35B reported total · text + image + video
These are editorial comparison candidates. We have not run a matched quality or serving benchmark, so this is not a ranking.
Prepare to explain it
The interview lesson
Walk through 70 GB BF16 versus 35 GB uniform-eight-bit arithmetic, then compare that screen with the actual FP8 recipe.
Attention layouts · Quantization · Courses and worked examples
For a deployment evaluation, record exact weights, precision, engine, device count, interconnect, prompt/output lengths and concurrency. Report task success, errors, TTFT, TPOT and useful throughput together.
Sources and citation
Our assessment is editorial judgment based on the linked architecture and deployment evidence. Model facts are publisher-reported or attributed to runtime maintainers; no independent GPU benchmark was run.
AI Infra Interviews, “Qwen3.5-35B-A3B: release, architecture and deployment”, checked 2026-09-13. Preserve this date and the exact checkpoint when citing.