A multimodal agent candidate where a documented NVIDIA deployment is useful. The pinned recipe provides a starting configuration; it is not our throughput measurement.
Publisher-reported rounded multimodal count. The 88 configured language blocks are 40 Mamba, 40 MoE and eight attention blocks; vision and MTP are separate.
Evaluate it for
Text, document-image and sampled-video evaluation on supported NVIDIA hardware.
Choose another path when
A generic backend without the required Mamba or quantization support is mandatory.
Release context
Why this release matters
The configuration separates forty Mamba blocks, forty expert blocks and eight attention blocks. Mamba cache precision and restoration are part of the serving contract, not properties inferred from the weight dtype.
The publisher card explicitly dates the Hugging Face release.
Start with an exact artifact and an engine that supports it. A configuration below is evidence of a documented path; its device count does not promise a particular throughput or concurrency.
Runtime recipe
1 × H200 SXM
141 GB per GPU
Mixed NVFP4; non-Blackwell activation path
vLLM, publisher model-specific setup
Publisher adapts the two-H100 command to one H200 with TP1. Maximum 128 sequences, three-token MTP and float16 Mamba state with stochastic rounding; up to 1M context does not mean 128 simultaneous 1M requests.
This is the index’s reported tensor payload, in decimal GB. It excludes file headers and runtime memory. Mixed precision, conversion and host/device placement determine how much GPU memory the loaded model needs.
This estimates inference memory for a hypothetical uniform precision. It is useful for rejecting an allocation that is too small. It does not establish a working deployment or estimate training memory.
A uniform-precision GPU count would hide this model’s component layout. Publisher-reported rounded multimodal count. The 88 configured language blocks are 40 Mamba, 40 MoE and eight attention blocks; vision and MTP are separate. Use the documented artifact and placement path above; no GPU count is inferred here.
Architecture in practice
What the serving engine has to do
Two kinds of layers retain two kinds of state.
Hybrid attention does not mean zero request state.
A full-attention formula cannot describe every layer.
Schematic of a hybrid attention stack, not a literal layer ordering. Recurrent layers update a compact state; attention layers retain their own history. Both kinds of state matter for prefix reuse and concurrent requests.
Mechanism schematic based on the pinned model card and configuration. It explains a design principle; it is not a full implementation graph.
Inspect the full checkpoint specifications
Inspected fields for this exact revision. KV heads alone do not describe latent, hybrid or shared-cache layouts.
For a deployment evaluation, record exact weights, precision, engine, device count, interconnect, prompt/output lengths and concurrency. Report task success, errors, TTFT, TPOT and useful throughput together.
Sources and citation
Our assessment is editorial judgment based on the linked architecture and deployment evidence. Model facts are publisher-reported or attributed to runtime maintainers; no independent GPU benchmark was run.
AI Infra Interviews, “Nemotron 3.5 Super VL 120B-A12B NVFP4: release, architecture and deployment”, checked 2026-11-16. Preserve this date and the exact checkpoint when citing.