15 Oct 2026
Nemotron 3.5 Super VL 120B-A12B NVFP4
Follow the mixed-precision recipe all the way to runtime state
120B reported total · text + image + video
Nemotron / Mamba-2 / attention / latent MoE
A 24 GB checkpoint still needs headroom
New to model sizes or GPU memory? Start with weights, parameters and quantization →
A useful candidate when local multimodal deployment needs stronger compression. Treat its packed representation as a kernel requirement as well as a storage saving.
Publisher-rounded multimodal count. The 88 language blocks include 40 Mamba, 40 MoE and eight attention blocks. Routed expert counts vary from 240 to 496 by block.
Evaluate it for
Local vision-language agent evaluation with an explicitly supported Bolt runtime.
Choose another path when
Only 24 GB of total available memory is assumed sufficient, or the runtime cannot execute this mixed packing.
Release context
Compression changes per-block expert counts, and the quantization config mixes formats. The global 512-expert default does not describe each block in this exported architecture.
The publisher card explicitly dates the Hugging Face release.
Deployment starting points
Start with an exact artifact and an engine that supports it. A configuration below is evidence of a documented path; its device count does not promise a particular throughput or concurrency.
Only 24 GB of total available memory is assumed sufficient, or the runtime cannot execute this mixed packing.
The pinned model card establishes the architecture. It does not, by itself, prove that a particular GPU count runs this checkpoint at your target context. The capacity screen below helps rule out undersized allocations; a successful load and workload test are still required.
This is the index’s reported tensor payload, in decimal GB. It excludes file headers and runtime memory. Mixed precision, conversion and host/device placement determine how much GPU memory the loaded model needs.
Inspect the exact index metadata →This estimates inference memory for a hypothetical uniform precision. It is useful for rejecting an allocation that is too small. It does not establish a working deployment or estimate training memory.
A uniform-precision GPU count would hide this model’s component layout. Publisher-rounded multimodal count. The 88 language blocks include 40 Mamba, 40 MoE and eight attention blocks. Routed expert counts vary from 240 to 496 by block. Use the documented artifact and placement path above; no GPU count is inferred here.
Architecture in practice
Hybrid attention does not mean zero request state.
A full-attention formula cannot describe every layer.
Mechanism schematic based on the pinned model card and configuration. It explains a design principle; it is not a full implementation graph.
| Exact checkpoint | nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB |
|---|---|
| Text model type | nemotron_h_puzzle |
| Layers | 88 |
| Hidden width | 4,096 |
| Attention / KV heads | 32 / 2 |
| Head dimension | 128 |
| Routed / selected experts | Not verified / 9 |
| Configured positions | 262,144 |
| Documented extension | Not recorded for this checkpoint |
| Inputs → output | text, image, video → text |
| License metadata | OpenMDW-1.1 · custom terms |
40 Mamba layers; 40 MoE layers; 8 attention layers. These counts describe the configured pattern.
Open weights do not imply unrestricted use. Read the applicable license terms linked from the pinned card.
A useful comparison
Super VL is the less-compressed parent comparison; the Liquid alternative is text-only. Hold modality and task requirements constant when measuring cost.
15 Oct 2026
Follow the mixed-precision recipe all the way to runtime state
120B reported total · text + image + video
13 Oct 2026
Active parameters are a compute clue, not a device budget
24B reported total · text
These are editorial comparison candidates. We have not run a matched quality or serving benchmark, so this is not a ranking.
Prepare to explain it
Read block_configs before reporting one expert count. Then separate the target artifact, optional drafter and per-request state.
Attention layouts · Quantization · Courses and worked examples
For a deployment evaluation, record exact weights, precision, engine, device count, interconnect, prompt/output lengths and concurrency. Report task success, errors, TTFT, TPOT and useful throughput together.
Our assessment is editorial judgment based on the linked architecture and deployment evidence. Model facts are publisher-reported or attributed to runtime maintainers; no independent GPU benchmark was run.
AI Infra Interviews, “Nemotron 3.5 Bolt 74B-A9B 24GB: release, architecture and deployment”, checked 2026-11-16. Preserve this date and the exact checkpoint when citing.