AI Infra Interviews logo

Nemotron / Lab profile

NVIDIA

Compression and runtime support must travel together.

The covered October models combine vision inputs, a Mamba-attention backbone and sparse experts. Super VL provides a mixed-NVFP4 serving path; Bolt further compresses the architecture and packed weights.

Architecture trajectory

What connects these models?

Mamba state, attention KV and visual preprocessing are different allocations. Bolt also varies expert counts by block, so inspect nested configuration instead of copying the global default.

Two kinds of layers retain two kinds of state.

A token enters a layerthe layer type selects itspathRecurrent layerupdate a compact stateAttention layerread retained tokenhistoryRecurrent statemaintained for thissequenceAttention cachelayout depends on themodel

Hybrid attention does not mean zero request state.

A full-attention formula cannot describe every layer.

Schematic of a hybrid attention stack, not a literal layer ordering. Recurrent layers update a compact state; attention layers retain their own history. Both kinds of state matter for prefix reuse and concurrent requests.

Newest first / grouped sequence, not a time scale

The Nemotron timeline

  1. Nemotron 3.5 Bolt 74B-A9B 24GB

    A 24 GB checkpoint still needs headroom

    Compression changes per-block expert counts, and the quantization config mixes formats. The global 512-expert default does not describe each block in this exported architecture.

    The publisher card explicitly dates the Hugging Face release.

    Release evidence ↗

  2. Nemotron 3.5 Super VL 120B-A12B NVFP4

    Follow the mixed-precision recipe all the way to runtime state

    The configuration separates forty Mamba blocks, forty expert blocks and eight attention blocks. Mamba cache precision and restoration are part of the serving contract, not properties inferred from the weight dtype.

    The publisher card explicitly dates the Hugging Face release.

    Release evidence ↗

Which other labs should you compare?

Compare parent and compressed checkpoints on identical visual tasks, output budgets and concurrency. A smaller artifact is only useful if the quality and latency requirements still pass.

Qwen

Alibaba Qwen

One family spans tiny dense models, coding MoEs and conditional memory.

Gemma

Google DeepMind

Gemma is a family of different design choices, not just different sizes.

Interview preparation

What you should be able to explain

Reconcile 120B nominal parameters with 75.01 GB mixed payload, then explain why Bolt’s 24.08 GB is not a whole-service peak.

Study the underlying systems → · Get the illustrated report →

Sources and scope

Covered checkpoints carry their own research dates. Assessments and comparison candidates are our interpretation of the cited designs, not measured cross-lab rankings. Each timeline entry links to its release evidence; model pages preserve pinned card and configuration sources.