multimodal

An image is not one token; it is hundreds to thousands of them after a second model has run. That changes the prefill budget, the batch shape and where the first bottleneck appears.

An image becomes a large number of tokens before the language model sees anything, so a request that looks small carries a prefill the size of a long document. The token arithmetic, the encoder that sits outside the usual parallelism, and the two capacity numbers that move.