31The open-weights model you are deploying is multimodal. What changes about serving it?▼mediumNewBasetenTogether AIFireworks AI4 replies◆ premiumAn image becomes a large number of tokens before the language model sees anything, so a request that looks small carries a prefill the size of a long document. The token arithmetic, the encoder that sits outside the usual parallelism, and the two capacity numbers that move.Open full answer →
27Design token-level billing and metering for an LLM API: count at the engine, aggregate, reconcile, and survive the failure modes.▼mediumNewOpenAITogether AI4 replies◆ premiumBilling is the one pipeline where a dropped event is money and a duplicated one is a support ticket. Where tokens are counted and by whom, the usage event and its idempotency key, the aggregation that closes a month, the reconciliation that proves the ledger against the engines, and the six ways it goes wrong.Open full answer →