19You have one week to get a newly released trillion-parameter model serving production traffic. Plan it.▼hardNewTogether AIFireworks AIBaseten4 replies○ sign inTwo steps consume most of the week and neither is the one people plan for. The gate that can end the project on day one, the multi-hour download, the correctness check almost nobody runs, and the canary that has to see real traffic before Friday.Open full answer →
27A model interleaves linear-attention and full-attention layers. What changes about serving it?▼hardNewTogether AIFireworks AIBaseten4 replies◆ premiumMost layers stop having a growing cache, which changes capacity planning by a large factor and demands a cache manager most engines did not have. What the memory model becomes, what the engine must implement, and the two features that were disabled by default while it settled.Open full answer →