28You serve a small, a medium and a large open-weights model. How do you route requests between them?▼mediumNewBasetenTogether AIModal4 replies◆ premiumThe cheapest correct answer beats the best answer at scale, so routing is a cost problem with a quality floor. The three routing signals that work, the cascade that trades latency for cost, and the measurement that catches quality loss no latency dashboard shows.Open full answer →
26Several unrelated jobs slow down at once and nothing is dropping packets. How do you confirm a pause cascade and find where it started?▼hardNewMetaxAI4 replies◆ premiumThe signature is unusual: many jobs degrade together, throughput collapses, and the drop counters stay at zero. Reading pause counters backward along the paths to find the origin, the three conditions that produce one, and the configuration change that stops it happening again.Open full answer →