26You upgraded the serving engine and throughput fell 15 percent. Find the cause.▼hardNewFireworks AITogether AIBaseten4 replies◆ premiumEngine upgrades change defaults, kernel selection and scheduling at once, so attribution is the whole problem. The bisection that works on a version rather than a commit, the four categories of cause, and the pre-upgrade baseline without which none of this is possible.Open full answer →
19How do you measure effective training time, and where does the missing ten percent of a well-run cluster actually go?▼mediumNewMetaAnthropic4 replies○ sign inThe number is easy to state and hard to make honest, because every minute has to be classified and the classification is where the value is. The definition that survives scrutiny, the five categories the missing time falls into, and the property that turns a metric into a work list.Open full answer →