Define request success, first-output latency and streaming continuity for each service class. Count bad outcomes against an explicit budget, then use paired long and short windows to detect significant ongoing degradation.
Updated Oct 2026 · Learn how AI infrastructure works through explanations, worked examples and diagrams, then practise applying it to interview questions.
Define request success, first-output latency and streaming continuity for each service class. Count bad outcomes against an explicit budget, then use paired long and short windows to detect significant ongoing degradation.
This answer is included with a free account. Sign in to read it and save your progress.
Use these concept explanations to review the ideas behind this question.
Keep request budgets separate from downtime equivalents. Check percentile boundaries, distinguish TPOT from individual stalls, and include both windows in each burn-rate rule.
No comments yet — be the first to share your approach.
