Implement bounded recovery probes, state-generation tickets and a recent failure window. Test concurrency and late completions.
This answer is included with a free account. Sign in to read it and save your progress.
free account · no card needed
← Coding for Infra / 20
mediumBasetenOpenAI
Implement a circuit breaker in front of a model endpoint. What are the states, and what does the half-open state buy?
Implement bounded recovery probes, state-generation tickets and a recent failure window. Test concurrency and late completions.
Updated Sep 2026 · Learn how AI infrastructure works through explanations, worked examples and diagrams, then practise applying it to interview questions.
The concepts behind this question
Use these concept explanations to review the ideas behind this question.
Foundational
Control Plane and API Design for GPU PlatformsA control plane records intended work and coordinates its execution. Clear retry, cancellation and listing semantics keep its API useful when requests or workers fail.📐 AI Systems Design
Advanced
Batching Queues and BackpressureA request batcher groups work to reduce processing overhead. Queue bounds, dispatch timing and failure handling keep that efficiency from becoming uncontrolled waiting.💻 Coding for Infra🔒 Premium
Advanced
Interval Merging and Utilization LogsInterval merging finds periods of activity and gaps between them. A sweep over start and end events also measures concurrency and utilization.💻 Coding for Infra🔒 Premium
Foundational
The GPU Credit Scheduler PatternA credit scheduler admits jobs without overspending an account’s balance. Clear state transitions make priority, cancellation and refunds easier to implement correctly.💻 Coding for Infra
UP NEXT ON YOUR JOURNEY
Next in this trackGiven per-rank heartbeats from a training job, detect which rank has stopped and when. What produces false positives?Next in this trackWrite a thread-safe counter in C++ that many threads increment. What are the memory-ordering and layout choices?Next in this trackWhy is std::unordered_map rarely the fastest hash map, and what would you use instead?
FEDITOR'S NOTE
Look for atomic admission, exactly one completion per ticket, failure classification and honest scope and timeout behavior.
DISCUSSION · 0
No comments yet — be the first to share your approach.