Core

Retry, Backoff and Idempotency

Retries can recover transient failures, but they also add load and may repeat side effects. Backoff, retry budgets and idempotency make that behavior explicit.

This concept is included with a free account. Sign in to continue reading.

free account · no card needed

RELATED CONCEPTS

Capacity and Backpressure Rate-Limiting Algorithms Batching Queues and Backpressure Concurrency in Python, Go and C++The Practical Coding Screen Playbook The GPU Credit Scheduler Pattern

PRACTICE THIS IN REAL QUESTIONS

Coding for InfraImplement retry with exponential backoff and jitter. Why is the jitter the part that matters, and what must never be retried?→LLM Inference & ServingYou stream tokens to clients over HTTP. What can go wrong, and how does the server handle each case?→CUDA, Triton & Kernel EngineeringGiven the addresses each thread in a warp touched, classify the access pattern: coalesced, strided or random. Write the classifier.→AI Infrastructure System DesignDesign an asynchronous batch inference API for a billion records with a completion deadline→Coding for InfraDesign the API for a GPU job scheduler in 45 minutes. What are the resources, states and semantics?→Coding for InfraImplement a bounded request log with timers that remain correct under concurrency→