← Coding for Infra / 35

hardNew

Implement a bounded request log with timers that remain correct under concurrency

Keep request timing independent, make snapshots consistent, and expose overwritten records. Test the ring after more concurrent writes than it can retain.

Updated Sep 2026 · Learn how AI infrastructure works through explanations, worked examples and diagrams, then practise applying it to interview questions.

Keep request timing independent, make snapshots consistent, and expose overwritten records. Test the ring after more concurrent writes than it can retain.

The full answer is part of Premium. A free account includes more answers in each topic, but does not unlock this one.

20 answers per topic instead of 10, plus saved progress and bookmarks · no cardor unlock all 303 remaining answers · ₹2,000 / $25

The concepts behind this question

Use these concept explanations to review the ideas behind this question.

📐 AI Systems Design🔒 Premium

Request Routing and Load Balancing for LLMsAn LLM router chooses a compatible replica, weighing queue time against reusable prompt state. Cached work helps only when the extra wait does not erase the saving.

🧮 Napkin Math & Capacity

KV Cache SizingDerive cache payload from the stored attention state, then account for mixed sequence lengths, block allocation and per-device limits. Worked examples distinguish exact bytes from deployable capacity.

💻 Coding for Infra🔒 Premium

Concurrency in Python, Go and C++Concurrency primitives coordinate overlapping work and shared state. Choose threads, processes or asynchronous tasks from the runtime and the work they must perform.

🩺 Fleet Reliability & ObservabilitySign in

DCGM and GPU TelemetryDCGM provides GPU telemetry and diagnostics. Combine activity, errors, environmental readings and application progress to understand a fleet’s health.

Explore the full curriculum →

UP NEXT ON YOUR JOURNEY

Next in this trackImplement a thread-safe in-memory KV store with snapshot transactions and TTL Next in this trackImplement sliding-window attention without confusing token positions with ring-buffer slots Next in this trackImplement symmetric INT8 quantization and prove where the reconstruction-error bound applies

DISCUSSION · 0

No comments yet — be the first to share your approach.