hardNewNVIDIAAnyscale

Implement a work-stealing deque. Why do the owner and the thieves take from opposite ends?

Two ends, two access patterns, and one design choice that removes almost all contention. Why the owner takes the newest task and thieves take the oldest, what that does to cache locality, and the measurement showing why this cannot be demonstrated in Python at all.

Updated Sep 2026 · Grounded in real AI infrastructure interview loops and written to a senior-engineer editorial bar, with every number worked and every diagram hand-built.

20 answers per topic instead of 10, plus saved progress and bookmarks · no cardor unlock all 283 remaining answers · ₹2,000 / $25

The concepts behind this question

Ranked by how closely each one overlaps this question's topic, so the first card is the thing to read if the answer above moved too fast.

Foundational

🧮 Napkin Math & Capacity

KV Cache SizingThe KV cache is the memory that decides how many users a serving replica can hold and how long their context can be. Its size per token comes from four numbers in the model's config file (layers, KV heads, head dimension, bytes per element) and one formula; multiplied by context and concurrency it is the number every capacity plan is built on. This page derives it, works it for four models including an MLA one, and shows the two places candidates get it wrong by a factor of eight.

Advanced

💻 Coding for Infra🔒 Premium

Concurrency in Python, Go and C++Infrastructure code is concurrent by nature: a loader feeding a GPU, a gateway holding ten thousand streams, a controller reconciling a fleet. The coding screen tests whether you know which primitive fits which problem in the language you claim, and the three languages the field uses answer differently: Python has one interpreter lock and an event loop, Go has cheap goroutines and channels, C++ has threads, mutexes and atomics with no safety net. This page gives the model of each, works the favourite problems (a thread-safe LRU, a worker pool, a bounded fan-out) in each, and derives when threads, processes or async buy throughput.

Advanced

💻 Coding for Infra🔒 Premium

Batching Queues and BackpressureWrite a request batcher is the coding round's version of the serving engine's scheduler: requests arrive one at a time, the GPU wants them in groups, and the batcher decides when a group is full enough to send without holding anyone too long or accepting more than it can hold. The two knobs are the maximum batch size and the maximum wait, the invariant is a bounded queue, and the follow-ups (priorities, cost-aware batching, cancellation, bounded in-flight batches) are the ideas the real engines carry. This page implements the batcher in asyncio, derives what each knob buys, and walks the follow-ups.

Advanced

💻 Coding for Infra🔒 Premium

Interval Merging and Utilization LogsGiven busy intervals per GPU, when was the whole cluster idle? What was the utilization per hour from a log of start and stop events? Which jobs overlapped? These are the interval problems of the infrastructure coding screen, and they share one tool: sort the endpoints and sweep. The sweep line turns every variant into a single pass with a counter, the sort is the only thing that costs more than linear time, and the edge cases (touching intervals, zero-length events, an unterminated start) are where candidates lose the round. This page works the standard problem and its relatives with code, tests and the complexity derivation.

Explore the full curriculum →

UP NEXT ON YOUR JOURNEY

Next in this trackSimulate speculative decoding to find the expected tokens per round and the batch size where it stops paying.Next in this trackDesign the API for a GPU job scheduler in 45 minutes. What are the resources, states and semantics?Next in this trackImplement dominant resource fairness for a cluster where jobs need different mixes of GPU, CPU and memory.

FEDITOR'S NOTE

Scored on opposite-end access as a contention and locality decision, on the owner's LIFO discipline giving cache reuse, and on recognising that a CPU-bound Python benchmark cannot show the benefit.

DISCUSSION · 0

No comments yet — be the first to share your approach.