AI Infra Interviews logo

python

AI infra interview questions tagged python, across every topic.

25 questions · 11 unlocked for you

Concepts behind "python"

The curriculum that explains the ideas these questions test.

Foundational
💻 Coding for Infra
The GPU Credit Scheduler PatternThe most widely reported coding problem in AI infrastructure loops is a small scheduler: accounts hold credits, jobs arrive with a cost and a priority, and you must decide which jobs run, in what order, without letting any account overspend, then extend it under follow-ups (refunds, reservations, concurrency limits, fairness). It is not a trick question; it is a test of whether you can model state cleanly, pick the right data structures, keep invariants under mutation, and talk about complexity while typing. This page works the problem from the first line to the fourth follow-up, with the code, the invariants, and the derivations.
Foundational
💻 Coding for Infra
The Practical Coding Screen PlaybookThe AI infrastructure coding screen is 45 to 60 minutes of building a small, realistic piece of systems code (a scheduler, a rate limiter, a batcher, a log parser, a cache) in the language you choose, with an interviewer who extends the problem twice and watches how you handle it. It is not a puzzle round: the score comes from working code early, tests that name the invariants, complexity said out loud, and calm follow-ups. Some companies allow an AI assistant and some ban it, and each policy changes what is measured. This page gives the minute-by-minute plan, the habits that score, and the mistakes that end the screen.
Core
💻 Coding for InfraSign in
Rate-Limiting AlgorithmsA rate limiter answers one question, 'may this request proceed now?', and the three classic algorithms answer it with different shapes of fairness and memory: the token bucket allows bursts up to a capacity and refills at a rate, the leaky bucket smooths output to a fixed rate, and sliding windows count recent requests exactly or approximately. AI platforms limit in tokens as well as requests, per tenant, across many gateways, which adds two twists: a request's cost is unknown until it finishes, and the counters must be shared. This page derives each algorithm, implements the token bucket correctly, and covers both twists.
Core
💻 Coding for InfraSign in
Retry, Backoff and IdempotencyA retry is a second request that the system did not budget for, and a thousand clients retrying at the same moment is a second outage that the first one caused. The craft is small and specific: retry only what is safe to retry, wait an exponentially growing random interval so the retries spread out, cap the total retries with a budget, and make every retried operation idempotent so a duplicate does not double-charge or double-train. This page derives why synchronized retries double the load, works the jitter arithmetic, implements the client correctly, and covers idempotency keys for the operations an AI platform exposes.
Advanced
💻 Coding for Infra🔒 Premium
Batching Queues and BackpressureWrite a request batcher is the coding round's version of the serving engine's scheduler: requests arrive one at a time, the GPU wants them in groups, and the batcher decides when a group is full enough to send without holding anyone too long or accepting more than it can hold. The two knobs are the maximum batch size and the maximum wait, the invariant is a bounded queue, and the follow-ups (priorities, cost-aware batching, cancellation, bounded in-flight batches) are the ideas the real engines carry. This page implements the batcher in asyncio, derives what each knob buys, and walks the follow-ups.
Advanced
💻 Coding for Infra🔒 Premium
Interval Merging and Utilization LogsGiven busy intervals per GPU, when was the whole cluster idle? What was the utilization per hour from a log of start and stop events? Which jobs overlapped? These are the interval problems of the infrastructure coding screen, and they share one tool: sort the endpoints and sweep. The sweep line turns every variant into a single pass with a counter, the sort is the only thing that costs more than linear time, and the edge cases (touching intervals, zero-length events, an unterminated start) are where candidates lose the round. This page works the standard problem and its relatives with code, tests and the complexity derivation.