AI Infrastructure Coding Interview Practice: Systems Problems and Follow-Ups

Practical infrastructure coding exercises ask you to implement a policy and explain what happens at its boundaries. A rate limiter, resumable iterator or worker pool gives you concrete requirements to test. These are useful preparation problems alongside algorithms, not evidence that every AI infrastructure employer avoids algorithm questions or uses the same interview format.

Confirm the permitted language, tools and timing before an assessment. The interview process guide links to official employer guidance. The tasks below are our practice recommendations, not a list of confirmed company questions.

A rate limiter begins with the policy

Define whether the requirement is a fixed calendar window, a strict rolling interval or an average rate with bursts. A fixed-window counter is inexpensive and valid for its own contract, but allows up to two windows’ worth of requests around a boundary. That makes it unsuitable for a strict rolling-window cap.

A sliding log tracks individual request times and spends memory proportional to retained traffic. A token bucket tracks available tokens with a refill rate and burst capacity. Choose from the requirement, then test requests exactly at the boundary. The systems coding track provides exercises for these choices.

For a concrete boundary test, suppose the policy permits 100 requests per calendar minute. Accept 100 just before the minute changes and another 100 immediately afterward. The fixed-window policy has behaved as specified, even though 200 requests arrived within a short rolling interval. If that burst is unacceptable, change the contract and implementation together. Also state whether rejected attempts consume quota and whether time comes from a trusted server clock; both choices affect the tests.

A versioned store makes history explicit

Implement writes with timestamps and reads as of a given time. Decide whether writes arrive in order and how equal timestamps are resolved. A sorted history can support binary-search reads, but insertion and memory costs depend on the write pattern.

Next add a retention limit. State what a read outside retained history returns instead of silently pretending the latest value is historical. If concurrent writers are allowed, define the synchronization rule. An extensible design is useful because its invariants are clear, not because every value was replaced with a complicated event log.

A resumable iterator needs a stable reference

An iterator token must identify both progress and the data version or snapshot to which that progress belongs. An offset alone may duplicate or omit items if the underlying collection changes between runs. Decide whether the contract permits that behavior.

Practice resuming after an empty page, a process restart and a changed dataset. Explain what the token stores and how it is validated. You can begin with an immutable file or list, then discuss what changes for a remote paginated service. Keep the first implementation’s scope explicit.

A worker pool must stop as well as start

Bound the worker count and queue size, and decide what happens when producers are faster than consumers. On cancellation, blocked senders and receivers need a way to stop. The Go pipeline and cancellation guide is one primary explanation of this failure mode; the same ownership question applies in other languages.

Test no work, one failed task, cancellation with a full queue and a slow consumer. Decide who closes each channel or queue and who releases resources. A timeout reported to the caller does not establish that background work has stopped.

For GPU workloads, the cluster scheduling track adds resource and placement constraints. A generic worker count cannot express which devices may cooperate efficiently or whether a job must start all of its workers together.

Add one requirement after the first solution

A useful rehearsal delivers a second requirement after the initial tests pass: bounded history, distributed rate limiting or cancellation. Identify which assumption changed and what must migrate. Do not assume the interviewer will reveal every later extension when asked.

Avoid implementing speculative features merely to anticipate a follow-up. A simple structure with named invariants is often easier to adapt than an abstraction designed around imagined requirements. Explain the cost of the change and preserve the tests for the behavior that remains valid.

Explain and test the result

State time and memory complexity for the actual implementation, including retained state. Test an empty input and the policy boundary, then identify what a restart or multiple replicas would change. The reliability questions help connect those extensions to observable failure behavior.

As a practice format, give yourself 30 minutes for a small implementation and then add a follow-up. That is a self-imposed exercise, not a verified employer time limit. Record where syntax, unclear requirements or a weak data model slowed you down, and repeat the part that caused the difficulty.

Use the question bank to choose a problem that matches the target role. A completed, tested exercise with a defensible policy is more informative than a count of how many problems you opened.