scheduler
AI infra interview questions tagged scheduler, across every topic.
4 questions · 2 unlocked for you
Concepts behind "scheduler"
The curriculum that explains the ideas these questions test.
Foundational
SGLang Server Arguments That MatterSGLang's tuning model is different from vLLM's in one way that matters: it exposes the scheduler's aggressiveness and the static memory fraction as direct knobs, and its own documentation gives target values for the runtime signals those knobs move. That makes tuning it a measurement loop rather than guesswork. Aim for a queue of a hundred to a couple of thousand requests, token usage above 0.9, and five to eight gigabytes of free GPU memory after startup, then adjust the flags that move each one.🧮 Open Weights & Serving Engines
Foundational
Inference Platform ArchitectureAn LLM inference platform is the layer between a product's API call and a GPU running a serving engine, and every design round starts from its reference shape: a gateway that authenticates and rate-limits, a router that picks a replica with the right model and a warm cache, a per-replica scheduler that batches, engines that run prefill and decode, a KV cache tier, an autoscaler, and the observability that makes it operable. This page draws that shape, sizes each box for a concrete workload, and walks the derivation from user demand to replica count that every design answer has to contain.📐 AI Systems Design
Foundational
The GPU Credit Scheduler PatternThe most widely reported coding problem in AI infrastructure loops is a small scheduler: accounts hold credits, jobs arrive with a cost and a priority, and you must decide which jobs run, in what order, without letting any account overspend, then extend it under follow-ups (refunds, reservations, concurrency limits, fairness). It is not a trick question; it is a test of whether you can model state cleanly, pick the right data structures, keep invariants under mutation, and talk about complexity while typing. This page works the problem from the first line to the fourth follow-up, with the code, the invariants, and the derivations.💻 Coding for Infra
Core
Slurm for AI ClustersSlurm is the scheduler most large training clusters still run, because it was built for exactly this shape of work: long jobs that need many nodes at once, launched with one command, placed with knowledge of the network. A candidate for a training-infrastructure role is expected to read an sbatch script, know how GPUs are requested and enforced, and explain why a job is stuck in the queue. This page covers the model, the commands that matter, the GPU-specific configuration, and the failure modes a platform engineer meets.🗂️ Scheduling & OrchestrationSign in