Simulate admission, decode growth and preemption. Check memory accounting and useful progress with the same finite workload.
This answer is included with a free account. Sign in to read it and save your progress.
free account · no card needed
← Coding for Infra / 15
hardBasetenAnyscale
Implement the scheduler loop of a continuous-batching engine: admit, step, preempt. Where does it livelock?
Simulate admission, decode growth and preemption. Check memory accounting and useful progress with the same finite workload.
Updated Sep 2026 · Learn how AI infrastructure works through explanations, worked examples and diagrams, then practise applying it to interview questions.
The concepts behind this question
Use these concept explanations to review the ideas behind this question.
Core
Continuous BatchingContinuous batching changes which requests run between model iterations. Learn how a waiting request joins, why an empty slot is not enough, and how to balance throughput against each user’s token latency.🚀 Inference & ServingSign in
Advanced
GEMM Work Partitioning and Wave QuantizationOutput tiles determine how much independent matrix work is available. Split-K and work-centric schedules can reduce an underfilled final wave, while adding partial-result reduction and workspace costs.⚡ Kernels & Compilers🔒 Premium
Advanced
Chunked PrefillFollow a long prompt through a shared token budget, including its final partial chunk. Learn why chunking can reduce stream pauses without guaranteeing a fixed token gap or a universal throughput gain.🚀 Inference & Serving🔒 Premium
Advanced
Multi-Tenancy, Quotas and Fair ShareFollow three teams through GPU borrowing and reclaim. Distinguish quotas, reservations and fair sharing; see why a returning team may leave another above its nominal share, and why failures or indivisible jobs change what can be promised.🗂️ Scheduling & Orchestration🔒 Premium
UP NEXT ON YOUR JOURNEY
Next in this trackIn Go, download many model shards concurrently with a concurrency limit, a timeout and clean cancellation. Write it.Next in this trackRoute requests to replicas by prefix using a consistent hash ring. Why virtual nodes, and how many?Next in this trackImplement sampling from a model's logits with temperature, top-k and top-p. What are the numerical traps?
FEDITOR'S NOTE
Look for complete runnable state transitions, preserved accepted output, verified completion and honest limits of the simulated timing model.
DISCUSSION · 0
No comments yet — be the first to share your approach.