scheduling

Compare the mechanisms in FlashAttention versions 1, 2 and 3: IO savings, work partitioning, asynchronous dependencies and FP8 accuracy.

Derive the ideal gain from filling a partial wave, then account for partial outputs, reduction order, workspace and the epilogue before selecting split-K.

One request finishes, another keeps writing, and a third waits outside. Follow the same workload through two schedules to see what continuous batching improves, and what it cannot promise.

Trace a 20,000-token prompt through a shared token budget, including the last partial chunk. Then choose a setting from measured first-token waits and streaming gaps.

A fourfold ratio is a symptom, not a diagnosis. Trace individual requests, separate waiting from execution and delivery, and test workload, scheduling, cache and capacity hypotheses against the service objective.

Use actual rank placement and measured job duration to decide whether waiting for locality beats starting immediately.

DRA gives the scheduler device claims and attributes to work with. Compare that with extended-resource counts, then check the deployed driver and controller support before promising sharing, partitioning or gang placement.

Choose the tenant boundary before planning base-model reuse. Account for frozen weights, adapter state and activations separately, then schedule by compatibility, fairness and the real cost of restarting training.

Implement priority admission and completion, preserve a preempted job’s sequence, and distinguish a sufficient greedy victim set from an optimal one.

Compare three GPU placement heuristics with executable traces. Show how admission and free-space shape change, then add workload and topology constraints.

Simulate admission, decode growth and preemption. Check memory accounting and useful progress with the same finite workload.

Allocate tasks by current dominant share and distinguish per-user fairness from cluster feasibility and physical placement.