BROWSE BY CONCEPT

Questions by tag

serving45 deployment39 capacity34 debugging31 napkin math30 behavioral30 kv cache27 latency27 python25 memory22 decode20 throughput19 training19 bandwidth18 moe17 nvlink16 cost16 utilization15 nccl14 fabric14 routing14 vllm13 fp812 kubernetes12 power12 prefill12 batching11 isolation11 scheduling11 quotas11 roofline10 quantization10 cooling10 benchmarking10 preemption10 capacity planning9 all reduce9 sharding9 topology9 operations9 mfu9 automation9 measurement9 concurrency9 tco8 multi tenancy8 checkpointing8 dcgm8 procurement8 thermal8 speculative decoding8 ttft8 goodput8 autoscaling8 judgment8 arithmetic intensity7 evaluation7 gpudirect7 rdma7 facility7 acceptance7 validation7 fine tuning7 storage7 sizing7 attention7 communication7 collectives7 tpot7 cold start7 gang scheduling7 influence7 cuda6 shared memory6 tensor cores6 pcie6 tensor parallel6 tiling6 mig6 infiniband6 profiling6 reliability6 throttling6 escalation6 drain6 long context6 all to all6 detection6 slo6 observability6 disaggregation6 prefix caching6 streaming6 device plugin6 hbm5 memory bandwidth5 numerics5 nsight compute5 littles law5 nvl725 rail optimized5 batch size5 nccl tests5 cabling5 design5 firmware5 decision5 failure rate5 canary5 planning5 expert parallelism5 sampling5 tensor parallelism5 stragglers5 lora5 fragmentation5 chunked prefill5 acceptance rate5 warm pool5 nvme5 quality5 incidents5 career5 registers4 mixed precision4 h1004 occupancy4 b2004 tpu4 ecc4 transceivers4 burn in4 testing4 rollout4 false positives4 cuda graphs4 fsdp4 deepseek4 mtbf4 timeouts4 flight recorder4 fleet sizing4 mla4 kv transfer4 agents4 eviction4 idempotency4 scheduler4 configuration4 onboarding4 telemetry4 versioning4 regression4 fairness4 data loading4 roce4 tokenization4 bin packing4 fair share4 alerting4 slos4 postmortem4 process4 state machine4 checkpoints4 billing4 warps3 latency hiding3 gemm3 memory bound3 bf163 scaling3 data parallel3 blackwell3 fp43 expert parallel3 trainium3 mi300x3 rocm3 mps3 gpu sharing3 cluster design3 xid3 fleet operations3 dataloader3 fabric manager3 drivers3 provisioning3 contracts3 management3 out of band3 security3 spares3 fat tree3 verification3 performance3 optics3 congestion3 triage3 coalescing3 launch overhead3 fusion3 flashattention3 reduction3 block table3 rollback3 overlap3 accumulation3 methodology3 data parallelism3 ddp3 pipeline parallelism3 flops3 parallelism layout3 load balancing3 data pipeline3 hangs3 fault tolerance3 hot spares3 resharding3 continuous batching3 draft model3 awq3 p993 sglang3 snapshots3 reproducibility3 open weights3 queueing3 kernels3 caching3 pfc3 object storage3 inference3 counters3 deduplication3 kueue3 quarantine3 diagnostics3 error budget3 clocks3 on call3 thresholds3 admission control3 system design3 tenancy3 intervals3 thesis3 prioritization3 memory hierarchy2 l2 cache2 simt2 ridge point2 fp162 hbm3e2 interconnect2 xla2 systolic array2 jax2 dataflow2 amd2 cost per token2 block scaling2 networking2 oversubscription2 row remapping2 numa2 bom2 hardware selection2 datacenter2 constraints2 pue2 liquid cooling2 redundancy2 bring up2 switches2 checkpoint2 rma2 inventory2 cohort2 rack scale2 colocation2 upgrades2 migration2 accelerators2 health checks2 intermittent faults2 statistics2 upgrade2 sectors2 trace analysis2 coding2 bank conflicts2 padding2 transpose2 torch compile2 triton2 softmax2 numerical stability2 launch bounds2 spills2 ilp2 online softmax2 tooling2 productivity2 warp shuffle2 atomics2 vectorized loads2 pagedattention2 grouped gemm2 master weights2 loss scaling2 top k2 portability2 zero2 optimizer state2 ring2 activation memory2 activations2 capacity factor2 3d parallelism2 llama 32 elastic training2 rlhf2 rl infrastructure2 on policy2 metadata2 compute budget2 context length2 tail latency2 gqa2 packing2 benchmarks2 multi lora2 adapters2 model loading2 consistent hashing2 sse2 backpressure2 metrics2 multimodal2 multi turn2 determinism2 router2 oom2 radixattention2 mxfp42 hybrid attention2 tuning2 parsers2 forecasting2 replicas2 context2 engine support2 api2 cache2 sparse attention2 platform2 multi model2 attribution2 cascade2 model selection2 tokens2 deploy2 structured output2 weights2 6nd2 fleet2 deadline2 unit economics2 batch2 ecn2 fabric design2 clos2 checkpoint io2 prefetch2 batch processing2 manifests2 write burst2 pipeline2 contention2 container toolkit2 dra2 time slicing2 slurm2 volcano2 deadlock2 distributed training2 scheduler design2 placement2 taints2 gpu operator2 cgroups2 spot2 driver2 cost attribution2 allocation2 idle reclaim2 drain policy2 queue wait2 burn rate2 power cap2 action items2 accounting2 shutdown2 tracing2 kv pressure2 culture2 root cause2 training cluster2 rate limiting2 token bucket2 failover2 sessions2 reconciliation2 queues2 budgets2 load balance2 data structures2 simulation2 cancellation2 cpp2 coding interview2 motivation2 researchers2 sequencing2 safety2 risk2 learning2 cluster2 growth2 leadership2 scope2