05Explain arithmetic intensity and the roofline model. Where is the ridge point on an H100, and what does it tell you about a kernel?▼medium★ EssentialNVIDIAFireworksTogether AI4 repliesunlockedCount two matrix shapes to derive the H100 ridge, throughput ceilings and time floors. Compare an illustrative measurement with its bound before deciding what to optimize.Open full answer →
06Is LLM decode memory-bound or compute-bound? Show me the arithmetic that decides it.▼medium★ EssentialOpenAIAnthropicTogether AI4 repliesunlockedDerive the weight-only shortcut, then add KV traffic and feasible eight-GPU placement. A worked latency floor shows why batch size alone cannot establish the decode bottleneck.Open full answer →
07bf16, fp16, fp8: what is the difference at the bit level, and where does each one belong in training and serving?▼mediumNVIDIAMeta4 repliesunlockedUse proportional bit layouts and adjacent values around one to distinguish range from precision. Then choose formats from numerical tests and actual kernel support.Open full answer →
08How much faster is an H200 than an H100, really? Which workloads see the gain and which do not?▼mediumNVIDIACoreWeaveLambda4 repliesunlockedSeparate H200 bandwidth and capacity gains from measured speedup, then test realistic memory headroom and cost per useful completed workload.Open full answer →
09NVLink versus PCIe: when does the interconnect actually matter for an LLM workload, and when can you get away with PCIe?▼mediumNVIDIALambdaCrusoe4 repliesunlockedNormalize link directions, count collective bytes, and see why a gradient reduction can outlast its real overlap window.Open full answer →
10What is warp divergence, why does it cost you, and how would you find and fix it in a kernel?▼mediumNVIDIA4 repliesunlockedCompare the same 1,024 items under mixed and grouped lane assignments. Derive issue-work waste, then include regrouping and memory costs before claiming a speedup.Open full answer →
11A kernel runs at 25% occupancy. Is that a problem? Walk me through what occupancy buys and when you would leave it low on purpose.▼mediumNVIDIAFireworks4 replies○ sign inDerive a block-level occupancy limit, then compare the extra residency with spill traffic and lost reuse. Use a matched experiment rather than a target percentage.Open full answer →
12Explain what HBM is and why memory bandwidth, not compute, is the wall for LLM inference.▼mediumNVIDIAAMDMicron4 replies○ sign inDerive HBM bandwidth from interface width and pin rate, then connect it to a model that actually fits. Separate raw interface capacity from measured decode time.Open full answer →
19MIG versus MPS: what isolation does each give you when sharing a GPU, and which would you pick for a multi-tenant inference node?▼mediumNVIDIACoreWeaveNebius4 replies○ sign inSee the GPU-instance and compute-instance boundaries, budget a small slice, and distinguish memory capacity controls from bandwidth isolation.Open full answer →
22Explain GPUDirect RDMA. What does it remove from the data path, and when does it change performance?▼mediumNVIDIACrusoe4 replies◆ premiumCompare host-staged and direct GPU-to-NIC transfers, count traffic at each boundary, and check transport selection before blaming a slow collective on missing GPUDirect.Open full answer →
24Why does the batch size change the TFLOPS a GEMM achieves? Draw the curve and explain each region.▼mediumTogether AIBaseten4 replies◆ premiumDerive how reusing weights changes arithmetic intensity, then separate the roofline ceiling from tile-wave effects and measured GEMM throughput.Open full answer →
26What does ECC on a GPU cost you, and why do you keep it on across a fleet?▼mediumCoreWeaveLambdaMicrosoft4 replies◆ premiumTrace a changed BF16 bit, distinguish corrected and uncorrectable errors, and calculate recovery cost without assuming every device has the same ECC overhead.Open full answer →
01You have budget for roughly 1,000 GPUs and a greenfield site. Specify the cluster.▼medium★ EssentialCoreWeaveCrusoeMeta4 repliesunlockedSpecify a cluster from workload and facility constraints, then derive node counts, network ports, storage bursts and acceptance tests.Open full answer →
02Your fleet is decode-heavy. Is a B300 worth 1.4 times a B200's power for 1.6 times the memory?▼mediumNVIDIABasetenTogether AI4 repliesunlockedCompare GPU memory, decode traffic and measured useful throughput. Work out a capacity ceiling without mistaking it for a performance forecast.Open full answer →
03Leadership bought GB300 NVL72 racks. Your datacenter is built for 15 kW cabinets. What do you tell them?▼medium★ EssentialMicrosoftCoreWeaveCrusoe4 repliesunlockedCompare rack-level power, cooling, structural and connectivity requirements with the offered site, then evaluate quoted deployment alternatives.Open full answer →
04After recabling a row to 800G, a third of the links will not come up. Find the cause.▼mediumNVIDIACoreWeaveMeta3 repliesunlockedDiagnose failed 800G links using endpoint compatibility, lane modes, inventory and controlled swaps. Learn why speed labels and failure patterns do not prove a cause.Open full answer →
05How much power does a 512-GPU cluster actually draw, end to end?▼mediumCrusoeCoreWeaveMicrosoft4 repliesunlockedBuild a cluster power budget with clear equipment boundaries, facility overhead and failover requirements. Distinguish energy use from reserved electrical capacity.Open full answer →
08New nodes arrive with no software. Walk me through bring-up, and say why the order matters.▼medium★ EssentialNVIDIACoreWeaveMeta4 repliesunlockedBring up the supported hardware and software stack one dependency at a time, then prove correctness and recovery at increasing scale.Open full answer →
10You are building a fine-tuning service for customer models under 30B. RTX PRO 6000 or H100?▼mediumLambda LabsModalBaseten4 repliesunlockedSize adapter and full-parameter jobs before comparing RTX PRO 6000 and H100. Test memory headroom, MIG packing, software support and real communication costs.Open full answer →
12Count the switches, cables and transceivers for a 2,048-GPU rail-optimized fabric.▼mediumCrusoeCoreWeaveNVIDIA4 replies○ sign inCount one complete rail, then distinguish logical links from cages, optical modules and cable assemblies. Derive power from the same physical inventory.Open full answer →
14Size the storage for a 2,048-GPU training cluster. What numbers actually drive it?▼mediumWEKAVAST DataCoreWeave4 replies○ sign inBuild a storage budget that includes retained checkpoints, the next save, overlapping datasets and failure protection. Then check whether the same tier can meet write and restore deadlines.Open full answer →
16Air, rear-door heat exchanger or direct-to-chip? Decide for a 60 kW rack.▼mediumMicrosoftCoreWeaveCrusoe4 replies○ sign inCompare cooling methods from the supported hardware and heat path. Calculate airflow or liquid flow, then verify capacity, residual air load and failure response.Open full answer →
18How many spare GPUs, nodes, cables and transceivers do you hold for a 2,048-GPU fleet?▼mediumCoreWeaveMetaLambda Labs5 replies○ sign inEstimate replacement demand over replenishment time, then size for shortage risk and failure correlation. Distinguish shelf parts from runnable spare nodes.Open full answer →
20A GPU cluster has four separate networks. Name them and say what each one carries.▼medium★ EssentialNVIDIACoreWeaveCrusoe3 replies○ sign inSeparate compute, storage and management responsibilities, then decide which need independent physical paths and count their endpoints correctly.Open full answer →
23Buy hardware, colocate, or rent from a GPU cloud? Work the decision for a 512-GPU need.▼mediumCoreWeaveLambda LabsCrusoe4 replies◆ premiumCompare ownership, colocation and rental using the same useful work, full annual costs and realistic demand. Work a break-even from explicit assumptions.Open full answer →
25What does nvidia-fabricmanager do, and what exactly breaks when it is not running?▼mediumNVIDIACoreWeaveMeta3 replies◆ premiumFabric Manager helps make an NVSwitch system ready. Learn the platform-specific roles, failure symptoms and checks that prove the communication path works.Open full answer →
27Walk me through cabling a rail-optimized cluster. What goes where, and what goes wrong?▼mediumCrusoeCoreWeaveNVIDIA4 replies◆ premiumMap stable endpoints to rails, choose supported cables from measured routes, and verify topology before interpreting collective performance.Open full answer →
31A vendor claims their accelerator beats an H100 at half the price. How do you evaluate that?▼mediumCoreWeaveCrusoeMeta4 replies◆ premiumTest the vendor claim on your workload, including quality, latency, scaling and operating costs. A lower peak or lower price alone does not settle it.Open full answer →
32Design the health check that decides whether a node may accept jobs.▼mediumMetaCoreWeaveCrusoe4 replies◆ premiumGate admission with fresh, platform-aware evidence. Use absolute and peer baselines, budget test overhead and keep automated cordons under control.Open full answer →
35Your cluster is going into a colocation facility rather than your own building. What changes?▼mediumCoreWeaveLambda LabsCrusoe4 replies◆ premiumDefine who owns each facility and hardware action, model restoration time, and verify power, cooling, telemetry and growth commitments before deployment.Open full answer →
37Design the system that tells you every firmware version on every device in the fleet.▼mediumMetaCoreWeaveMicrosoft4 replies◆ premiumCollect stable device identities, versions and freshness, then compare them with approved configurations and retain the history for investigations.Open full answer →
39Two thousand GPUs are being replaced. What do you do with the old ones?▼mediumMetaCoreWeaveLambda Labs4 replies◆ premiumCompare the old fleet's useful value with operating costs and resale, verify its health, and complete data handling before reassignment or transfer.Open full answer →
03Given the addresses each thread in a warp touched, classify the access pattern: coalesced, strided or random. Write the classifier.▼mediumNVIDIA3 repliesunlockedBuild a mask-aware classifier that counts complete byte intervals, separates duplicate lane demand from distinct bytes, and streams trace summaries.Open full answer →
04Explain shared memory bank conflicts with the bank arithmetic, show a kernel that has them, and fix it with padding.▼mediumNVIDIATogether AI4 repliesunlockedDerive the bank mapping before and after padding, trace a fixed-tile kernel, and measure whether the saved conflict outweighs other resource costs.Open full answer →
06Write a fused row softmax in Triton, explain its memory traffic, and say where it stops scaling.▼mediumOpenAIFireworksTogether AI4 repliesunlockedTrace stable row normalization, handle independent strides and fully masked inputs, then separate fusion traffic from measured speed.Open full answer →
08Explain occupancy and register pressure: launch bounds, spills, the calculator, and why 50% occupancy can beat 100%.▼mediumNVIDIA4 repliesunlockedDerive resident-block limits and distinguish resident warps from ready work before trading registers for more occupancy.Open full answer →
09Explain FlashAttention. Why is it called IO-aware, and what does it actually save?▼medium★ EssentialFireworksTogether AIAnthropic4 repliesunlockedFollow the running maximum and weighted sum across attention tiles, with executable arithmetic and an honest memory-traffic comparison.Open full answer →
11When do you write a kernel in Triton, and when do you have to drop down to CUDA?▼mediumOpenAITogether AI4 replies○ sign inStart with the operation, supported hardware and performance target. Compare ordinary Triton, lower-level Triton interfaces and CUDA using a measured gap that is large enough to justify maintaining custom code.Open full answer →
13Sum 100 million floats on the GPU as fast as the hardware allows. Write the kernel and justify each step.▼mediumNVIDIA4 replies○ sign inBuild a reduction with a checked tail, one partial result per block and a final combine. Count the input bytes, then investigate the instructions, synchronization and numerical behavior that a bandwidth estimate leaves out.Open full answer →
14Transpose a large matrix at close to copy bandwidth. Why is the naive version slow, and what does each fix buy?▼mediumNVIDIA3 replies○ sign inTrace how a shared-memory tile turns coalesced input rows into coalesced output rows. Then separate global sector footprints, shared-bank conflicts and measured bandwidth, including the cost of handling edge tiles.Open full answer →
15What does torch.compile actually do to your model, and when does it fail to help?▼mediumOpenAIAnthropicMeta4 replies○ sign inFollow graph capture, guards, differentiation and generated kernels. Then distinguish graph breaks from recompilation and measure whether warm execution repays compilation and startup costs.Open full answer →
21How do you overlap host-to-device transfers with compute, and what has to be true before the overlap actually happens?▼mediumNVIDIALambda4 replies◆ premiumAn asynchronous call returns early, but the timeline still shows copies and kernels taking turns. Derive the pipeline's fill and drain costs, then check dependencies, buffers and hardware before expecting overlap.Open full answer →
22Why do tensor cores accumulate in fp32 when the inputs are bf16, and why do optimizers keep fp32 master weights?▼mediumNVIDIAMeta4 replies◆ premiumWatch a low-precision sum stop changing and a small weight update round away. Separate storage precision, accumulation, optimizer state and dynamic range before choosing a mixed-precision training policy.Open full answer →
25Select the top k logits from a 128,000-token vocabulary on the GPU. What shape does the kernel take and why not just sort?▼mediumNVIDIAOpenAI4 replies◆ premiumChoose between local candidates, radix selection and sorting from k and the required output. A tested FP32 reference makes ties, signed zero, NaNs and threshold selection explicit.Open full answer →
26You added three local variables to a working kernel and it got 30 percent slower. Explain what happened and how you would confirm it.▼mediumNVIDIA4 replies◆ premiumA few extra live values can change register allocation, residency, generated instructions or spills. Compare compiler evidence with runtime behavior before attributing a 30% slowdown to any one of them.Open full answer →
05FSDP or DeepSpeed ZeRO-3: which would you pick for a new training codebase today, and why?▼mediumMetaMicrosoftDatabricks4 repliesunlockedCompare FSDP2 and ZeRO-3 through one parameter group’s lifetime, a scoped traffic model and a save-resume experiment, rather than a universal speed ranking.Open full answer →
06You need to train a 100B dense model and it does not fit on one node. Walk me through how you would lay it out.▼medium★ EssentialOpenAIAnthropicxAI4 repliesunlockedSeparate the 100B memory floor from the training deadline, then compare two 1,024-GPU layouts with their actual state ownership and communication schedules.Open full answer →
07Derive the cost of a ring all-reduce. Why is it bandwidth-optimal, and where does it stop scaling?▼mediumNVIDIAGoogle4 repliesunlockedTrace a chunk, derive both traffic phases, and state the endpoint model behind bandwidth optimality. A worked example shows why nearly constant bytes do not imply constant completion time.Open full answer →
08Do the pipeline bubble math for me. How do micro-batches shrink it, and what does 1F1B change?▼mediumNVIDIAMeta4 repliesunlockedTrace a legal two-stage schedule and count its idle slots. Separate bubble denominators and explain how 1F1B changes activation lifetimes under explicit assumptions.Open full answer →
09Does tensor parallelism have to stop at 8? Show me the numbers.▼mediumNVIDIAAnthropic4 repliesunlockedCount tensor-parallel messages and compare their payload service time on two explicit paths. Explain why TP8 is a common placement choice, not a fixed ceiling.Open full answer →
11When does activation checkpointing pay for itself, and what would you checkpoint first?▼mediumMetaOpenAI4 replies○ sign inFollow saved and recomputed values through backward. Compare complete update time and peak memory instead of promising a fixed memory saving or slowdown.Open full answer →
16Explain mixed-precision training. Why does fp16 need loss scaling and bf16 not, why keep fp32 master weights, and what changes with fp8?▼mediumNVIDIAMeta4 replies○ sign inSeparate gradient range from update precision using a small numerical example, and make the training-state recipe explicit.Open full answer →
17Your training run delivers only 60% of the throughput you projected. How do you find out whether it is compute, memory, network or I/O?▼medium★ EssentialAnthropicOpenAICrusoe4 replies○ sign inUse per-rank arrival times and controlled substitutions to explain a slow step without mistaking collective wait for network transfer.Open full answer →
22You see 'Watchdog caught collective operation timeout' in the logs. What happened, and how do you find which rank caused it?▼mediumCoreWeaveCrusoeMeta4 replies◆ premiumRead a watchdog timeout without blaming the reporting rank. Compare group-scoped traces, separate missing evidence from mismatched calls, and distinguish CPU submission from GPU completion.Open full answer →
27Your team wants to replace a PPO-style RLHF pipeline with DPO. What leaves the cluster, and what does the training job look like afterwards?▼mediumAnthropicOpenAIDatabricks◆ premiumOffline DPO trains on existing preference pairs, so online rollout and reward services leave the update loop. Account for both responses, decide whether reference scores can be cached, and retain the data-generation and evaluation costs outside training.Open full answer →
04How does PagedAttention work, and what problem was it solving?▼medium★ EssentialvLLMTogether AIBaseten4 repliesunlockedTrace one cache position through a block table, calculate the memory recovered from oversized reservations, and explain why sharing still needs careful writes and matching.Open full answer →
06You have eight H100s and Llama 3.1 70B. How many concurrent users can you serve, and what changes the number?▼medium★ EssentialTogether AIBasetenCoreWeave4 repliesunlockedWork from each GPU's cache budget, reserve room for generated tokens, then check latency separately. A limiting-rank example shows why fitting today's prompts is not enough to admit tomorrow's work.Open full answer →
07A long prompt arrives while sixty users are mid-generation. What happens, and how does chunked prefill fix it?▼mediumvLLMAnthropicBaseten4 repliesunlockedTrace a 20,000-token prompt through a shared token budget, including the last partial chunk. Then choose a setting from measured first-token waits and streaming gaps.Open full answer →
08When does speculative decoding speed up serving, and when does it break even or hurt?▼medium★ EssentialTogether AIFireworksAnthropic4 repliesunlockedTrace an accepted prefix, a rejection and the correction token; derive emitted progress and compare it with a measured round cost.Open full answer →
09Compare the KV cache footprint of multi-head, grouped-query and multi-head latent attention with numbers.▼mediumDeepSeekMetaFireworks4 repliesunlockedDraw query-head sharing, count actual stored state and compare payloads with explicit model geometry and binary units.Open full answer →
10You need to quantize a model for serving. Which method do you pick, and what do you measure before shipping it?▼mediumFireworksTogether AIRed Hat4 repliesunlockedSeparate weight storage, matrix arithmetic and attention-cache precision. Choose a supported candidate using task quality, memory and service measurements.Open full answer →
12Every request shares a 2,000-token system prompt. How does prefix caching exploit that, and how does the radix tree work?▼mediumSGLangAnthropicOpenAI4 replies○ sign inTwo requests share 2,000 leading tokens but ask different questions afterward. Follow the shared state through a branch, calculate the reusable blocks, and distinguish memory savings from measured latency savings.Open full answer →
13You have one GPU and a synchronous API that receives 100 documents at once. Design the batching, and show the latency math.▼medium★ EssentialAnthropic4 replies○ sign inPreserve document identity through packed prompts and completion reordering. Derive rounded cache reservations, then measure feasible wave sizes against the API contract.Open full answer →
21You stream tokens to clients over HTTP. What can go wrong, and how does the server handle each case?▼mediumOpenAIBaseten4 replies◆ premiumTrace a dropped stream through bounded queues, engine cancellation and safe resource release; define what retries and incomplete delivery mean.Open full answer →
24Draw the throughput versus latency curve for LLM serving and explain where you would operate on it.▼mediumTogether AIAnyscale4 replies◆ premiumSeparate device throughput bounds from measured service curves, then choose a feasible operating point using latency, quality and goodput.Open full answer →
26nvidia-smi shows 30 percent utilization on our serving fleet. Is that a problem, and what would you look at instead?▼mediumBasetenModal4 replies◆ premiumThirty percent GPU utilization means kernels ran during about thirty percent of the sampled time. Find out what limits those kernels, what happens between them, and whether requests are meeting their latency targets.Open full answer →
29A customer sends the same prompt twice at temperature zero and gets different answers. Explain why, and what you can promise them.▼mediumOpenAIAnthropic4 replies◆ premiumGreedy decoding chooses the highest-scoring token, but identical prompt text does not guarantee identical inputs or numerical execution. Check the effective request and deployment, investigate the first differing token, and define the reproducibility guarantee precisely.Open full answer →
31Your speech model runs faster than real time. Why do users still wait or hear gaps?▼mediumNVIDIAGoogleBaseten4 replies◆ premiumWork two audio timelines: input accumulation before transcription and playback starvation after synthesis. Compare real-time factor, response delay and safe concurrent capacity without mixing their units.Open full answer →
01You need to deploy GLM-5.3. How many GPUs do you need, and what would you verify first?▼medium★ EssentialTogether AIFireworks AIBaseten4 repliesunlockedDerive a deployment candidate from the official tensor index, then account for latent-cache replication and the product workload. A worked example shows why enough aggregate HBM can still leave every GPU out of memory.Open full answer →
02vLLM crashes with out of memory during startup on a model that should fit. Debug it.▼mediumFireworks AIBasetenTogether AI4 repliesunlockedFind which startup step runs out of memory, count the allocations on that GPU, and change the setting that controls the shortage.Open full answer →
03A model you have never heard of trends on Hugging Face. Estimate the deployment before downloading it.▼medium★ EssentialBasetenTogether AIFireworks AI4 repliesunlockedTurn public metadata into a deployment candidate with explicit uncertainty. Inspect architecture, stored formats and engine support, then work a per-rank memory example that explains why an aggregate fit can fail.Open full answer →
05You are serving an agent product with long shared prompts. vLLM or SGLang?▼mediumBasetenTogether AIModal4 repliesunlockedMeasure reachable prefix reuse, separate token savings from latency, and compare both engines on the same agent contract and resource budget.Open full answer →
08Set max-model-len and max-num-seqs for a chat product from first principles.▼mediumBasetenModalTogether AI4 repliesunlockedDerive a prompt-plus-output contract, calculate per-rank state capacity, and measure the concurrency that meets latency and overload targets.Open full answer →
09Your new model deployment produces fluent answers that score badly on evaluations. Where do you look?▼mediumBasetenTogether AIModal4 repliesunlockedThe answers sound fluent, but the scores dropped. Compare inputs, generation settings, raw output, parsing and scoring to find where the new deployment differs.Open full answer →
10Design the benchmark you would run before putting a new open-weights model into production.▼mediumTogether AIFireworks AIBaseten4 repliesunlockedDefine a reproducible workload and service contract, account for every offered request, and choose a measured operating region with evidence and margin.Open full answer →
11The model ships in FP8. Should you requantize to four bits to fit more of it on fewer GPUs?▼mediumFireworks AITogether AIBaseten4 replies○ sign inCompare actual encoded tensors, per-rank runtime fit, supported kernels and held-out task quality. Four-bit weight-only kernels can accelerate wider arithmetic, and smaller weights need not reduce the GPU count.Open full answer →
12Prefix caching cut your benchmark's latency in half. Why might production see none of that?▼mediumBasetenTogether AIModal4 replies○ sign inA repeated-prompt test measures one reuse pattern. Production gains depend on token-weighted reuse, routing, residency, alignment and the prefill share of latency; a falling hit rate does not uniquely identify eviction.Open full answer →
15Your product forecasts 12,800 output tokens per second at peak. Size the fleet.▼mediumBasetenTogether AIModal4 replies○ sign inThe output forecast is a starting point. Match the prompt and state workload, derive replica counts from an SLO-tested rate, check simultaneous loss scenarios and keep the cost denominator consistent with the forecast.Open full answer →
17Your deployment is limited by KV cache rather than by compute. What are the options?▼mediumBasetenTogether AIFireworks AI4 replies○ sign inConfirm that requests are waiting for usable state capacity. Separate allocated memory, live blocks and evictable prefixes, then compare admission, lower-precision state, prefix sharing and hardware changes with explicit assumptions.Open full answer →
20Self-host a 753B open-weights model or call a hosted API? Work the crossover.▼mediumBasetenTogether AIModal4 replies○ sign inPrice the same workload and service guarantees, including input tokens, idle capacity, failures and operations. Then check whether the algebraic crossover fits the fleet before treating it as a buying decision.Open full answer →
21One replica in a serving fleet returns tokens slower than its siblings. Find out why.▼mediumBasetenModalTogether AI4 replies◆ premiumAn apparent outlier can reflect workload mix, queueing, runtime state or a degraded execution path. Normalize the comparison, run a matched replay and use critical-path arithmetic to test possible causes.Open full answer →
23The model authors publish a new revision. How do you roll it out without a quality regression?▼mediumBasetenTogether AIModal4 replies◆ premiumA minor revision can change weights, encoding or response behavior. Compare pinned deployment bundles, evaluate task slices, monitor a representative canary and keep enough capacity to restore the old service.Open full answer →
25An SGLang deployment underperforms. Tune it against the project's own published targets.▼mediumTogether AIBasetenModal4 replies◆ premiumApply the project’s offline tuning guidance in context, derive the memory tradeoff, and use workload-specific acceptance criteria instead of universal queue and headroom targets.Open full answer →
28You serve a small, a medium and a large open-weights model. How do you route requests between them?▼mediumBasetenTogether AIModal4 replies◆ premiumSeparate pre-routing from a sequential cascade, charge every attempted generation, and measure checker mistakes. A lower escalation rate is not proof of higher quality, and the largest model is not a correctness oracle.Open full answer →
31The open-weights model you are deploying is multimodal. What changes about serving it?▼mediumBasetenTogether AIFireworks AI4 replies◆ premiumFollow media through fetching, preprocessing, encoding and language-model execution. Derive visual positions from the actual processor output, account for encoder placement and separate the caches and latency stages that text-only monitoring misses.Open full answer →
32Your agent product's tool-call success rate fell from 94 to 61 percent after a deploy. Debug it.▼mediumBasetenTogether AIModal4 replies◆ premiumA partial drop does not rule out the parser. Define success and its denominator, trace failures through encoding, generation, parsing and execution, then replay the relevant boundary with controlled versions.Open full answer →
34Every response must be valid JSON against a schema. How do you serve that at scale?▼mediumBasetenTogether AIFireworks AI4 replies◆ premiumConstrained decoding restricts valid continuations; it does not guarantee completion or correct values. Define success and error contracts, validate supported schemas, and measure compilation, masking and queueing under the actual workload.Open full answer →
36Three open-weights models could serve your product. How do you choose?▼mediumBasetenTogether AIModal4 replies◆ premiumScreen exact checkpoints for hard requirements, then compare deployable candidates on held-out product tasks, service limits and total cost. Smaller artifacts, leaderboard scores and token rates each answer only part of the decision.Open full answer →
39A replica restarts and loads different weights than its siblings. How did that happen?▼mediumBasetenTogether AIModal4 replies◆ premiumA mutable branch or local model directory can change what a restarted replica loads. Establish the actual artifact identity, separate disk evidence from loaded state, and make rollout and restart behavior reproducible.Open full answer →
41Your model's weights fit in GPU memory, but the engine cannot load them. How do you inspect the model package?▼mediumBasetenHugging FaceFireworks AI4 replies◆ premiumFollow a shard index to the tensors a loader actually receives. Diagnose a missing file and a shape mismatch, then check architecture, quantization and request compatibility before changing hardware.Open full answer →
44Which GPUs can run an FP8 or AWQ checkpoint, and what do you actually get on an older card?▼mediumNewNVIDIABasetenTogether AI2 replies◆ premiumMap a checkpoint format to compute capability, then say which part of the speedup survives on each generation. The same file can load on two GPUs and take two different execution paths.Open full answer →
46You need product copy for a 150-item catalog by tomorrow. Batch job or the online endpoint, and how do you deliver it?▼mediumNewBasetenModalTogether AI2 replies◆ premiumA measured run shows where the time goes in a small batch job, then why every result parsed but none was deliverable. The answer turns on startup cost, co-tenant latency and a per-item contract.Open full answer →
47Deploy one open-weights model on Kubernetes with vLLM. What goes in the manifest, and what breaks first?▼mediumNewBasetenModalTogether AI2 replies◆ premiumWalk a single-GPU vLLM Deployment from pod scheduling to the first token, then show the three settings that turn a working manifest into restart loops and stuck rollouts.Open full answer →
05How many H100s do you need to train a 70B model on 15 trillion tokens in 30 days?▼medium★ EssentialOpenAIAnthropicxAI4 repliesunlockedSolve for useful throughput, round to a supported parallel layout, and account for calendar time once. The right GPU count need not be a power of two.Open full answer →
07What does it cost per million output tokens to serve a 70B model on eight H100s?▼medium★ EssentialBasetenTogether AIFireworks4 repliesunlockedCompute cost from a matched bill and token count. Show when an activity factor belongs in the denominator and when it would double-count idle time.Open full answer →
08How much memory does it take to fine-tune a 70B model, full fine-tuning versus LoRA?▼mediumDatabricksAnyscale4 repliesunlockedSeparate frozen weights from trainable state, count rank-16 adapters from matrix shapes, and check peak memory before choosing a fine-tuning layout.Open full answer →
09A kernel does 4 TFLOP and moves 40 GB in one call. On an H100, is it memory-bound or compute-bound?▼mediumNVIDIAFireworks4 repliesunlockedTurn FLOPs and HBM bytes into resource bounds, then compare them with elapsed time. A tighter memory bound suggests an investigation rather than proving the measured bottleneck.Open full answer →
10How much network bandwidth does data-parallel training need?▼mediumMetaCrusoe4 repliesunlockedCount gradient traffic with the actual dtype, convert one-direction rates, and expose the bucket-readiness constraint that an average bandwidth division misses.Open full answer →
11How long does prefill take for an 8k-token prompt on a 70B model?▼mediumAnthropicFireworks4 replies○ sign inWork through projection and attention FLOPs for an 8,192-token prompt. Separate a peak-based lower bound, an assumed-rate estimate and observed TTFT.Open full answer →
12Estimate how many H100-hours it took to train Llama 3.1 405B, then check it against the paper▼mediumMeta4 replies○ sign inDerive GPU-hours from an explicit FLOP convention, compare Meta’s 30.84 million reported hours, and avoid mistaking the residual for a measured MFU.Open full answer →
15Estimate how long it takes to write a checkpoint for a 405B training run▼mediumMetaMicrosoft4 replies○ sign inInventory a resumable checkpoint instead of rounding its bytes per parameter. Trace snapshot stall, durable drain and publication, then calculate the storage floor.Open full answer →
16What is the power budget for a cluster of 1,024 H100s?▼mediumCrusoeCoreWeaveMicrosoft4 replies○ sign inBuild a power budget from complete servers to the facility meter, then check rack limits and distinguish a design maximum from an annual electricity bill.Open full answer →
18How much does moving from bf16 to fp8 save in serving cost?▼mediumFireworksTogether AI4 replies○ sign inCalculate FP8 savings by separating weights, KV cache and other step costs. Compare the same batch first, then check whether larger batches meet latency and quality targets.Open full answer →
19How long does an all-reduce of a 70B model's gradients take on eight GPUs?▼mediumNVIDIAMeta4 replies○ sign inDerive ring all-reduce traffic, match directional bytes to bandwidth, and check memory feasibility before estimating the communication exposed on a training step.Open full answer →
20How many tokens per second can one B200 decode for a 70B model?▼mediumNVIDIACoreWeave4 replies○ sign inCount weight and cache traffic on a specified B200 configuration, check finite capacity, and separate aggregate throughput from per-user pace. Worked bounds guide a benchmark rather than replace it.Open full answer →
26A startup wants to train a 7B model on 1 trillion tokens in three weeks. What cluster do they rent?▼mediumLambdaRunPodNebius4 replies◆ premiumCalculate the fleet, duration and rental budget for a 7B training run. Check the deadline, actual sharding scheme and paid reservation period before placing the order.Open full answer →
03InfiniBand or RoCE version 2 for a new GPU training cluster. Make the call and say what would reverse it.▼medium★ EssentialNVIDIACrusoexAI4 repliesunlockedChoose between supported InfiniBand and RoCE designs using workload tests, congestion behavior and a complete cost model.Open full answer →
05Explain a rail-optimized fabric. What does it buy over a plain fat tree, and how many switches does one need?▼medium★ EssentialNVIDIAMetaCoreWeave4 repliesunlockedFollow corresponding GPU positions through a rail, including the leaf and spine links needed when one rail outgrows a switch.Open full answer →
06A 64-node job's all-reduce runs at a third of the expected bandwidth. Walk me through finding the cause.▼medium★ EssentialCrusoeLambdaCoreWeave4 repliesunlockedA bandwidth regression narrows the search only after the experiment is defined. Work through rank-count normalization, controlled path tests and the difference between collective time and training progress.Open full answer →
09When does GPUDirect Storage actually help, and when is it just a more complicated read?▼mediumNVIDIAWEKA4 repliesunlockedCompare serialized and overlapped storage paths, then test whether direct GPU I/O improves the actual pipeline stage.Open full answer →
10Parallel filesystem or object storage for training data? Make the call and explain the two-tier pattern most clusters end up with.▼mediumCrusoeMetaDatabricks4 repliesunlockedDesign a durable dataset and optional working-set cache around access semantics, request overhead and versioned readiness.Open full answer →
11The GPUs are idle between steps and the profiler says the data loader. Find the actual constraint and fix it.▼medium★ EssentialMetaAnthropicDatabricks4 replies○ sign inMeasure batch readiness and producer capacity, then size prefetch for bursts without hiding a sustained throughput deficit.Open full answer →
15Which NCCL environment variables are worth knowing, what does each change, and which should you almost never set?▼mediumNVIDIACoreWeave4 replies○ sign inSeparate NCCL logging, device selection and execution overrides from PyTorch failure handling. Work through a faster collective that makes the job slower, and learn which timeout actually fired.Open full answer →
16How do you establish that the link between two GPU nodes is healthy, and what numbers should each test return?▼mediumLambdaCrusoe4 replies○ sign inSeparate host-buffer, GPU-buffer and collective tests, with explicit message-size models and bandwidth metric definitions.Open full answer →
17Why do large training clusters provision roughly 400 gigabits per second per GPU rather than more or less?▼mediumNVIDIAMetaxAI4 replies○ sign inDerive bandwidth from actual per-rank traffic and readiness deadlines, showing why bucketing cannot erase a capacity shortfall.Open full answer →
20Does multi-node inference need the same fabric as training, or can it run on ordinary networking?▼mediumTogether AIBaseten4 replies○ sign inCompare tensor, pipeline and expert traffic with per-request KV transfer. Work through declared payload and latency assumptions before choosing a serving network.Open full answer →
22Why does object storage behave badly for random reads, and what does a training pipeline do about it?▼mediumDatabricksSnowflake4 replies◆ premiumCalculate how many reads a training loader needs in flight, then see when larger requests waste bandwidth. Worked examples separate useful samples, fetched bytes, prefetch and the limits of Little's Law.Open full answer →
25Checkpoints and datasets both live on storage. Why does one system sized for both usually serve neither well?▼mediumCrusoeWEKA4 replies◆ premiumCan one storage system serve datasets and checkpoints? Follow a save from staging to recovery, calculate its budget beside protected reads, and learn when separate paths actually solve the contention.Open full answer →
28How would you know GPUDirect RDMA is not being used, given that nothing reports an error when it is disabled?▼mediumNVIDIALambda4 replies◆ premiumDistinguish GPU-memory capability, registration and selected transport. Compare direct and staged paths without relying on a half-speed rule or one log substring.Open full answer →
03Slurm or Kubernetes for a 2,000-GPU training cluster? Make the case, and tell me what you lose either way.▼medium★ EssentialCoreWeaveCrusoeNebius4 repliesunlockedChoose the control plane from job admission, placement, recovery and service needs, with a concrete partial-allocation failure.Open full answer →
05Design a GPU-aware scheduler that supports fractional GPUs: what isolation does each fraction get, and where does it break?▼medium★ EssentialTogether AICoreWeave4 repliesunlockedDesign fractional GPU admission around explicit shapes and enforceable guarantees, then reconcile reservations with the runtime.Open full answer →
06Two identical 8-GPU jobs get 8 GPUs each. One runs at half the speed of the other. What did the scheduler do, and how do you stop it?▼mediumNVIDIACoreWeave4 repliesunlockedUse actual rank placement and measured job duration to decide whether waiting for locality beats starting immediately.Open full answer →
07Eight research teams share 1,024 GPUs. Design the quota and fairness policy, and tell me how they will game it.▼mediumOpenAIAnthropicMeta4 repliesunlockedSeparate quota, borrowing, reclaim latency and fairness, with a demand-conserving example of when pooling improves utilization.Open full answer →
08Here is a Kubernetes GPU node. Pods requesting nvidia.com/gpu stay Pending. Fix it in front of me and narrate what you check.▼medium★ EssentialNebiusCoreWeave4 repliesunlockedDiagnose GPU pod failures by scheduling, container startup and execution boundaries, without treating Allocatable as a health certificate.Open full answer →
12Your fine-tunes run on spot GPUs preempted about once every four hours. How often should they checkpoint, and when does spot stop paying?▼mediumAnyscaleLambdaRunPod4 replies○ sign inBalance durable checkpoint cost against replay and restart loss, with explicit assumptions and a warning sign when the approximation breaks down.Open full answer →
15What is Ray on Kubernetes good for, and where do its scheduler and the Kubernetes scheduler fight each other?▼mediumAnyscaleDatabricks4 replies○ sign inTrace Kubernetes pod placement and Ray bundle reservation as separate layers, including logical GPU limits and autoscaling delay.Open full answer →
16Our GPU pods take four minutes to start on a fresh node and sometimes fail with 'no CUDA-capable device'. Walk me through both.▼mediumModalBaseten4 replies○ sign inMeasure scheduling, image transfer, unpacking and model initialization separately. A missing-device error needs its own checks for allocation, container access and CUDA compatibility.Open full answer →
17We rent GPUs. When should we buy committed capacity instead of paying on demand, and what do we do with the rest of the demand?▼mediumCoreWeaveLambdaAWS4 replies○ sign inCompare the cost of committed capacity with the demand each additional GPU would actually serve. Keep billing discounts separate from capacity guarantees, and account for bursts without counting the same work twice.Open full answer →
19Kubernetes device plugins versus Dynamic Resource Allocation: what changes for GPU scheduling, and what would you adopt in 2026?▼mediumNVIDIAGoogle4 replies○ sign inDRA gives the scheduler device claims and attributes to work with. Compare that with extended-resource counts, then check the deployed driver and controller support before promising sharing, partitioning or gang placement.Open full answer →
20What do you run on a GPU node before you let a job land on it, how long does it take, and what happens on failure?▼mediumCoreWeaveCrusoeLambda4 replies○ sign inUse a short admission check for recent changes and a deeper acceptance suite for new or repaired nodes. Set thresholds from matched healthy baselines, and make quarantine and return-to-service decisions explicit.Open full answer →
22Explain how a Slurm job actually gets its GPUs: GRES, partitions, cgroups, and the sbatch that lands 8 GPUs on one node.▼mediumCrusoeLambda4 replies◆ premiumFollow a GPU request from node inventory through allocation, device confinement and task launch. A single-node, eight-task example shows how resource shape and per-task device numbering fit together.Open full answer →
23We want to run LLM inference on Kubernetes. What does the deployment actually need beyond a Deployment with a GPU limit?▼mediumBasetenRed Hat4 replies◆ premiumMake readiness mean that the engine can serve, scale from measured demand and ready capacity, and drain streams before termination. Kubernetes can implement these controls; an operator packages and reconciles them.Open full answer →
27Design a notebook platform for 300 researchers on 64 GPUs. How do you share, reclaim and account for the GPUs?▼mediumDatabricksGoogle4 replies◆ premiumSize notebook capacity from concurrent demand, memory needs and burst tolerance. Choose a sharing mode for the trust model, and make idle release explicit about which state can actually be restored.Open full answer →
04Design observability for a large training cluster. What do you collect, what does each signal answer, and what pages someone?▼medium★ EssentialAnthropicOpenAIMeta4 repliesunlockedBuild observability around durable useful progress and aligned phase timing, showing why full-step timers can hide the late rank.Open full answer →
05One GPU's correctable memory error rate has been climbing for a week. What does that predict and what do you do about it?▼mediumMetaCoreWeave4 repliesunlockedInvestigate rising corrected-memory errors without promising a failure prediction, and follow recovery flags and vendor repair criteria.Open full answer →
06A node reports a GPU has fallen off the bus. What happened, what can software do, and what should the platform do automatically?▼mediumLambdaCoreWeave4 repliesunlockedHandle an inaccessible GPU with bounded recovery and preserved evidence, without treating a reboot or repeat count as proof of root cause.Open full answer →
08One node in a job runs at half the speed of its peers. What do you check, in what order, and what does each answer rule out?▼mediumCrusoeLambda4 repliesunlockedFind the critical-path delay using local phase timing and controlled comparisons, rather than diagnosing a node from utilization or clock ratios.Open full answer →
09At 16,384 GPUs something fails every three hours. How often should you checkpoint, and what goodput does that leave?▼medium★ EssentialMetaMicrosoft4 repliesunlockedDerive a checkpoint interval from measured job interruptions and exposed save cost. Trace durability, lost work and recovery before treating an estimate as goodput.Open full answer →
10What service level objectives would you promise researchers on a training platform, and how would you measure each?▼mediumGoogleMetaOpenAI4 repliesunlockedPromise measurable job outcomes: time to useful work, recoverable progress and successful restoration. Define the population and exclusions before choosing targets, and give each error budget an agreed operating policy.Open full answer →
11Define the service level objectives for an LLM serving fleet, and the alerting that tells you when one is about to be missed.▼mediumOpenAIAnthropicBaseten4 replies○ sign inDefine request success, first-output latency and streaming continuity for each service class. Count bad outcomes against an explicit budget, then use paired long and short windows to detect significant ongoing degradation.Open full answer →
12A new GPU node arrives. What do you run before it serves a customer, and what does each test actually catch?▼mediumCoreWeaveLambdaCrusoe4 replies○ sign inBuild an acceptance suite around the node’s intended workload and supported hardware limits. Combine configuration, correctness, communication and sustained-load evidence, then record a baseline and an explicit return-to-service decision.Open full answer →
13How would you detect that GPUs are thermally throttling, and what is the right response when they are?▼mediumCoreWeaveCrusoe4 replies○ sign inCorrelate clock-event reasons, temperature, configured limits and workload timing. Use the scope of the slowdown to choose the next investigation, and distinguish intentional power limits from unexpected thermal constraints.Open full answer →
16You own the on-call rota for a GPU fleet. What is allowed to wake someone at 3am, and what must not?▼mediumCoreWeaveModal4 replies○ sign inPage when material impact or risk needs timely human action. Verify that automation actually recovered the workload, group related events, and use measured responder load to improve the policy.Open full answer →
18Error rate on an inference fleet tripled ten minutes after a deploy. What do you do first, and what should have caught it?▼mediumBasetenOpenAI4 replies○ sign inStop rollout expansion, preserve evidence and use the safest available mitigation. Compare versions under representative load, and size canary exposure from the regression you need to detect rather than a fixed traffic percentage.Open full answer →
19How do you measure effective training time, and where does the missing ten percent of a well-run cluster actually go?▼mediumMetaAnthropic4 replies○ sign inDefine which work survived, count the resources it occupied, and explain the remaining time without counting the same delay twice.Open full answer →
23Design the alerting rules for a GPU platform so that a page is always worth waking for. What are the mechanics?▼mediumCoreWeaveModal4 replies◆ premiumPage when timely human action matters. Use budget burn, careful grouping, bounded maintenance rules, and regular testing to keep those pages useful.Open full answer →
25Trace one request through an inference platform. What are the spans, what does each cost, and where does latency actually hide?▼mediumOpenAIBaseten4 replies◆ premiumFollow a request from the client's connection pool to the first text on screen. Work the timing budget, locate a buffering delay and choose what to trace without losing the slow requests.Open full answer →
26One serving replica has a per-token latency 40 percent worse than its peers. Find out why.▼mediumBasetenTogether AI4 replies◆ premiumCompare equivalent requests, inspect the replica’s software and device state, and reduce new traffic only after checking where that load will go.Open full answer →
29How would you build a postmortem practice that people take seriously and that actually reduces incidents?▼mediumGoogleAnthropic4 replies◆ premiumBuild an incident-review process that leads to verified changes. Set review thresholds, budget the work and interpret recurrence without mistaking a metric for proof.Open full answer →
03Design an LLM batching system end to end: the queue, the batch, the KV cache and streaming. Give me numbers.▼medium★ EssentialAnthropic4 repliesunlockedTrack request admission through a concrete page-reservation example, then connect scheduling, growth and stream cleanup to measured service limits.Open full answer →
04Design a GPU credit system: accounts, spending rates, priorities and fairness. Start with the data model.▼medium★ EssentialOpenAI4 repliesunlockedTrace GPU credits through reservation, allocation, metering and settlement, with crash-safe ownership and a separate fairness policy.Open full answer →
09Design the eval pipeline for a frontier model: thousands of evals per checkpoint, sharded inference, caching, reproducible results.▼mediumOpenAIAnthropic4 repliesunlockedFour result rows can still hide a missing case. Build immutable evaluation identities, retry-safe commits, versioned scoring and a capacity estimate tied to measured prompt throughput.Open full answer →
11Design serving for 100 fine-tuned variants of one 70B base. Multi-LoRA on shared replicas or a replica per variant?▼mediumAnyscaleFireworksDatabricks4 replies○ sign inDerive adapter bytes from the target matrices, share compatible base weights, and keep each request's cache identity separate. A worked pool design avoids applying the mixed-adapter cost twice.Open full answer →
12Design autoscaling for an LLM service whose traffic triples within a minute. What signal, what cold-start budget, what warm pool?▼mediumBasetenModal4 replies○ sign inA 60-second demand ramp meets a 70-second startup delay. Compare the rate and ready-capacity curves, then design a controller that tracks pending launches without a blind cooldown.Open full answer →
14Design the registry and deploy pipeline for model weights: 141 GB artifacts, versioning, distribution to a thousand nodes, integrity.▼mediumDatabricksHugging Face4 replies○ sign inBind weights, tokenizer, configuration and evaluation to one immutable release. Follow verification, prefetch, canary readiness and rollback, with bandwidth floors that do not pretend to be rollout measurements.Open full answer →
15Design rate limiting for an LLM API. Why tokens instead of requests, and how does a bucket work when the cost is unknown until the end?▼mediumOpenAIAnthropic4 replies○ sign inDivide one token budget across gateways instead of copying it. Follow output reservation and settlement, then handle grants, retries and outages without inventing extra burst capacity.Open full answer →
17Design the playground behind an LLM API: streaming, sessions, quotas and abuse, for a million free-tier users.▼mediumOpenAI4 replies○ sign inSeparate durable conversation state from live generation, bound streaming buffers, and make reconnect, cancellation and free-tier budgets explicit.Open full answer →
19Design observability for an inference fleet: per-request spans, GPU metrics, and cost attribution per tenant.▼mediumBasetenOpenAI4 replies○ sign inTrace a stalled stream without unbounded telemetry, understand what sampling loses, and reconcile GPU allocation with a stated cost policy.Open full answer →
25Design quota and fairness for a shared research cluster: hierarchical quotas, preemption, and the incentives that keep it honest.▼mediumOpenAIAnthropicMeta4 replies◆ premiumA team returns to claim its GPUs, but another team is borrowing them in one indivisible job. Work through the quota tree, whole-job reclaim, recovery costs and release-to-ready budget before promising a start time.Open full answer →
27Design token-level billing and metering for an LLM API: count at the engine, aggregate, reconcile, and survive the failure modes.▼mediumOpenAITogether AI4 replies◆ premiumTrace usage from execution to one settled charge. Distinguish requests, attempts and events, then handle cancellation, late records, replay and reconciliation.Open full answer →
29Design an internal LLM gateway for a company: routing across providers and models, caching, fallbacks, and cost controls.▼mediumDatabricksSnowflake3 replies◆ premiumDefine model aliases, permission-aware routing, response reuse and bounded retries. Trace a request whose cached answer becomes invalid after its source permissions change.Open full answer →
05Implement a key-value store where a read can ask for the value as of an earlier version. What is the data structure?▼medium★ EssentialOpenAI4 repliesunlockedPer key, an ascending list of versions and a parallel list of values, with reads doing a binary search. The three cases that decide whether the design is right: a key that did not exist yet, a key that was deleted, and a version at which some other key was written.Open full answer →
06Implement a batcher that flushes when the batch is full or when a timeout expires. What breaks in the timer path?▼medium★ EssentialBasetenTogether AIAnthropic4 repliesunlockedImplement size- and timer-triggered batching, then test cancellation, handler errors and the latency added before dispatch.Open full answer →
07Implement a scheduler that admits jobs by priority and preempts lower-priority work when it must. What are the rules?▼mediumOpenAIAnyscale4 repliesunlockedImplement priority admission and completion, preserve a preempted job’s sequence, and distinguish a sufficient greedy victim set from an optimal one.Open full answer →
08You are handed a 40 GB kernel trace. Write the parser that turns it into per-instruction access statistics without running out of memory.▼medium★ EssentialNVIDIA4 repliesunlockedStream a trace with an explicit record format, bounded access groups and measured aggregation size. Preserve group identity and report rejected data.Open full answer →
09Implement a cache for model weights on a serving node. What makes it different from a normal LRU?▼mediumModalBaseten4 repliesunlockedBuild a byte-budget cache with reference-counted pins and admission that preserves existing entries when it fails.Open full answer →
10Write a producer-consumer pipeline with a bounded queue. What are the three bugs that show up in every first attempt?▼mediumAnthropicOpenAI4 repliesunlockedImplement a bounded producer-consumer pipeline with explicit shutdown and error reporting. Keep work outside collection locks and account for memory beyond the input queue.Open full answer →
11Place GPU jobs onto nodes. Compare first fit, best fit and worst fit, and say which one a training cluster wants.▼mediumCoreWeaveTogether AI4 replies○ sign inCompare three GPU placement heuristics with executable traces. Show how admission and free-space shape change, then add workload and topology constraints.Open full answer →
12Tokens arrive as byte fragments and a character can span several. Write the decoder that streams text correctly.▼mediumOpenAI4 replies○ sign inDecode fragmented UTF-8 with an incremental codec. Test malformed bytes, truncated endings and the boundary between byte decoding and event parsing.Open full answer →
13Simulate a ring all-reduce on arrays. Verify both the result and the bytes each rank sends.▼mediumNVIDIAGoogle4 replies○ sign inSimulate both phases of ring all-reduce and verify values, preserved inputs and each rank’s payload bytes.Open full answer →
16In Go, download many model shards concurrently with a concurrency limit, a timeout and clean cancellation. Write it.▼mediumCoreWeaveCrusoe4 replies○ sign inWrite a bounded Go downloader that propagates cancellation, joins every worker and publishes files under explicit unique names.Open full answer →
19Compute per-tenant GPU-hours for a billing period from a log of allocation start and stop events.▼mediumCoreWeaveModal4 replies○ sign inReconstruct GPU-hour intervals, clip them to a period and preserve provisional usage and conflicting-event evidence.Open full answer →
20Implement a circuit breaker in front of a model endpoint. What are the states, and what does the half-open state buy?▼mediumBasetenOpenAI4 replies○ sign inImplement bounded recovery probes, state-generation tickets and a recent failure window. Test concurrency and late completions.Open full answer →
21Given per-rank heartbeats from a training job, detect which rank has stopped and when. What produces false positives?▼mediumMetaAnthropic4 replies◆ premiumDetect missing and silent ranks from a roster, distinguish liveness from progress, and derive thresholds without assuming independent delays.Open full answer →
22Write a thread-safe counter in C++ that many threads increment. What are the memory-ordering and layout choices?▼mediumNVIDIACerebras4 replies◆ premiumImplement shared and sharded C++ counters, explain relaxed ordering and test final counts without promising unmeasured scaling.Open full answer →
23Why is std::unordered_map rarely the fastest hash map, and what would you use instead?▼mediumCerebrasNVIDIA4 replies◆ premiumCompare standard-map guarantees with flat-table layouts. Implement bounded probing and choose from semantics and representative measurements.Open full answer →
25Simulate speculative decoding to find the expected tokens per round and the batch size where it stops paying.▼mediumTogether AI4 replies◆ premiumDerive and simulate speculative output counts, then compare them with explicitly assumed or measured round costs.Open full answer →
26Design the API for a GPU job scheduler in 45 minutes. What are the resources, states and semantics?▼medium★ EssentialOpenAIAnthropicCoreWeave4 replies◆ premiumDesign job creation, cancellation and listing around retries, durable ownership and an explicit pagination contract.Open full answer →
28Write a tiled matrix multiply and explain the cache effect. Then say honestly what your benchmark actually measured.▼mediumNVIDIAGoogle4 replies◆ premiumWrite and verify tiled NumPy matmul, derive its logical footprint and distinguish a cache model from the causes of measured runtime.Open full answer →
04A researcher needs 256 GPUs today and the cluster is full. How do you handle it?▼medium★ EssentialOpenAIAnthropicGoogle DeepMind4 repliesunlockedClarify what the 256-GPU request must accomplish, then compare it with real placement and quota constraints. Offer a feasible alternative and explain who can decide when teams’ priorities conflict.Open full answer →
05Describe a migration nobody wanted. How did you get it done without losing the team's trust?▼mediumMetaDatabricksGoogle4 repliesunlockedExplain the reason for an unpopular migration, choose informative rollout cohorts and describe recovery, communication and transition cost without inventing an incident.Open full answer →
06What is your view on AI safety, and what does it actually mean for infrastructure work?▼medium★ EssentialAnthropicOpenAIGoogle DeepMind4 repliesunlockedConnect your view on AI safety to systems you could build: model access, audit evidence, evaluation capacity and a tested way to stop or reverse a release. Explain the costs and limits of each control.Open full answer →
08Your team is drowning in manual work. How do you decide what to automate first?▼mediumCoreWeaveGoogleModal4 repliesunlockedCompare net time saved, maintenance and failure risk before automating. Work a payback example and explain why concurrency limits alone do not bound repeated damage.Open full answer →
09Describe a time you were confidently wrong about a root cause. What did it cost?▼mediumMetaAnthropicCoreWeave4 repliesunlockedExplain how a plausible diagnosis became too certain, what disproved it and what changed in your investigation. Separate observed symptoms from causal evidence.Open full answer →
10You are joining a team running a 10,000 GPU cluster. What do you do in your first month?▼mediumMetaxAIAnthropic4 repliesunlockedPlan a useful first month by tracing a real job, reviewing incidents, helping under supervision and shipping a bounded improvement with the team.Open full answer →
11Two teams both need the same capacity and both have escalated. How do you resolve it?▼mediumOpenAIGoogleMeta4 replies○ sign inSeparate a disagreement about measurements from a real capacity shortage. Work a reclaim example, expose the missing reserve and escalate an authorized decision.Open full answer →
14Tell me about a cost reduction you led. How did you prove reliability did not suffer?▼mediumBasetenCoreWeaveModal4 replies○ sign inShow a cost reduction with comparable service evidence, realized savings and a feasible recovery plan. Distinguish a quiet trial from proof of unchanged reliability.Open full answer →
15How do you run a postmortem that actually changes something?▼mediumGoogleAnthropicCoreWeave4 replies○ sign inUse an evidence-based timeline, specific contributing conditions and verified follow-up to make a postmortem useful.Open full answer →
18A vendor says your configuration is the problem. You think it is their firmware. How do you proceed?▼mediumCoreWeaveLambda LabsNVIDIA4 replies○ sign inMove a vendor case forward with controlled comparisons, a faithful reproduction and a safe mitigation. Escalate by impact while evidence improves.Open full answer →
19How do you bring a junior engineer through their first serious incident?▼mediumGoogleMetaAnthropic4 replies○ sign inSupport a new responder without turning an outage into an exam. Choose an appropriate role, explain handoffs and review the decisions afterward.Open full answer →
21Three incidents are open, your pager is going off, and a customer is escalating. What do you do first?▼mediumCoreWeaveModalBaseten4 replies◆ premiumCoordinate simultaneous incidents with the people available. Contain urgent harm, assign next actions and revisit priorities as evidence changes.Open full answer →
22Tell me about a decision you had to make without enough information.▼mediumOpenAIMetaAnthropic4 replies◆ premiumExplain a decision under uncertainty through the cost of waiting, bounded exposure and evidence that changes the next action.Open full answer →
23You think your tech lead's design is wrong. How do you handle it?▼mediumGoogleNVIDIAMeta4 replies◆ premiumTurn a design disagreement into a concrete comparison. Use a gang-scheduling example to explain evidence, decision ownership and responsible follow-through.Open full answer →
25Why would you leave a hyperscaler for a GPU cloud, or a GPU cloud for a hyperscaler?▼mediumCoreWeaveGoogleAWS4 replies◆ premiumExplain a career move using the actual role, team and your priorities. Test assumptions about ownership, hardware access and support.Open full answer →
26Describe something you built that made researchers meaningfully more productive.▼mediumAnthropicMetaGoogle DeepMind4 replies◆ premiumConnect a researcher tool to the workflow it improved. Measure the whole loop, meaningful adoption and effects on other users.Open full answer →