Select the top k logits from a 128,000-token vocabulary on the GPU. What shape does the kernel take and why not just sort?

Choose between local candidates, radix selection and sorting from k and the required output. A tested FP32 reference makes ties, signed zero, NaNs and threshold selection explicit.

Updated Sep 2026 · Learn how AI infrastructure works through explanations, worked examples and diagrams, then practise applying it to interview questions.

Choose between local candidates, radix selection and sorting from k and the required output. A tested FP32 reference makes ties, signed zero, NaNs and threshold selection explicit.

The full answer is part of Premium. A free account includes more answers in each topic, but does not unlock this one.

20 answers per topic instead of 10, plus saved progress and bookmarks · no cardor unlock all 316 remaining answers · ₹2,000 / $25

Work through the numbers

Change the assumptions in a free calculator, then explain what the result establishes and what still needs measurement.

Roofline calculator

The concepts behind this question

Use these concept explanations to review the ideas behind this question.

Advanced

⚡ Kernels & Compilers🔒 Premium

Tiled Matrix MultiplicationTiled matrix multiplication reuses input values across many output calculations. Shared-memory and register tiles reduce repeated data movement while consuming limited on-chip resources.

Advanced

⚡ Kernels & Compilers🔒 Premium

Shared Memory and Bank ConflictsShared memory lets threads in a block exchange and reuse data. Bank conflicts occur when an access needs multiple distinct words from the same bank.

Advanced

🚀 Inference & Serving🔒 Premium

Tokens, Embeddings and LogitsTrace token IDs, embedding rows and next-token scores. Work out which tensors grow with a batch, when all-position logits waste memory, and why token probabilities do not establish factual confidence.

Foundational

🧩 GPU & Accelerator Architecture

GPU Memory HierarchyTrace storage scope and data reuse from registers through shared memory, caches and HBM. Worked register and traffic examples explain why a source-level load is not automatically an HBM transfer.

Explore the full curriculum →

UP NEXT ON YOUR JOURNEY

Next in this trackYou added three local variables to a working kernel and it got 30 percent slower. Explain what happened and how you would confirm it.Next in this trackA training step runs at 20 percent model FLOPs utilization. Profile it and find where the missing time goes.Next in this trackPort a hand-written CUDA kernel to MI300X. What translates mechanically, and what silently computes the wrong answer?

FEDITOR'S NOTE

Define ordering and edge cases before selecting an algorithm. Explain why local top-k candidates suffice, what radix passes do and why a cache-capacity estimate does not establish the bottleneck.

DISCUSSION · 0

No comments yet — be the first to share your approach.