mediumNewCerebrasNVIDIA

Why is std::unordered_map rarely the fastest hash map, and what would you use instead?

The standard forces one heap allocation per entry and a pointer dereference per lookup, and that is a specification requirement rather than a library weakness. The measured cost on this machine, the open-addressing layout that removes it, and the two cases where the standard container is still the right pick.

Updated Sep 2026 · Grounded in real AI infrastructure interview loops and written to a senior-engineer editorial bar, with every number worked and every diagram hand-built.

20 answers per topic instead of 10, plus saved progress and bookmarks · no cardor unlock all 283 remaining answers · ₹2,000 / $25

The concepts behind this question

Ranked by how closely each one overlaps this question's topic, so the first card is the thing to read if the answer above moved too fast.

Foundational

💻 Coding for Infra

Cache-Friendly Data StructuresA cache line is 64 bytes and it is the unit of coherence, so where data sits decides how fast code runs more often than which algorithm it uses. Two consequences dominate infrastructure code: a lookup that chases a pointer pays two dependent memory stalls instead of one, and two threads updating adjacent variables contend for a line they do not logically share. Both are layout problems with layout fixes.

Foundational

🧩 GPU & Accelerator Architecture

Roofline ModelThe roofline plots a kernel's attainable throughput against its arithmetic intensity, FLOPs per byte moved from memory. Below the ridge point (peak FLOPS divided by memory bandwidth, about 295 on an H100 in bf16) a kernel is memory-bound and no amount of clever code reaches the peak; above it, compute is the limit. One picture explains why decode runs at under 1% of peak and why fusion and batching are the two levers that move it.

Foundational

💻 Coding for Infra

The Practical Coding Screen PlaybookThe AI infrastructure coding screen is 45 to 60 minutes of building a small, realistic piece of systems code (a scheduler, a rate limiter, a batcher, a log parser, a cache) in the language you choose, with an interviewer who extends the problem twice and watches how you handle it. It is not a puzzle round: the score comes from working code early, tests that name the invariants, complexity said out loud, and calm follow-ups. Some companies allow an AI assistant and some ban it, and each policy changes what is measured. This page gives the minute-by-minute plan, the habits that score, and the mistakes that end the screen.

Foundational

💻 Coding for Infra

The GPU Credit Scheduler PatternThe most widely reported coding problem in AI infrastructure loops is a small scheduler: accounts hold credits, jobs arrive with a cost and a priority, and you must decide which jobs run, in what order, without letting any account overspend, then extend it under follow-ups (refunds, reservations, concurrency limits, fairness). It is not a trick question; it is a test of whether you can model state cleanly, pick the right data structures, keep invariants under mutation, and talk about complexity while typing. This page works the problem from the first line to the fourth follow-up, with the code, the invariants, and the derivations.

Explore the full curriculum →

UP NEXT ON YOUR JOURNEY

Next in this trackImplement a work-stealing deque. Why do the owner and the thieves take from opposite ends?Next in this trackSimulate speculative decoding to find the expected tokens per round and the batch size where it stops paying.Next in this trackDesign the API for a GPU job scheduler in 45 minutes. What are the resources, states and semantics?

FEDITOR'S NOTE

Scored on naming the iterator-stability requirement as the cause rather than calling the implementation slow, on the cache-miss arithmetic, and on saying when the standard container wins.

DISCUSSION · 0

No comments yet — be the first to share your approach.