consistent hashing
AI infra interview questions tagged consistent hashing, across every topic.
2 questions · 0 unlocked for you
Concepts behind "consistent hashing"
The curriculum that explains the ideas these questions test.
Foundational
Consistent Hashing and ShardingSplitting work across N servers with a modulo of N moves almost everything when N changes, which for a cache means throwing away almost all of it. Consistent hashing places servers and keys on a ring so adding or removing one moves only its share, and virtual nodes fix the imbalance a small ring otherwise has. In LLM serving the same structure routes requests by prompt prefix so a conversation reaches the replica already holding its cache.💻 Coding for Infra
Advanced
Request Routing and Load Balancing for LLMsA load balancer for stateless web services spreads requests evenly and is done. A router for LLM replicas has two things a web balancer never had to think about: each replica holds a cache (the KV pages of recent prefixes) that makes some replicas far cheaper than others for a given request, and each request costs a wildly different amount, so counting connections is meaningless. This page builds the router that handles both: prefix-aware placement with load-aware fallback, cost-aware queue estimates, session affinity, and the failure handling when a replica restarts and its cache is gone.📐 AI Systems Design🔒 Premium