Advanced

Reranking and Retrieval Budgets

A reranker spends extra computation on retrieved candidates. Count query-document pairs, protect permission boundaries and measure whether better evidence justifies its added queueing and latency.

This concept is part of Premium, which includes all concepts, course lessons and answers.

Get Premium · ₹2,000 / $256 months of access · one payment · no auto-renew

RELATED CONCEPTS

Embedding Services and Vector Retrieval Evaluation Metrics and Data Splits Capacity and Backpressure Latency Metrics: TTFT, TPOT and Goodput Model Metadata and Artifact Storage

LESSONS THAT TEACH THIS

From Model to ServiceBuild retrieval as a budgeted pipeline22 min→

PRACTICE THIS IN REAL QUESTIONS

AI Infrastructure System DesignDesign a distributed search system with an LLM answer layer at 10,000 queries per second. Size both tiers and name the SLOs.→Open-Weights Models & Serving EnginesWould you split prefill and decode onto separate pools for a 753B model? Work the decision.→Open-Weights Models & Serving EnginesSet max-model-len and max-num-seqs for a chat product from first principles.→Open-Weights Models & Serving EnginesDesign a deployment that serves a trillion-parameter model at a million tokens of context with usable latency.→Open-Weights Models & Serving EnginesThree open-weights models could serve your product. How do you choose?→AI Infrastructure System DesignDesign a multi-region inference deployment: capacity per region, routing, failover, and getting the weights everywhere.→