Every response must be valid JSON against a schema. How do you serve that at scale?

Constrained decoding restricts valid continuations; it does not guarantee completion or correct values. Define success and error contracts, validate supported schemas, and measure compilation, masking and queueing under the actual workload.

Updated Sep 2026 · Learn how AI infrastructure works through explanations, worked examples and diagrams, then practise applying it to interview questions.

The full answer is part of Premium. A free account includes more answers in each topic, but does not unlock this one.

20 answers per topic instead of 10, plus saved progress and bookmarks · no cardor unlock all 316 remaining answers · ₹2,000 / $25

Work through the numbers

Change the assumptions in a free calculator, then explain what the result establishes and what still needs measurement.

The concepts behind this question

Use these concept explanations to review the ideas behind this question.

Advanced

🚀 Inference & Serving🔒 Premium

Structured Outputs and Constrained DecodingConstrained decoding rules out tokens that break a grammar. Follow a two-choice example through masking, completion and rollback, then separate valid structure from correct, authorized tool arguments.

Foundational

🧮 Open Weights & Serving Engines

Model Onboarding: From Hugging Face to ProductionTurn a model release into a reproducible service: pin the complete artifact, confirm the exact engine path, budget transfer and runtime memory, validate request and response behavior, then benchmark and canary against explicit acceptance criteria.

Foundational

🧮 Open Weights & Serving Engines

Serving Benchmarks That Do Not LieDesign reproducible serving experiments with realistic arrivals, cache state and failure accounting. Interpret latency-qualified throughput and conditional per-rank bounds without turning optimistic kernel estimates into production capacity.

Advanced

🧮 Napkin Math & Capacity🔒 Premium

Bandwidth-Bound Decode ThroughputEstimate decode throughput from bytes per step, then separate aggregate output from per-user pace. A finite-memory example shows how batching, cache reads and quantization change the bound.

Explore the full curriculum →

UP NEXT ON YOUR JOURNEY

Next in this trackOne tenant's traffic is pushing everyone else past their latency objective. What do you do?Next in this trackThree open-weights models could serve your product. How do you choose?Next in this trackThe engine logs say CUDA graphs are disabled for your deployment. Does it matter?

FEDITOR'S NOTE

Scored on bounded validity guarantees, explicit incomplete/error outcomes, grammar-backend support, cache and CPU capacity arithmetic, streaming semantics and semantic evaluation.

DISCUSSION · 0

No comments yet — be the first to share your approach.