← Coding for Infra / 31

hardNew

Implement byte-pair encoding merges, then explain why token counts differ across languages

Apply a learned merge order without retraining on the input. Test overlapping pairs, preserve UTF-8 bytes, and turn tokenizer differences into a capacity estimate.

Updated Sep 2026 · Learn how AI infrastructure works through explanations, worked examples and diagrams, then practise applying it to interview questions.

Apply a learned merge order without retraining on the input. Test overlapping pairs, preserve UTF-8 bytes, and turn tokenizer differences into a capacity estimate.

The full answer is part of Premium. A free account includes more answers in each topic, but does not unlock this one.

20 answers per topic instead of 10, plus saved progress and bookmarks · no cardor unlock all 303 remaining answers · ₹2,000 / $25

The concepts behind this question

Use these concept explanations to review the ideas behind this question.

🔌 Networking & Storage🔒 Premium

Data Loading Pipelines for TrainingTrace batches from storage to GPU readiness, budget workers and prefetch memory, and resume from the input position saved with training state. An executable example shows how a read-ahead cursor can silently skip samples.

🔌 Networking & Storage

Dataset Lifecycle: Ingest, Shard and RetainFollow a dataset from source records to verified shards, a pinned version and safe retention. Calculate token storage, distinguish caches from authoritative data, and learn why deleting a version does not necessarily free its shards.

📐 AI Systems Design🔒 Premium

Evaluation and Data Pipeline InfrastructureData pipelines prepare traceable training inputs; evaluation pipelines compare model behavior. Both need versioned inputs, reproducible execution and useful failure reporting.

🧮 Napkin Math & Capacity🔒 Premium

Capacity Planning and UtilizationCapacity planning connects demand, useful throughput and acceptable waiting time. Serving and training need different models of bursts, job shape and spare capacity.

Explore the full curriculum →

UP NEXT ON YOUR JOURNEY

Next in this trackImplement beam search without losing finished hypotheses or corrupting KV ownership Next in this trackImplement a LoRA forward pass and merge two adapters without inventing cross terms Next in this trackImplement scalar autograd and prove that a shared input receives every gradient contribution

DISCUSSION · 0

No comments yet — be the first to share your approach.