← Coding for Infra / 34

hardNew

Implement scalar autograd and prove that a shared input receives every gradient contribution

Build a reverse-mode graph for addition and multiplication. Use a repeated input to expose overwritten gradients, then check the derivative numerically.

Updated Sep 2026 · Learn how AI infrastructure works through explanations, worked examples and diagrams, then practise applying it to interview questions.

Build a reverse-mode graph for addition and multiplication. Use a repeated input to expose overwritten gradients, then check the derivative numerically.

The full answer is part of Premium. A free account includes more answers in each topic, but does not unlock this one.

20 answers per topic instead of 10, plus saved progress and bookmarks · no cardor unlock all 303 remaining answers · ₹2,000 / $25

The concepts behind this question

Use these concept explanations to review the ideas behind this question.

🕸️ Distributed Training🔒 Premium

Forward Pass, Backpropagation and Optimizer StepsFollow one parameter through prediction, loss, gradient and update. Understand why backward does not itself train the weights, why training needs extra memory, and how microbatches differ from optimizer steps.

💻 Coding for Infra🔒 Premium

Batching Queues and BackpressureA request batcher groups work to reduce processing overhead. Queue bounds, dispatch timing and failure handling keep that efficiency from becoming uncontrolled waiting.

💻 Coding for Infra🔒 Premium

Interval Merging and Utilization LogsInterval merging finds periods of activity and gaps between them. A sweep over start and end events also measures concurrency and utilization.

💻 Coding for Infra

The GPU Credit Scheduler PatternA credit scheduler admits jobs without overspending an account’s balance. Clear state transitions make priority, cancellation and refunds easier to implement correctly.

Explore the full curriculum →

UP NEXT ON YOUR JOURNEY

Next in this trackImplement a bounded request log with timers that remain correct under concurrency Next in this trackImplement a thread-safe in-memory KV store with snapshot transactions and TTL Next in this trackImplement sliding-window attention without confusing token positions with ring-buffer slots

DISCUSSION · 0

No comments yet — be the first to share your approach.