Lambda Napkin Math, Cost & Capacity interview questions

Compare ownership with the actual rental contract, using explicit cash flows, resale assumptions and the hours that can really be avoided.

Calculate the fleet, duration and rental budget for a 7B training run. Check the deadline, actual sharding scheme and paid reservation period before placing the order.

Compare raw weight sizes at 16, 8 and 4 bits, then budget KV cache and runtime memory for Llama 3.1 70B. Follow the worked per-GPU example.

Work from training FLOPs to fleet throughput, days and GPU-hours. Then see how utilization and downtime change the estimate.

Solve for useful throughput, round to a supported parallel layout, and account for calendar time once. The right GPU count need not be a power of two.

Separate frozen weights from trainable state, count rank-16 adapters from matrix shapes, and check peak memory before choosing a fine-tuning layout.

Calculate Llama 3.1 70B KV cache memory at 128k context using eight KV heads. Worked BF16 and FP8 payloads, GB versus GiB, and per-GPU capacity checks.

The 6ND rule, where the 6 comes from, and the 6.3e24 that every training-time and GPU-count question in the loop is built on.

Compute cost from a matched bill and token count. Show when an activity factor belongs in the denominator and when it would double-count idle time.

Count gradient traffic with the actual dtype, convert one-direction rates, and expose the bucket-readiness constraint that an average bandwidth division misses.

Check model placement first, then derive a TP8 traffic floor and add context. Separate a single stream’s pace from an ideal bandwidth ratio.

Turn FLOPs and HBM bytes into resource bounds, then compare them with elapsed time. A tighter memory bound suggests an investigation rather than proving the measured bottleneck.

Separate resident parameters, active work and the union of experts touched by a batch. Compare toy dense and MoE budgets without promising a fixed speedup.

Use bandwidth per dollar as a first comparison, then check feasible memory, supported execution and complete service cost before ranking accelerators.

Work through projection and attention FLOPs for an 8,192-token prompt. Separate a peak-based lower bound, an assumed-rate estimate and observed TTFT.

Derive GPU-hours from an explicit FLOP convention, compare Meta’s 30.84 million reported hours, and avoid mistaking the residual for a measured MFU.

Translate active streams into replicas using growth reservations and matched latency tests. Show peak and failure headroom separately, with no invented benchmark.