13Simulate a ring all-reduce on arrays. Verify both the result and the bytes each rank sends.▼mediumNewNVIDIAGoogle4 replies○ sign inTwo phases of N-1 steps each, a chunk index that rotates with the step, and a byte count that should come out to exactly the textbook formula. The implementation, the verification against a direct sum, and the measured traffic matching 2(N-1)/N times the message at every rank count tested.Open full answer →
33Implement a LoRA forward pass and merge two adapters without inventing cross terms▼hardNew2 replies◆ premiumFollow the matrix shapes through a low-rank update. Verify merged inference numerically and show why multiplying sums of adapter factors gives the wrong weights.Open full answer →
37Implement sliding-window attention without confusing token positions with ring-buffer slots▼hardNew2 replies◆ premiumBuild a small CPU reference, overwrite a bounded KV ring, and compare every step with a recomputed attention window. Keep model semantics separate from eviction.Open full answer →
38Implement symmetric INT8 quantization and prove where the reconstruction-error bound applies▼hardNew2 replies◆ premiumDerive the scale, handle zero tensors, clamp before casting, and separate rounding error from clipping. Compare one outlier under tensor and channel scales.Open full answer →