06Write a fused row softmax in Triton, explain why it is one HBM pass, and say where it stops scaling.▼mediumNewOpenAIFireworksTogether AI4 repliesunlockedOne program per row, the row in registers, max then exp then sum then divide, one read and one write of HBM. The runnable kernel with its launch, the numerics (subtract the max, accumulate in fp32), the block-size rule and the wide-row limit, and the arithmetic that says the kernel is done at 90% of copy speed.Open full answer →
11When do you write a kernel in Triton, and when do you have to drop down to CUDA?▼mediumNewOpenAITogether AI4 replies○ sign inTriton hands you a block of data and writes the thread-level code for you, which covers most of what a serving or training stack actually needs. The four things it does not give you, the performance you give up in each case with numbers, the reversal as Triton gains Hopper features, and how to decide in the room.Open full answer →
15What does torch.compile actually do to your model, and when does it fail to help?▼mediumNewOpenAIAnthropicMeta4 replies○ sign inFour stages: a bytecode interpreter captures a graph with guards, an autograd pass splits forward from backward, a compiler emits Triton for everything but the matmuls, and a mode that replays the step as one launch. What each buys, and the two failures that silently give it back.Open full answer →
