cuda graphs

Follow graph capture, guards, differentiation and generated kernels. Then distinguish graph breaks from recompilation and measure whether warm execution repays compilation and startup costs.

Use a timeline to establish launch overhead, then trace what graph replay changes. Stable buffers, page-table updates, capture-safe operations and batch buckets determine whether the serving engine can use it safely.

Check the bound’s assumptions, locate the critical path, and quantify which optimization can actually improve one user’s decode latency.

Measure exposed launch and dispatch delay, actual graph coverage, capture memory and warmup. Batch size influences the result, but does not alone determine whether graphs improve the service.