NVRTC
NVRTC is a library for compiling CUDA C++ device source at runtime. Applications can use it to specialize kernels for shapes or types discovered while running.
How it works
Compilation produces an artifact such as PTX or supported native code. Loading or linking that artifact and launching it are separate steps. A cache can avoid repeating compilation, provided its identity includes the inputs that affect generated code.
Work through an example
A service’s first request may pay compilation and loading costs while later requests reuse the artifact. Report cold and warm timings separately and test a changed shape that misses the cache.
Where the term can mislead
Runtime compilation is not free and is not automatically performed on every launch. Measure the actual compilation path before attributing a pause to it.
Connect the idea
Take it into practice
Continue with the existing concept, apply it to an interview problem, or follow a course lesson. Access requirements are shown before you open a page.
New to GPUs? Start with the free execution-model concept.
Sources and scope
Examples here are teaching calculations, not measured GPU performance. Consult the linked documentation for the exact device, instruction and tool version. Source review: 25 October 2026.