Understand the words.
Follow the work.

A CUDA core is an arithmetic execution lane in an NVIDIA streaming multiprocessor. The label commonly counts floating-point arithmetic capacity; it does not describe an independent CPU-like processor.

A GPU architecture describes the execution units, memory paths and instructions a family of processors provides. CUDA is the programming platform through which software uses supported NVIDIA devices.

GPU device memory stores allocations such as model weights, activations and output buffers. HBM and GDDR are memory technologies used by different GPU products.

Graphics processing clusters and texture processing clusters are physical groupings of resources inside NVIDIA GPUs. They sit above individual streaming multiprocessors in architecture diagrams.

The L1 data cache is an on-SM cache that can satisfy supported memory requests without fetching the data again from a more distant cache or device memory.

L2 is a cache shared across the GPU that sits between SM memory requests and device memory. Its capacity and bandwidth affect how much traffic reaches DRAM.

A load/store unit handles memory instructions issued by GPU threads, connecting register operands with the relevant memory path.

The register file is the SM storage backing thread register values. Its finite capacity limits how much thread state can remain resident at once.

A special function unit executes particular mathematical operations, such as supported approximations used in reciprocal, exponential or trigonometric calculations.

A streaming multiprocessor is the NVIDIA GPU unit that keeps thread blocks resident and executes their warps. It contains schedulers, registers, arithmetic pipelines and on-chip memory resources.

A Tensor Core is an NVIDIA execution unit for supported matrix multiply-accumulate operations. It computes small matrix products that software combines into larger operations.

Tensor Memory is specialized on-SM storage used by supported Blackwell tensor instructions, including their matrix accumulators. It is distinct from ordinary thread registers, shared memory and device DRAM.

The Tensor Memory Accelerator is an asynchronous data-movement facility introduced with NVIDIA Hopper. It moves tensor regions between global and shared memory without routing every element through a thread register.

A warp scheduler chooses an instruction from a ready resident warp and issues it to an available execution pipeline.

Compute capability identifies a set of NVIDIA GPU hardware features and limits. Compilers use targets such as sm_90 to generate suitable native device code.

The CUDA SIMT programming model expresses a kernel as many threads organized into blocks and a grid. Each thread has its own state and can take its own control-flow path.

CUDA Tile is a programming model that expresses computation over tiles of values. Its compiler maps tile operations onto supported NVIDIA execution and memory resources.

A grid is the collection of thread blocks created by one CUDA kernel launch. Its dimensions describe how much block-level work the launch supplies.

A GPU kernel is a function launched to execute work on the device. In CUDA SIMT, many threads execute that function with different indices.

PTX is NVIDIA’s virtual instruction-set representation for device code. It describes operations and execution semantics before final lowering to a particular GPU’s machine instructions.

SASS is the commonly used name for NVIDIA’s native GPU assembly. A cubin stores executable device code and metadata for a particular target.

A CUDA thread is one logical execution of a kernel, with its own index, register values and control-flow state.

A thread block is a group of CUDA threads that can share block-scoped memory and synchronize together. PTX commonly calls it a cooperative thread array, or CTA.

A thread-block cluster is an optional CUDA cooperation scope introduced with Hopper. Supported launches group blocks that can coordinate and access one another’s shared memory within the cluster.

A warp is a group of 32 NVIDIA CUDA threads through which the hardware issues instructions. An instruction may act on fewer than 32 lanes when some threads are inactive.

A warpgroup is a cooperating group of four consecutive warps, used by particular NVIDIA instructions such as Hopper warpgroup matrix multiply-accumulate.

A wavefront is an AMD GPU group of work-items that execute instructions together. Its width depends on the architecture and execution mode.

Global memory is the CUDA address space through which threads access data with device-wide visibility and allocation lifetime beyond one block.

The GPU memory hierarchy combines thread registers, shared memory, caches and device memory. These levels differ in scope, capacity, latency and how software controls them.

CUDA local memory is a thread-private address space whose storage is backed by device memory and caches. Despite its name, it is not the fast on-chip shared-memory region.

Registers hold values used by an individual GPU thread, such as loop indices, addresses and partial sums. They are backed by a finite register file on the SM.

Shared memory is programmer-managed on-chip storage that cooperating threads in a block can use to exchange and reuse values.

cuBLAS is NVIDIA’s library of GPU linear-algebra routines. cuBLASLt offers additional controls for supported matrix multiplications, including layouts, algorithm selection and fused epilogues.

CUDA Binary Utilities inspect compiled device-code artifacts. cuobjdump can extract or display device information embedded in executables; nvdisasm disassembles supported cubins.

CUDA C++ extends C++ with facilities for defining device functions, launching kernels and managing GPU execution. It exposes the SIMT thread, block and grid model.

The CUDA Driver API is a lower-level interface for device contexts, modules, memory and kernel launches. On Linux, its implementation is exposed through libcuda.so.

A CUDA event marks progress in a stream and can be used for supported synchronization or device timing.

A CUDA graph describes a collection of operations and their dependencies so a prepared executable graph can be launched repeatedly with less host submission work.

CUDA is NVIDIA’s GPU computing platform, including programming interfaces, compilers, libraries and development tools. The toolkit is a distribution of development components; the GPU driver is a separate part of the installed stack.

The CUDA Runtime API supplies common application operations such as allocation, copies, stream management and kernel launch support. Its shared-library form on Linux is libcudart.so.

A CUDA stream is an ordered queue of device work. Operations in one stream observe its ordering rules, while separate streams can express work that may overlap.

cuDNN is NVIDIA’s library for supported deep-learning operations, including convolution, attention and graph-based operation combinations.

CUPTI is NVIDIA’s interface for building tools that observe CUDA activity, API calls and supported performance information.

CuTe is a layout and tensor abstraction used in CUTLASS to describe how multidimensional coordinates map to data and execution resources.

CuTe DSL provides a Python-based way to express supported CuTe and CUTLASS kernel constructions that compile to GPU code.

CUTLASS is NVIDIA’s collection of building blocks and examples for constructing high-performance matrix and related GPU kernels.

Nsight Compute is a kernel profiler that collects supported GPU counters and relates them to source, native instructions and resource limits.

Nsight Systems is a timeline profiler for investigating how CPU activity, GPU work and transfers interact across an application.

nvcc is the CUDA compiler driver that coordinates compilation of CUDA C++ host and device code. It invokes multiple compilation stages rather than being a single device-code transformation.

The NVIDIA GPU driver provides the operating-system and user-space components that let applications manage and execute work on supported GPUs.

NVML is NVIDIA’s management and monitoring API for supported devices. It exposes information such as device identity, memory use, utilization and power under the platform’s permissions and capabilities.

nvidia-smi is NVIDIA’s command-line management and monitoring utility, built on NVML. It reports device state and exposes supported administrative operations.

NVRTC is a library for compiling CUDA C++ device source at runtime. Applications can use it to specialize kernels for shapes or types discovered while running.

Triton is a language and compiler for writing GPU kernels using blocks of values and compiler-managed mapping to execution resources.

An active cycle is a cycle classified as active for a named unit and metric. Elapsed cycles count the broader measurement interval, including its inactive time.

Arithmetic intensity is the number of arithmetic operations performed per byte transferred across a specified memory boundary.

Arithmetic throughput is the rate of arithmetic operations, commonly expressed as floating-point operations per second. Its interpretation depends on the operation and numerical format.

Branch efficiency describes how uniformly threads take branch paths under a particular tool’s branch-counting convention. It is a control-flow statistic, not overall GPU efficiency.

A compute-bound workload is limited by the rate at which the relevant arithmetic work can execute, under the current input and implementation.

Issue efficiency describes how often a scheduler issues work relative to the issue opportunities being counted. It must be interpreted with the metric’s cycle and scheduler scope.

Latency hiding keeps an execution resource productive by doing independent work while another operation waits for its result.

Little’s law relates average items in a stable system to average arrival rate and average time spent there: L = λW.

Memory bandwidth is the rate at which bytes cross a particular memory interface. A specification gives a peak; a benchmark or profiler gives a rate under stated conditions.

Memory coalescing combines the byte addresses requested by active lanes of a warp instruction into the memory sectors needed to serve them.

A memory-bound workload makes progress at a rate limited by its memory accesses. The cause can be transfer bandwidth, dependency latency or another constrained part of the memory path.

Occupancy is resident warps divided by the device’s supported resident-warp limit at the stated scope. It measures available thread state, not useful arithmetic throughput.

Overhead is time or work spent supporting a computation rather than performing the particular useful operation being counted. Its meaning depends on the measurement boundary.

A peak rate is a reference maximum for a specified resource, operation and operating assumption. It is the denominator of many relative-throughput metrics.

A performance bottleneck is a resource or dependency that limits progress for the workload and interval being measured.

Pipeline utilization describes activity or throughput on a particular execution pipeline, under the profiler’s specified normalization.

Register pressure is the demand created by values that must remain live in registers at the same time. It influences resource allocation and possible spills.

The roofline model bounds arithmetic throughput using the smaller of a compute ceiling and a bandwidth ceiling multiplied by arithmetic intensity.

A scoreboard stall occurs when a warp cannot issue its next instruction because a tracked dependency has not completed.

A bank conflict occurs when a shared-memory instruction requests different addresses that the hardware must serve through the same bank, requiring additional service steps.

SM utilization is an ambiguous label unless the tool and counter are named. Device activity, resident warps and execution-pipeline throughput measure different properties.

Warp divergence occurs when threads in the same warp need different control-flow paths. Instructions for those paths may execute with different subsets of lanes active.

A resident warp holds execution state on an SM. An eligible warp has an instruction ready to issue. An issued warp is one selected by the scheduler at that issue opportunity.