GPU glossary

A definition for the term, and a reading that puts it to use.

Barrier

A point where a specified group of threads coordinates its progress. A CUDA block barrier separates writes from peer reads across that barrier; it does not coordinate other blocks. The two-warp example →

Block

A group of CUDA threads that can cooperate using the block’s shared memory and barriers. The block’s shape determines how local coordinates are assigned. A 4 × 4 block →

Coalescing

Combining global memory accesses from the active threads of a warp into the memory transactions needed for their addresses. The arrangement and alignment of addresses matter. Memory access patterns →

Grid

All blocks launched for one kernel invocation. Its dimensions count blocks; multiplying by block dimensions gives launched thread positions, which may extend past the data. The image boundary →

Lane

A thread’s position within its warp. A lane number is local to that warp; lane 7 in two different warps names two different threads. Warp exchange →

Pitch

The physical spacing between the starts of adjacent rows. CUDA’s pitched allocation API reports bytes; the logical width counts elements. 44 bytes versus 64 bytes →

PTX

NVIDIA’s virtual instruction set. It describes typed operations, memory spaces, and virtual registers before translation to native machine instructions for a GPU target. An increment in PTX →

Register

Storage for a live value during instruction execution. PTX register names are virtual; the compiler assigns machine registers and can reuse them after a value’s last use. Why R7 changes purpose →

SASS

The name commonly used for NVIDIA GPU native assembly. A disassembly shows the instructions in a compiled binary for a specific architecture. A native listing →

Shared memory

On-chip storage through which threads in a block can exchange data. Giving peers access to the same slot does not by itself establish when its contents are safe to read. Exchanging values across warps →

Stride

The distance in storage when a logical coordinate advances by one along an axis. Strides can be specified in elements or bytes; the unit is part of the layout. Coordinates and storage →

Tensor Core

A GPU execution unit for supported matrix operations. In a warp-level matrix multiply-accumulate instruction, the lanes cooperate on matrix fragments. One matrix tile →

Warp

A group of 32 CUDA threads formed from consecutive thread ranks within a block. A particular instruction may have fewer than 32 active lanes. How lanes exchange values →