TL;DR: Coalescing groups a warp instruction's active byte addresses into memory sectors. For 32 aligned four-byte loads on the modern 32-byte sector model, contiguous, stride-two and stride-32 patterns touch 4, 8 and 32 sectors. These are request footprints, not guaranteed HBM traffic or 1×, 2× and 8× runtimes. Check the active mask, access width, cache behavior and the bottleneck before claiming a speedup.
How to approach it
Start with byte addresses for one executed instruction. Count sectors, then explain how a matrix's thread mapping creates those addresses. Use the profiler to connect the request pattern to cache misses and exposed time. Close with a fix that preserves the algorithm's result.
A strong answer
A matrix kernel uses few arithmetic operations but runs slowly. The first temptation is to increase its block size. A better first check is the addresses touched by neighboring lanes, the threads within a warp, at one load: a larger block cannot repair a mapping that scatters each warp across many sectors.
Count the byte footprint
A sector is a 32-byte chunk of memory, the unit the memory system fetches for a load. When the 32 lanes of a warp load neighboring addresses, a few sectors serve them all; when their addresses are scattered, each lane can need its own sector, so more bytes move for the same useful data.
NVIDIA's coalescing guide describes the 32-byte transaction model for compute capability 6.0 and later. For the examples below, all 32 lanes are active, each reads one four-byte value, the base is sector-aligned, and the values are distinct.
lane t, t in 0..31:
contiguous address = base + 4t → 4 sectors, 128 / 128 = 100% unique-byte fraction
stride 2 address = base + 8t → 8 sectors, 128 / 256 = 50%
stride 32 address = base + 128t → 32 sectors, 128 / 1024 = 12.5%
base shifted by 4 bytes, contiguous → 5 sectors, 128 / 160 = 80%
sanity: the same 128 distinct bytes can occupy very different request footprints.
The set of addressed sectors matters more than whether lane addresses are in ascending order. A permutation of the same 32 contiguous values still touches four sectors. A broadcast has many lane requests for one word; it should not be called wasteful merely because distinct-byte utilization is low. Memory Coalescing covers the address model, and GPU Memory Hierarchy explains what caches do with the resulting requests.
Change the mapping without changing the output
For row-major A[row * N + col], neighboring columns are adjacent in memory. At a fixed column, neighboring rows are N elements apart. Map a warp's lanes to columns when the algorithm permits it. A 32 × 8 block is a common starting layout: each warp occupies one thread-row because x varies fastest. Boundary masks and row alignment still affect the last rows and columns.
Here is an executable address check, not a GPU benchmark:
def sectors_for_row_mapping(n: int, row: int, col0: int = 0) -> set[int]:
if n < 0 or row < 0 or col0 < 0:
raise ValueError("dimensions and indices must be nonnegative")
if row >= n:
return set()
return {(4 * (row * n + col)) // 32
for col in range(col0, min(col0 + 32, n))}
assert len(sectors_for_row_mapping(4096, 0)) == 4
assert len({(4 * (row * 4096)) // 32 for row in range(32)}) == 32
assert len(sectors_for_row_mapping(33, 1)) == 5
assert sectors_for_row_mapping(0, 0) == set()
This checks naturally aligned FP32 element starts; none of those four-byte accesses straddles a sector. A general trace analyzer must include every byte of each access, especially for abstract unaligned or wider accesses. The trace-classifier question handles that separately.
Reordering a reduction, transpose or gather needs more care than swapping two variable names. Keep the output ownership and data dependencies correct. Shared-memory staging may convert an inconvenient global layout into a useful local one, but introduces its own bank and synchronization costs. Shared Memory and Bank Conflicts explains that second problem.
Read the correct profiler level
Nsight Compute's L1/TEX table reports sectors per request. For 32 active lanes, its ideal distinct aligned access ratios are 4 for 32-bit, 8 for 64-bit and 16 for 128-bit accesses. A lower value may mean masked lanes or overlapping requests, not a magical improvement. A source-level vector load can compile into different instructions, so inspect the executed instruction width too.
Follow L1 misses to L2 and L2 misses to device memory. A warp touching 32 sectors can reuse cached sectors from neighboring instructions; it does not necessarily fetch 1,024 fresh bytes from HBM every time. High logical useful bandwidth and high actual memory traffic are different metrics. Use source attribution to locate the expensive instruction, then timing and stall evidence to establish whether changing it improves the whole kernel.
For stores, count the byte footprint but also inspect the write path and duplicate destinations. Overlapping non-atomic stores are a correctness issue, not a beneficial broadcast. Avoid a blanket claim that every partial-sector write has the same read-modify-write behavior on every architecture and cache/ECC configuration.
Prefer a coalesced mapping when it preserves the operation and removes a measured bottleneck. If access is irregular, sorting, bucketing, row-level grouping or caching may help, but include preprocessing and restoration of output order. Reverse a layout change when its extra work outweighs the saved traffic. A scattered address pattern is evidence to investigate, not a complete performance diagnosis.
What interviewers probe next
- "Is four always ideal?" No. Width, active lanes, alignment and duplicate addresses determine the reference footprint.
- "Can permuted lanes still coalesce?" Yes. The addressed byte set can remain the same.
- "Does 12.5% utilization mean one-eighth runtime?" No. That fraction describes this request's distinct bytes, not the full execution.
Common mistakes
- Mixing L1 sectors, L2 requests and device-memory traffic into one number.
- Assuming source-code order alone establishes actual instruction width or cache misses.
- Fixing a layout without checking output correctness or the cost of the conversion.
Key takeaways
- Under the stated FP32 model, contiguous/stride-two/stride-32 touch 4/8/32 sectors.
- Moving the contiguous base four bytes can increase the footprint from four to five sectors.
- Count addresses first, then use cache and timing evidence to decide whether to change the mapping.