AI Infra Interviews logo
Reading performance · free reference

Warp divergence

Warp divergence occurs when threads in the same warp need different control-flow paths. Instructions for those paths may execute with different subsets of lanes active.

Divergent branchActive mask

How it works

Execution cost depends on the paths, their active masks and the generated code. Predication can replace a branch for some short operations. Divergence between separate warps is different from divergence among lanes of one warp.

Work through an example

If 16 lanes take a long path and 16 take a short path, instructions on each path can leave part of the warp inactive. This alone does not imply an exact twofold runtime penalty.

Where the term can mislead

Independent thread scheduling does not make divergent paths free, and implicit warp lockstep is not a safe synchronization contract.

Connect the idea

Take it into practice

Continue with the existing concept, apply it to an interview problem, or follow a course lesson. Access requirements are shown before you open a page.

New to GPUs? Start with the free execution-model concept.

Sources and scope

Examples here are teaching calculations, not measured GPU performance. Consult the linked documentation for the exact device, instruction and tool version. Source review: 25 October 2026.

  1. NVIDIA CUDA Programming Guide: programming model ↗
  2. NVIDIA CUDA C++ Best Practices Guide ↗