1,045 technical terms and definitions
kernel performance profiling, gpu occupancy analysis, memory throughput profiling, instruction throughput analysis
GPU Profiling with Nsight Compute is the systematic analysis of GPU kernel performance characteristics — including compute throughput, memory throughput, occupancy, stall reasons…
nsight profiler, occupancy analysis, kernel optimization gpu, performance bottleneck gpu
GPU Performance Profiling and Optimization is the systematic analysis methodology that identifies and eliminates performance bottlenecks in GPU kernels — using hardware…
cuda thread block, warp execution, thread hierarchy gpu, cooperative groups
GPU Programming Model and Thread Hierarchy is the software abstraction that organizes millions of GPU threads into a hierarchical structure — grids of thread blocks (each…
cuda ray tracing optimization, optix ray tracing, gpu graphics compute, rtx ray tracing performance
GPU Ray Tracing and Graphics Compute is the parallel implementation of ray tracing algorithms and graphics computations on GPUs — where dedicated RT Cores on NVIDIA RTX GPUs…
rt core bvh traversal, hardware ray triangle intersection, real time ray tracing, bvh acceleration structure
GPU Hardware Ray Tracing is the dedicated fixed-function hardware (NVIDIA RT Cores, AMD Ray Accelerators, Intel Ray Tracing Units) that accelerates the computationally intensive…
parallel reduction cuda, warp reduction optimization, cuda reduce performance, hierarchical reduction gpu
GPU Reduction Patterns are the parallel algorithms for combining array elements into single value through associative operations — where hierarchical reduction using warp…
register spilling local memory, occupancy register tradeoff, ptx register allocation, kernel register count
GPU Register Optimization addresses the critical trade-off between register availability for instruction-level parallelism and kernel occupancy, directly impacting throughput…
register allocation gpu, register spill, occupancy register, gpu register file
GPU Register Pressure is the conflict between a kernel's per-thread register demand and the GPU's fixed register file capacity — where each additional register per thread reduces…
parallel scan cuda, cuda prefix sum optimization, inclusive exclusive scan, scan algorithm gpu
GPU Scan (Prefix Sum) is the parallel algorithm that computes cumulative sums or other associative operations across array elements — where inclusive scan produces [a0, a0+a1…
heterogeneous task scheduling, cpu gpu co-scheduling, device affinity
Heterogeneous Task Scheduling is the algorithmic and runtime discipline of assigning computational tasks to the most appropriate processing element — CPU cores, GPU compute units…
shared memory optimization, smem, scratchpad
GPU Shared Memory Optimization — using the fast, programmer-managed on-chip memory (shared memory / SMEM) within each GPU Streaming Multiprocessor (SM) to drastically reduce…
shared memory optimization, bank conflict resolution, shared memory access pattern
GPU Shared Memory Bank Conflicts are the performance penalties that occur when multiple threads in a warp simultaneously access different addresses that map to the same shared…
shared memory optimization cuda, bank conflict avoidance, shared memory padding, warp shared memory access
GPU Shared Memory Bank Conflicts represent the performance hazard that occurs when multiple threads within a warp simultaneously access different addresses mapped to the same…
shared memory bank conflict, shared memory tiling, scratchpad memory gpu, shared memory programming
GPU Shared Memory Optimization is the performance-critical programming technique that uses the GPU's fast, software-managed on-chip memory (shared memory / scratchpad) to cache…
shared memory bank conflict, tiling gpu kernel, shared memory usage cuda, local data share
GPU Shared Memory Optimization is the critical CUDA/GPU programming technique of using on-chip shared memory (32-228 KB per Streaming Multiprocessor) as a programmer-managed cache…
streaming multiprocessor, cuda core, gpu compute unit, sm design
GPU Streaming Multiprocessor (SM) Architecture is the fundamental compute building block of NVIDIA GPUs, where each SM contains a set of CUDA cores, warp schedulers, register…
streaming multiprocessor, warp occupancy, gpu occupancy tuning
GPU SM Occupancy Optimization is the tuning of GPU kernel resource usage (registers, shared memory, block size) to maximize the number of concurrent warps executing on each…
cuda radix sort, parallel sorting gpu, gpu sort performance, cuda sort optimization
GPU Sorting Algorithms are the parallel implementations of sorting that leverage thousands of GPU threads to achieve 100-300 GB/s throughput — where radix sort (optimal…
cuda sparse linear algebra, cusparse optimization, sparse matrix gpu performance, csr coo format gpu
GPU Sparse Matrix Operations are the specialized algorithms for matrices where most elements are zero, exploiting sparsity to reduce memory and computation — where Compressed…
gpu event synchronization, asynchronous gpu execution, cuda stream event ordering, gpu stream dependency management
GPU stream-event synchronization is the practice of expressing the minimum ordering relationships required among asynchronous GPU kernels, memory transfers, and host actions.…
matrix multiply accelerator, mixed precision hardware, tensor operation unit, wmma instruction
GPU Tensor Cores are the specialized matrix multiplication accelerator units embedded in modern GPU architectures (NVIDIA Volta and later, AMD Matrix Cores) that perform small…
wmma api matrix multiply, mixed precision tensor core, mma ptx instruction, tensor core accumulator fp32
GPU Tensor Core Programming is the practice of utilizing specialized matrix multiply-accumulate (MMA) hardware units in NVIDIA GPUs that perform small matrix operations (e.g.…
tensor core matrix multiply, wmma api cuda, tensor core mixed precision, tensor core performance optimization
GPU Tensor Core Programming is the technique of leveraging specialized matrix-multiply-and-accumulate hardware units in modern GPUs to achieve dramatic speedups for linear algebra…
constant memory, read only cache, cuda texture, cuda cache
GPU Texture and Constant Memory are specialized GPU memory spaces with dedicated caches — optimized for specific access patterns that offer higher effective bandwidth than global…
gpu boost clock, thermal design power, gpu temperature, tdp throttle, gpu power limit
GPU Thermal Throttling and Power Management is the hardware and firmware mechanism that dynamically reduces GPU clock frequency and voltage when the chip temperature or power…
optimization
GPU utilization measures the percentage of a GPU's computational resources that are actively being used at any given moment. In the context of AI and LLM workloads, achieving high…
mig multi instance, gpu sharing, vgpu, gpu partitioning
GPU Virtualization and Multi-Instance GPU (MIG) is the technology enabling a single physical GPU to be partitioned into multiple isolated instances — each with dedicated compute…
difference between gpu and cpu, gpu vs cpu chip, what is the difference between a cpu and gpu, cpu vs gpu explained, why gpu for ai
A CPU and a GPU are both chips built by the same lithography and fabrication process, but they are laid out around opposite bets on what kind of work matters most: a CPU devotes…
sm streaming multiprocessor, cuda core execution, register file gpu, warp scheduler hardware design
GPU Streaming Multiprocessor (SM) Architecture is the fundamental execution unit of GPU chips, containing dozens of CUDA cores, tensor cores, warp schedulers, and hierarchical…
thread divergence, simt divergence, branch divergence gpu, warp efficiency
GPU Warp Divergence is the performance penalty that occurs when threads within the same warp (typically 32 threads executing in lockstep) take different paths at a branch…
branch divergence simt, thread divergence penalty, predication gpu, warp execution efficiency
GPU Warp Divergence is the performance penalty that occurs when threads within the same warp (NVIDIA, 32 threads) or wavefront (AMD, 64 threads) take different execution paths…
thread divergence cuda, branch divergence penalty, predicated execution gpu, control flow efficiency
GPU Warp Divergence is the performance degradation that occurs when threads within a warp take different execution paths at a branch — forcing the hardware to serialize both paths…
simt execution, warp divergence
GPU Warp Scheduling — the mechanism by which a GPU's streaming multiprocessor (SM) manages and interleaves execution of warps (groups of 32 threads) to hide memory latency. SIMT…
warp divergence, cuda thread branching, simt single instruction multiple thread, warp execution
GPU Warp Scheduling and Divergence represents the critical, uncompromising hardware execution mechanic within NVIDIA GPUs where 32 loosely independent software threads are…
warp scheduler hardware, instruction level parallelism gpu, dual issue gpu, warp stall reason
GPU Warp Scheduling is the hardware mechanism that selects which ready warp to execute each clock cycle on a Streaming Multiprocessor (SM) — where the warp scheduler's ability…
warp execution model cuda, thread divergence penalty, warp scheduler hardware, simt divergence handling
GPU Warp Scheduling and Divergence is the hardware mechanism by which a GPU streaming multiprocessor (SM) selects warps of 32 threads for execution each cycle and handles…
simt warp divergence, warp occupancy optimization, gpu thread scheduling, streaming multiprocessor warps
GPU Warp Scheduling and Execution Model — GPU architectures organize threads into warps (typically 32 threads) that execute instructions in lockstep using the Single Instruction…
gpudirect rdma gdr, gpudirect storage gds, gpu direct peer access, gpudirect async
GPUDirect Technology is NVIDIA's suite of technologies that enable direct data paths between GPUs and other system components (other GPUs, network adapters, storage) — bypassing…
gqa, general question answering, evaluation
GQA (General Question Answering) is a dataset for compositional visual reasoning — focusing on real-world images but using procedurally generated questions to rigorously test…
grace hopper, nvidia grace hopper, what is grace hopper
A coherent CPU-GPU superchip. NVIDIA Grace Hopper, commonly identified as GH200, combines an Arm-based Grace CPU with a Hopper GPU using NVLink-C2C. It is intended for accelerated…
reliability
Graceful Degradation is the system design principle ensuring that applications maintain core functionality when components fail, resources become constrained, or dependencies…
optimization
Graceful Degradation is a resilience strategy that serves reduced functionality when full capability is unavailable - It is a core method in modern semiconductor AI serving…
graceful degradation fallback, graceful degradation, degradation fallback
Graceful Degradation Fallback Introduction & Motivation Graceful Degradation Fallback addresses a central problem in cloud process control and degradation-mode-service-quality…
graph neural networks
Graclus Pooling is a fast graph-clustering based pooling method for multilevel graph coarsening. - It greedily matches nodes to form compact clusters used in graph CNN…
Grad-CAM Class Activation Mapping Introduction & Motivation Grad-CAM Class Activation Mapping addresses a central problem in interpretability and explanation of xai predictions…
grad-cam class activation mapping prediction, grad-cam class activation, class activation mapping, activation mapping prediction, grad-cam class, class activation
Grad-CAM Class Activation Mapping Prediction with Machine Learning Introduction & Motivation Grad-CAM Class Activation Mapping Prediction with Machine Learning addresses a central…
Grad-CAM Class Activation Mapping (Extended Study) Introduction & Motivation Grad-CAM Class Activation Mapping (Extended Study) addresses a central problem in interpretability…
Grad-CAM Class Activation Mapping (Extended Study) Introduction & Motivation Grad-CAM Class Activation Mapping (Extended Study) addresses a central problem in interpretability…
Grad-CAM Class Activation Mapping (Extended Study) Introduction & Motivation Grad-CAM Class Activation Mapping (Extended Study) addresses a central problem in interpretability…
Grad-CAM++ for Multi-Instance Visual Feature Attribution Introduction & Motivation Grad-CAM++ for Multi-Instance Visual Feature Attribution addresses a central problem…