cuda programming basics

CUDA Programming — NVIDIA's parallel computing platform that enables general-purpose computation on GPUs, leveraging thousands of cores for massive parallelism.

Execution Hierarchy

• Thread: Smallest unit of execution
• Warp: 32 threads executing in lockstep (SIMT)
• Block: Group of threads (up to 1024) sharing fast shared memory
• Grid: Collection of all blocks launched for a kernel

Basic Pattern

__global__ void add(float *a, float *b, float *c, int n) {
    int i = blockIdx.x * blockDim.x + threadIdx.x;
    if (i < n) c[i] = a[i] + b[i];
}
// Launch: add<<<numBlocks, threadsPerBlock>>>(a, b, c, n);

Memory Hierarchy

• Registers: Fastest, per-thread (~256KB per SM)
• Shared Memory: Per-block, programmer-managed cache (~100KB per SM). 100x faster than global
• Global Memory: Large (up to 80GB HBM) but slow (~500 cycles latency). Coalesced access is key

Performance Keys

• Maximize occupancy (fill SMs with enough warps to hide latency)
• Coalesce global memory accesses (consecutive threads access consecutive addresses)
• Use shared memory to reduce global memory traffic
• Minimize warp divergence (avoid branches within a warp)

CUDA powers the majority of GPU computing workloads from deep learning training to scientific simulation.