grace-hopper

A coherent CPU-GPU superchip. NVIDIA Grace Hopper, commonly identified as GH200, combines an Arm-based Grace CPU with a Hopper GPU using NVLink-C2C. It is intended for accelerated AI, data analytics, and high-performance computing. The name describes a heterogeneous computing assembly and associated platform, rather than a single CPU architecture or a new transistor process. It should also be distinguished from later Grace Blackwell products.

Coherence changes how processors access data. NVLink-C2C provides a coherent connection between the CPU and GPU, allowing a shared memory model across the connected processors. This can simplify access to data that spans their memory systems and reduce the need for explicit copies in supported software. Coherence does not make every access equally fast: locality, bandwidth, latency, and the placement of frequently used data still influence application performance.

CPU memory and GPU memory serve different roles. Grace provides CPU-attached memory, while the Hopper GPU uses high-bandwidth memory. NVIDIA lists GH200 implementations with HBM3 and HBM3e. The exact capacity and bandwidth depend on the selected configuration. Combining addressability across memory domains does not convert all available capacity into GPU-local HBM, so working-set placement and memory traffic remain important when a model exceeds the fastest memory tier.

A system can combine multiple superchips. NVIDIA also describes configurations that connect more than one GH200 through NVLink. Such systems add another level of communication beyond the CPU-GPU connection inside each superchip. A benchmark must therefore identify the number of assemblies, their memory configuration, topology, and software. Results from a multi-superchip system cannot be assigned directly to one Hopper GPU or one coherent link.

Evaluate workloads that benefit from close coupling. Grace Hopper can be useful when CPU and GPU stages exchange substantial data or when memory capacity and movement limit an application. Compute-heavy kernels with little CPU interaction may respond differently. A useful evaluation measures end-to-end runtime, data movement, memory placement, power, and output quality under the intended software stack. The important architectural feature is integrated heterogeneous computing with explicit locality and communication costs.

Sources and further reading. [NVIDIA GH200 platform specifications](https://www.nvidia.com/en-us/data-center/grace-hopper-superchip/); [NVIDIA CUDA programming documentation](https://docs.nvidia.com/cuda/cuda-programming-guide/).

Read NVIDIA Grace Hopper as a closely coupled CPU-GPU platform whose benefit depends on data locality and end-to-end workload behavior.