simd vectorization
SIMD Vectorization is the technique of executing a single instruction that operates simultaneously on multiple data elements packed into wide vector registers — where modern CPUs provide 128-bit (SSE), 256-bit (AVX2), or 512-bit (AVX-512) registers that can process 4-16 single-precision floats per instruction, achieving 4-16x throughput improvement for data-parallel operations without any multi-threading overhead.
SIMD Register Widths
| ISA Extension | Register Width | FP32 Elements | FP64 Elements | Available On |
|---|---|---|---|---|
| SSE/SSE2 | 128 bit | 4 | 2 | All x86 since ~2001 |
| AVX/AVX2 | 256 bit | 8 | 4 | Intel Haswell+ (2013), AMD Zen+ |
| AVX-512 | 512 bit | 16 | 8 | Intel Skylake-SP+, AMD Zen 4+ |
| ARM NEON | 128 bit | 4 | 2 | All ARMv8 |
| ARM SVE/SVE2 | 128-2048 bit (scalable) | 4-64 | 2-32 | ARMv9, Graviton3+ |
Auto-Vectorization (Compiler)
// Compiler auto-vectorizes this loop:
for (int i = 0; i < N; i++)
C[i] = A[i] + B[i];
// Becomes (conceptually, AVX2):
// for (int i = 0; i < N; i += 8)
// _mm256_store_ps(&C[i], _mm256_add_ps(_mm256_load_ps(&A[i]), _mm256_load_ps(&B[i])));Compiler flags: `-O3 -march=native` (GCC), `/O2 /arch:AVX2` (MSVC).
What Prevents Auto-Vectorization
| Blocker | Example | Fix |
|---|---|---|
| Loop-carried dependency | `a[i] = a[i-1] + b[i]` | Restructure algorithm |
| Non-unit stride | `a[i*3]` | Use gather or restructure data layout |
| Function calls | `a[i] = sin(b[i])` | Use SVML/libmvec vector math |
| Pointer aliasing | `void f(float *a, float *b)` | Add `restrict` keyword |
| Conditionals | `if (a[i] > 0) ...` | Use masked operations |
| Unknown trip count | `while (*ptr)` | Hard to vectorize |
Intrinsics (Manual SIMD)
#include <immintrin.h>
// AVX2: 8-wide float multiply-add
__m256 a = _mm256_load_ps(&A[i]);
__m256 b = _mm256_load_ps(&B[i]);
__m256 c = _mm256_load_ps(&C[i]);
__m256 result = _mm256_fmadd_ps(a, b, c); // result = a*b + c
_mm256_store_ps(&D[i], result);- Intrinsics give full control but are platform-specific and harder to maintain.
- Best for performance-critical inner loops where compiler fails to auto-vectorize.
Data Layout for SIMD
- AoS (Array of Structs): `struct {float x,y,z;} points[N]` — bad for SIMD (non-contiguous).
- SoA (Struct of Arrays): `float x[N], y[N], z[N]` — good for SIMD (contiguous per field).
- Converting AoS → SoA can yield 2-4x speedup from better vectorization alone.
SIMD vectorization is the most accessible form of parallelism available on every modern CPU — achieving significant speedups without the complexity of multi-threading, making it the first optimization technique to reach for in any compute-bound application, from scientific computing to image processing to database query execution.