score matching

Diffusion Model is a generative model that learns to reverse a gradual noising process — trained by predicting and removing noise step-by-step, producing state-of-the-art image, audio, and video generation.

Forward Process (Noising)

• Gradually add Gaussian noise to data over T steps (typically T=1000).
• At step T, data is pure noise: $x_T \sim N(0, I)$.
• Mathematically: $q(x_t | x_{t-1}) = N(x_t; \sqrt{1-\beta_t} x_{t-1}, \beta_t I)$

Reverse Process (Denoising)

• A neural network (usually U-Net) learns to predict the noise added at each step.
• Generation: Start from pure noise $x_T$, iteratively denoise to get $x_0$.
• The network is conditioned on timestep $t$ and optionally on a text prompt.

Key Architectures

• DDPM (Denoising Diffusion Probabilistic Models): Original formulation (Ho et al., 2020).
• DDIM: Deterministic sampling — 10-50 steps instead of 1000 (10-100x faster).
• Latent Diffusion (Stable Diffusion): Runs diffusion in compressed latent space — 8x smaller, much faster.
• Score-Based Models: Equivalent formulation using score functions $\nabla_x \log p(x)$.

Why Diffusion Models Won

• Quality: Sharper, more diverse samples than GANs.
• Stability: No adversarial training — GANs suffer from mode collapse and training instability.
• Controllability: Easy to condition on text (CLIP guidance, classifier-free guidance).
• Likelihood: Tractable likelihood computation unlike GANs.

Applications

• Image generation: DALL-E 2, Stable Diffusion, Midjourney (FLUX), Imagen.
• Video: Sora, Runway Gen-2.
• Audio: WaveGrad, DiffWave.
• Protein structure: RFDiffusion.

Diffusion models are the dominant paradigm for generative AI — they have replaced GANs across virtually every generation task and continue to advance rapidly.