puzzlemix

PuzzleMix is a data augmentation technique that optimizes the mixing mask to maximize the saliency (importance) of the mixed regions — cutting and mixing the most informative regions from each training image, guided by the model's gradient-based saliency maps.

How Does PuzzleMix Work?

• Saliency: Compute gradient-based saliency maps for both images.
• Optimal Transport: Find the mixing mask that maximizes the total saliency of visible regions.
• Mix: Apply the optimized mask to create a training sample with the most useful features from both images.
• Labels: Mixed proportionally to the visible saliency-weighted area.
• Paper: Kim et al. (2020).

Why It Matters

• Intelligent Mixing: Unlike random CutMix, PuzzleMix ensures informative regions are visible, not occluded.
• Accuracy: Consistently outperforms CutMix and Mixup by 0.5-1.0% on ImageNet.
• Saliency-Guided: Uses the model's own understanding to create maximally informative training samples.

PuzzleMix is CutMix with intelligence — using saliency maps to mix the most important parts of each image together.