evolutionary architecture search

Evolutionary Architecture Search is a NAS method that uses evolutionary algorithms — selection, crossover, and mutation — to evolve neural network architectures over generations — maintaining a population of candidate architectures and iteratively improving them through biologically-inspired operations.

How Does Evolutionary NAS Work?

• Population: Initialize a set of random architectures.
• Fitness: Train each architecture and evaluate accuracy (and optionally latency/size).
• Selection: Keep the fittest architectures. Remove the worst.
• Mutation: Randomly modify operations, connections, or hyperparameters.
• Crossover: Combine parts of two parent architectures to create children.
• Examples: AmoebaNet, NEAT, Large-Scale Evolution (Real et al., 2019).

Why It Matters

• No Gradient Required: Works for non-differentiable search spaces and objectives.
• Exploration: Better at exploring diverse regions of the search space than gradient-based methods.
• Quality: AmoebaNet achieved state-of-the-art ImageNet accuracy, matching RL-based NASNet.

Evolutionary NAS is natural selection for neural networks — breeding and evolving architectures over generations until the fittest designs emerge.