memory buffer

A memory buffer in continual learning is a fixed-size storage that holds representative examples from previously learned tasks, enabling rehearsal and preventing catastrophic forgetting. The design of the memory buffer — its size, what it stores, and how it manages capacity — is crucial for continual learning performance.

What a Memory Buffer Stores

• Raw Examples: The original input-output pairs (x, y). Most straightforward approach.
• Features: Intermediate representations from the model — more compact than raw data.
• Logits: The model's output distribution (soft labels) at the time the example was stored. Used for knowledge distillation during replay.
• Gradients: Gradient vectors from previous tasks, used to constrain optimization direction (e.g., GEM).

Buffer Management Strategies

• Reservoir Sampling: Each new example has a probability of replacing an existing buffer entry, ensuring the buffer is a uniform random sample of all seen data. Simple and theoretically sound.
• Class-Balanced: Maintain equal representation of each class/task in the buffer. Prevents bias toward recent or dominant classes.
• Herding: Select examples that best approximate the class mean in feature space — keeps the most representative examples.
• FIFO (First-In-First-Out): Evict the oldest examples. Simple but may lose important early knowledge.
• Loss-Based: Keep examples with the highest loss (hardest examples) or most diverse coverage.

Buffer Size Trade-Offs

• Larger Buffer: Better knowledge retention, higher accuracy on old tasks, but more memory consumption and potential privacy concerns.
• Smaller Buffer: Lower memory cost, faster sampling, but more forgetting. Even 20 examples per class can significantly reduce forgetting.
• Typical Sizes: Research benchmarks use 200–5,000 examples total across all tasks.

Advanced Techniques

• Compressed Buffers: Store compressed representations to fit more examples in the same space.
• Generative Buffers: Replace stored examples with a generative model that can produce synthetic examples from old tasks on demand.
• Dynamic Sizing: Adjust buffer allocation as the number of tasks grows — each task gets a smaller slice.

The memory buffer is the heart of rehearsal-based continual learning — its design directly determines how well the system balances remembering old knowledge with learning new information.