split learning

Split Learning is a distributed learning technique that splits a neural network at a "cut layer" between the client and server — the client processes data through the lower layers (keeping data private), sends intermediate activations (smashed data) to the server, which completes the forward pass and backpropagation.

How Split Learning Works

• Client Side: Forward pass through layers 1-$k$ on private data → produce activations $a_k$.
• Server Side: Receive $a_k$, forward through layers $k+1$-$L$, compute loss, backpropagate to layer $k+1$.
• Gradient Return: Server sends $\nabla a_k$ (gradient of loss w.r.t. activations) back to client.
• Client Backward: Client backpropagates through layers 1-$k$ using $\nabla a_k$.

Why It Matters

• Low Client Compute: Client only runs part of the model — suitable for resource-constrained edge devices.
• Privacy: Raw data never leaves the client — only intermediate activations are shared.
• Caveat: Intermediate activations can leak information — additional protections (noise, quantization) may be needed.

Split Learning is dividing the neural network — clients process their data through the bottom half, servers complete the computation through the top half.