Pruning — the quick intro (5m)

TL;DR

Keep the most important weights/filters, remove the rest. Importance can be measured by simple norms or smarter scores like leverage/volume.

Key ideas - Structured vs unstructured pruning - Simple importance: \(\ell_1, \ell_2\) norms - Better signals: leverage scores, volume scores

Why it works: many parameters are redundant; pruning reduces compute/memory with small accuracy loss.