HEAPr: Hessian-based Efficient Atomic Expert Pruning in Output Space

HEAPr enables fine-grained pruning of mixture-of-experts models through atomic expert decomposition and efficient second-order importance estimation.

Paper · Code