Researchers have proposed a new method to optimize Convolutional Neural Networks (CNNs) by utilizing the Multi-Armed Bandits (MAB) framework. The study, published on arXiv, aims to eliminate redundant feature maps to reduce storage and inference costs without compromising model performance.
Context & Origin
Modern CNNs often contain highly redundant feature maps, leading to inefficient hardware utilization during inference. While previous pruning techniques attempted to address this, most focused on removing isolated scalar weights, resulting in sparse network structures that are difficult to optimize on standard hardware. In contrast, structured pruning targets the removal of entire output channels and their corresponding filters, offering more tangible hardware acceleration. However, determining which channels can be safely removed without degrading network accuracy remains a major challenge.
Technical Analysis & Technology
To address this issue, the research team developed a loss-aware feature-map pruning framework based on Multi-Armed Bandits. Within this setup, each candidate feature map is treated as an "arm" of a multi-armed bandit. At each play step, the system temporarily masks a feature map and evaluates the change in loss on a sampled mini-batch. The map is then restored, and the observed loss change is converted into a reward representing the safety of its removal.
After a fixed evaluation budget, candidate feature maps are ranked based on scores learned by the MAB algorithms. The study evaluated two popular algorithms, UCB1 and Thompson Sampling, to optimize this selection process. The top-ranked maps (indicating minimal impact on loss when removed), along with their corresponding filters, biases, and next-layer input-channel kernels, are permanently pruned from the neural network.
Expert Opinions & Insights
The research team conducted extensive empirical evaluations across multiple datasets ranging from simple to complex, including MNIST, CIFAR-10, CIFAR-100, SVHN, CUB-200-2011, and Oxford Flowers 102. The results demonstrated that both UCB1 and Thompson Sampling maintained accuracy levels remarkably close to the original unpruned models. Statistical Friedman and Nemenyi tests confirmed that UCB1 achieved the highest mean rank, significantly outperforming traditional greedy and magnitude-based pruning methods.
Impact & Future
The success of this research opens up significant potential for deploying complex deep learning models on resource-constrained hardware, such as mobile devices, IoT equipment, and embedded systems. By intelligently minimizing convolutional computations through channel pruning, this technique not only saves energy but also accelerates inference speeds for real-world AI applications. This represents a highly practical step forward in bridging the gap between theoretical AI research and practical market deployment.