Review on the Principles and Cutting-Edge Methods of Deep Learning Dynamic Pruning and Model Compression Techniques
DOI:
https://doi.org/10.62051/n26pzx93Keywords:
Pruning; Neural Network; Deep Learning; Computer Visio.Abstract
Deep learning models have achieved remarkable success in fields such as computer vision, natural language processing, and speech recognition. However, their massive parameter counts and high computational complexity severely restrict their deployment in resource-constrained scenarios like edge devices and embedded systems. As a core model compression technology, neural network pruning achieves model lightweight while maintaining performance by removing redundant connections, neurons, or filters. Based on research related to neural network pruning and more cutting-edge achievements, this paper systematically sorts out the core principles, method classifications, application scenarios, and future challenges of pruning technology, providing a comprehensive reference for the research and application of pruning technology.
Downloads
References
[1] Ganguli, T., & Chong, E. K. P. (2024). Activation-Based Pruning of Neural Networks. Algorithms, 17 (1), 48. https://doi.org/10.3390/a17010048. DOI: https://doi.org/10.3390/a17010048
[2] Yu, V. F., Santiyuda, G., Lin, S. -W., Pasaribu, U. S., & Afrianti, Y. S. (2025). Neural Network Pruning for Lightweight Metal Corrosion Image Segmentation Models. IEEE Access, 13, 71673 - 71687. https://doi.org/10.1109/ACCESS.2025.3562435. DOI: https://doi.org/10.1109/ACCESS.2025.3562435
[3] Yu, T., Zhang, C., Ma, M., & Wang, Y. (2023). Recursive least squares method for training and pruning convolutional neural networks. Applied Intelligence, 53 (24), 24603 - 24618. https://doi.org/10.1007/s10489 - 023 - 04740 - z. DOI: https://doi.org/10.1007/s10489-023-04740-z
[4] Ben Letaifa, L., & Rouas, J. -L. (2023). Variable Scale Pruning for Transformer Model Compression in End-to-End Speech Recognition. Algorithms, 16 (9), 398. https://doi.org/10.3390/a16090398. DOI: https://doi.org/10.3390/a16090398
[5] Rajpal, M., Zhang, Y., & Low, B. K. H. (2023). Pruning during training by network efficacy modeling. Machine Learning, 112 (14), 2653 - 2684. https://doi.org/10.1007/s10994 - 023 - 06304 - 1. DOI: https://doi.org/10.1007/s10994-023-06304-1
[6] Sivakumar M, Padmapriya S T. Improving Efficiency of Brain Tumor Classification Models Using Pruning Techniques [J]. Current Medical Imaging, 2024, 20 (1): e15734056303076. DOI: https://doi.org/10.2174/0115734056303076240614113525
[7] Malihi L, Heidemann G. Matching the ideal pruning method with knowledge distillation for optimal compression [J]. Applied System Innovation, 2024, 7 (4): 56. DOI: https://doi.org/10.3390/asi7040056
[8] Hu C, Zhang S, Tao K, et al. SFPBL: Soft Filter Pruning Based on Logistic Growth Differential Equation for Neural Network [J]. Computers, Materials & Continua, 2025, 82 (3). DOI: https://doi.org/10.32604/cmc.2025.059770
[9] Koo K, Kim H. V-skp: Vectorized kernel-based structured kernel pruning for accelerating deep convolutional neural networks [J]. IEEE Access, 2023, 11: 118547 - 118557. DOI: https://doi.org/10.1109/ACCESS.2023.3326534
[10] Bibi U, Mazhar M, Sabir D, et al. Advances in pruning and quantization for natural language processing [J]. IEEE Access, 2024. DOI: https://doi.org/10.1109/ACCESS.2024.3465631
[11] Tian D, Yamagiwa S, Wada K. Heuristic method for minimizing model size of CNN by combining multiple pruning techniques [J]. Sensors, 2022, 22 (15): 5874. DOI: https://doi.org/10.3390/s22155874
[12] Wu T, Song C, Zeng P. Model pruning based on filter similarity for edge device deployment [J]. Frontiers in Neurorobotics, 2023, 17: 1132679. DOI: https://doi.org/10.3389/fnbot.2023.1132679
[13] Pachon C G, Pinzon-Arenas J O, Ballesteros D. Pruning Policy for Image Classification Problems Based on Deep Learning [C]//Informatics. MDPI, 2024, 11 (3): 67. DOI: https://doi.org/10.3390/informatics11030067
[14] Ding Y, Chen D R. Optimization based layer-wise pruning threshold method for accelerating convolutional neural networks [J]. Mathematics, 2023, 11 (15): 3311. DOI: https://doi.org/10.3390/math11153311
[15] Macdonald C, Ounis I, Tonellotto N. Upper-bound approximations for dynamic pruning[J]. ACM Transactions on Information Systems (TOIS), 2011, 29 (4): 1 - 28. DOI: https://doi.org/10.1145/2037661.2037662
[16] Liu S Q, Yang Y X, Gao X J, et al. Dynamic channel pruning via activation gates [J]. Applied Intelligence, 2022, 52 (14): 16818 - 16831. DOI: https://doi.org/10.1007/s10489-022-03383-w
[17] Dai Q, Han X. An efficient ordering-based ensemble pruning algorithm via dynamic programming [J]. Applied Intelligence, 2016, 44 (4): 816 - 830. DOI: https://doi.org/10.1007/s10489-015-0729-z
[18] Gonzalez-Carabarin L, Huijben I Am M, Veeling B, et al. Dynamic probabilistic pruning: A general framework for hardware-constrained pruning at different granularities [J]. IEEE Transactions on Neural Networks and Learning Systems, 2022, 35 (1): 733 - 744. DOI: https://doi.org/10.1109/TNNLS.2022.3176809
[19] Pentsos V, Spantidi O, Anagnostopoulos I. Dynamic image difficulty-aware DNN pruning [J]. Micromachines, 2023, 14 (5): 908. DOI: https://doi.org/10.3390/mi14050908
[20] Zheng X, Yang C, Zhang S, et al. Ddpnas: Efficient neural architecture search via dynamic distribution pruning [J]. International Journal of Computer Vision, 2023, 131 (5): 1234 - 1249. DOI: https://doi.org/10.1007/s11263-023-01753-6
[21] Tsai C Y, Gao D Q, Ruan S J. An effective hybrid pruning architecture of dynamic convolution for surveillance videos [J]. Journal of Visual Communication and Image Representation, 2020, 70: 102798. DOI: https://doi.org/10.1016/j.jvcir.2020.102798
Downloads
Published
Issue
Section
License
Copyright (c) 2025 Transactions on Computer Science and Intelligent Systems Research
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
