An Efficient Deep Learning Model for Simultaneous Detection of Human Body, Hands, and Face

Andi Regina Ayu Karlina, Hebron Prasetya, Alwin Melkie Sambul, Vecky Canisius Poekoel, Muhammad Dwisnanto Putro

Abstract


Accurate and efficient detection of multi-scale human body parts in a single pass, particularly the body, hands, and face, is essential for real-world applications such as pedestrian safety systems and fine-grained human-computer interaction. Existing detectors typically rely on multi-model pipelines or heavyweight architectures, resulting in increased complexity and limited real-time performance on edge devices. This study proposes a deep learning-based simultaneous human-body, hand, and face detection approach by modifying the YOLO11-nano architecture to obtain a more lightweight and efficient model on the Human Parts dataset. The proposed modifications focus on reducing the number of channels in the backbone and neck, thereby achieving an approximately 10% reduction, decreasing 2,590,425 to 2,330,687 of the model parameters and the computational complexity from 6.4 GFLOPs to 6.2 GFLOPs. Experimental results demonstrate that the proposed model achieves 93.5% mAP@0.5 and with 17.19 frame per second, while maintaining comparable detection accuracy to one-stage detectors. These results indicate a more favorable accuracy and efficiency trade-off, making the proposed model suitable for deployment in safety-critical and interactive real-world applications with limited computational resources.

Keywords


Deep Learning; Human-parts Detection; Lightweight Detection; YOLO11

Full Text:

PDF

References


D.-L. Nguyen, X.-T. Vo, A. Priadana, and K.-H. Jo, ‘Simultaneous person, face, and hand detector based on improved yolov5’, in 2023 International Workshop on Intelligent Systems (IWIS), 2023, pp. 1–7.

Y. Xu, Y. Zhang, Y. Leng, and Q. Gao, ‘AIP-Net: An anchor-free instance-level human part detection network’, Neurocomputing, vol. 573, p. 127254, 2024.

Z. Gao et al., ‘Pbadet: A one-stage anchor-free approach for part-body association’, arXiv preprint arXiv:2402. 07814, 2024.

Q. Li, Y. Bi, R. Cai, and J. Li, ‘Occluded pedestrian detection through bi-center prediction in anchor-free network’, Neurocomputing, vol. 507, pp. 199–207, 2022.

P. Danesh, E. Amini, and H. Kusetogullari, ‘Efficient CNN-based Object ID Association Model for Multiple Object Tracking’, in 2024 4th Internation-al Conference on Electrical, Computer, Communications and Mechatronics Engineering (ICECCME), 2024, pp. 1–6.

H. Zhou, F. Jiang, J. Si, Y. Ding, and H. Lu, ‘BPJDET: extended object representation for generic body-part joint detection’, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 46, no. 6, pp. 4314–4330, 2024.

N. Perwaiz, M. M. Fraz, and M. Shahzad, ‘Smart surveillance with simultaneous person detection and re-identification’, Multimedia tools and applica-tions, vol. 83, no. 5, pp. 15461–15482, 2024.

W. Chen, H. Huang, S. Peng, C. Zhou, and C. Zhang, ‘YOLO-face: a real-time face detector’, The Visual Computer, vol. 37, no. 4, pp. 805–813, 2021.

Z. Wang, B. Huang, G. Wang, P. Yi, and K. Jiang, ‘Masked face recognition dataset and application’, IEEE Transactions on Biometrics, Behavior, and Identity Science, vol. 5, no. 2, pp. 298–304, 2023.

M. Pu, C. Y. Chong, and M. K. Lim, ‘Robustness evaluation in hand pose estimation models using metamorphic testing’, in 2023 IEEE/ACM 8th Inter-national Workshop on Metamorphic Testing (MET), 2023, pp. 31–38.

A. Mujahid et al., ‘Real-time hand gesture recognition based on deep learning YOLOv3 model’, Applied Sciences, vol. 11, no. 9, p. 4164, 2021.

C. M. Badgujar, A. Poulose, and H. Gan, ‘Agricultural object detection with You Only Look Once (YOLO) Algorithm: A bibliometric and systematic literature review’, Computers and Electronics in Agriculture, vol. 223, p. 109090, 2024.

E. Arkin, N. Yadikar, Y. Muhtar, and K. Ubul, ‘A survey of object detection based on CNN and transformer’, in 2021 IEEE 2nd international confer-ence on pattern recognition and machine learning (PRML), 2021, pp. 99–108.

Z. Zou, K. Chen, Z. Shi, Y. Guo, and J. Ye, ‘Object detection in 20 years: A survey’, Proceedings of the IEEE, vol. 111, no. 3, pp. 257–276, 2023.

L. Liu et al., ‘Deep learning for generic object detection: A survey’, International journal of computer vision, vol. 128, no. 2, pp. 261–318, 2020.

M.-T. Pham, L. Courtrai, C. Friguet, S. Lefèvre, and A. Baussard, ‘YOLO-Fine: One-stage detector of small objects under various backgrounds in remote sensing images’, Remote Sensing, vol. 12, no. 15, p. 2501, 2020.

C.-Y. Wang, H.-Y. M. Liao, and Others, ‘YOLOv1 to YOLOv10: The fastest and most accurate real-time object detection systems’, APSIPA Transac-tions on Signal and Information Processing, vol. 13, no. 1, 2024.

Y. Ji, D. Zhang, Y. He, J. Zhao, X. Duan, and T. Zhang, ‘Improved YOLO11 Algorithm for Insulator Defect Detection in Power Distribution Lines’, Electronics, vol. 14, no. 6, p. 1201, 2025.

F. Ludja, R. M. Lintong, F. Sumarauw, A. M. Sambul, S. R. Sentinuwo, and M. D. Putro, ‘DOLPHIN DETECTION USING AN ENHANCED LIGHTWEIGHT YOLO ARCHITECTURE’, JIPI (Jurnal Ilmiah Penelitian dan Pembelajaran Informatika), vol. 10, no. 3, pp. 2874–2884, 2025.

R. Khanam and M. Hussain, ‘Yolov11: An overview of the key architectural enhancements’, arXiv preprint arXiv:2410. 17725, 2024.

L. T. Ramos and A. D. Sappa, ‘A decade of you only look once (yolo) for object detection: A review’, IEEE Access, vol. 13, pp. 192747–192794, 2025.

W. Kurniawan, A. Kurniasih, and M. A. Ghani, ‘Real or Deepfake Face Detection in Images and Video Data using YOLO11 Algorithm’, Journal of Artificial Intelligence and Engineering Applications (JAIEA), vol. 4, no. 2, pp. 1514–1521, 2025.

Q. Wang and Q. L. Wang, ‘BT-YOLO11: Automatic Driving Road Target Detection in Complex Scenarios’, IEEE Access, 2025.

L. Wang, L. Ye, X. Chen, and N. Chu, ‘MSFN-YOLOv11: A Novel Multi-Scale Feature Fusion Recognition Model Based on Improved YOLOv11 for Real-Time Monitoring of Birds in Wetland Ecosystems’, Animals, vol. 15, no. 23, p. 3472, 2025.

X. Gao, C. Cao, and X. Yi, ‘Using the improved YOLOv11 model to enhance computer vision applications for building crack detection algorithms’, Scientific Reports, vol. 15, no. 1, p. 38843, 2025.




DOI: https://doi.org/10.29100/jipi.v11i3.9885

Refbacks

  • There are currently no refbacks.


Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

JIPI (Jurnal Ilmiah Penelitian dan Pembelajaran Informatika)
ISSN 2540-8984
Published by
Prodi Pendidikan Teknologi Informasi
Universitas Bhinneka PGRI

Website :https://jurnal.stkippgritulungagung.ac.id/index.php/jipi/index
Email: jipistkippti@gmail.com


Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.