ANALYSIS OF DIGITAL IMAGE RECOGNITION OF INDONESIAN SIGN LANGUAGE USING THE DEEP LEARNING CNN ARCHITECTURE VGG19 METHOD

Dimas Prayoga - [ https://orcid.org/0009-0007-1945-2161 ]
Ema Utami
Dhani Ariatmanto


DOI: https://doi.org/10.29100/jipi.v10i3.7353

Abstract


This study examines the application of the CNN method with the VGG19 architecture for digital image analysis in recognizing Indonesian sign language. The data used in this study is the BISINDO data set type, with 8,814 samples divided into 26 alphabetical categories. Implementing sign language recognition using the VGG19 architecture produces good accuracy results, reaching 93.24% with epoch 25 (without hyper-parameters tuning).These results confirm the model's extraordinary ability in image recognition and performing precise analysis. However, the results of this study can be improved again by performing Hyper parameters tuning on the architecture used, namely VGG19, by changing certain variables that affect increasing accuracy. Other aspects can be improved to achieve optimal performance, considering the excellent results. By integrating modern hyper-parameter tuning approaches and incorporating a variety of additional data, the model generalization is expected to be improved, leading to higher accuracy in many real-world settings

Keywords


Artificial Intelligence, Machine Learning, Deep Learning, Sign language

Full Text:

PDF

Article Metrics :

References


Nurul Khotimah, W., Andika Saputra, R., Suciati, N., & Rahman Hariadi, R. (n.d.). Khotimah, Saputra, Suciati, and Hariadi-Alphabet Sign Language Recognition Using Leap Motion Technology and Rule Based Backpropagation-Genetic Algorithm Neural Network (RBBPGANN) ALPHABET SIGN LANGUAGE RECOGNITION USING LEAP MOTION TECHNOLOGY AND RULE BASED BACKPROPAGATION-GENETIC ALGORITHM NEURAL NETWORK (RBBPGANN).

Kothadiya, D., Bhatt, C., Sapariya, K., Patel, K., Gil-González, A. B., & Corchado, J. M. (2022). Deepsign: Sign Language Detection and Recognition Using Deep Learning. Electronics (Switzerland), 11(11). https://doi.org/10.3390/electronics11111780

Bora, J., Dehingia, S., Chetia, A. B. A. A., & Gogoi, D. (2023). Real-time Assamese Sign Language Recognition using MediaPipe and Deep Learning. Procedia Computer Science, 2018, 1384–1393. https://doi.org/10.1016/j.procs.2023.01.117

Badiger, R. M., & Lamani, D. (2022). DEEP LEARNING BASED SOUTH INDIAN SIGN LANGUAGE RECOGNITION BY STACKED AUTOENCODER MODEL AND ENSEMBLE CLASSIFIER ON STILL IMAGES AND VIDEOS. Journal of Theoretical and Applied Information Technology, 100(21), 6587-6597. https://doi.org/10.1016/j.compeleceng.2020.106898

Mia, K., Islam, T., Assaduzzaman, M., Shaha, S. P., Saha, A., Razzak, M. A., Dhar, A., Sarker, T., & Alvy, A. I. (2023). Isolated sign language recognition using hidden transfer learning. International Journal of Computer Science and Information Technology Research, 11(1), 28-38. https://doi.org/10.5281/zenodo.7615847

Das, P., Ahmed, T., & Ali, M. F. (2020). Static Hand Gesture Recognition for American Sign Language using Deep Convolutional Neural Network. ResearchGate Conference Paper.

Shin, J., Musa Miah, A. S., Hasan, M. A. M., Hirooka, K., Suzuki, K., Lee, H. S., & Jang, S. W. (2023). Korean Sign Language Recognition Using Transformer-Based Deep Neural Network. Applied Sciences, 13(5), 3029.

Perdana, I., Darma Putra, I., & Arya Dharmaadi, I. (2021). Classification of Sign Language Numbers Using the CNN Method. JITTER : Jurnal Ilmiah Teknologi Dan Komputer, 2(3), 485-493. Retrieved from https://ojs.unud.ac.id/index.php/jitter/article/view/78264

Rexona. (n.d.). Mengenal Dua Jenis Bahasa Isyarat di Indonesia | Rexona ® ID. Retrieved July 19, 2023, from https://www.rexona.com/id/zona-keringat/mengenal-dua-jenis-bahasa-isyarat-di-indonesia/

Kembuan, O., Caren Rorimpandey, G., & Milian Tompunu Tengker, S. (2020, October 27). Convolutional Neural Network (CNN) for Image Classification of Indonesia Sign Language Using Tensorflow. 2020 2nd International Conference on Cybernetics and Intelligent System, ICORIS 2020. https://doi.org/10.1109/ICORIS50180.2020.9320810

Ma'aruf, A. (2024). Indonesian Sign Language - BISINDO. Kaggle. Retrieved January 13, 2025, from https://www.kaggle.com/datasets/agungmrf/indonesian-sign-language-bisindo

geeksforgeeks. (2023, May 6). Data Preprocessing in Data Mining. GeeksforGeeks. Retrieved July 19, 2023, from https://www.geeksforgeeks.org/data-preprocessing-in-data-mining/

Kaushik, A., & Castrejon, L. (n.d.). Understanding the VGG19 Architecture. OpenGenus IQ. Retrieved July 19, 2023, from https://iq.opengenus.org/vgg19-architecture/

Tanwar, K. (n.d.). What is the VGG-19 neural network? Quora. Retrieved July 19, 2023, from https://www.quora.com/What-is-the-VGG-19-neural-network

Erickson, B. J., & Kitamura, F. (2021). Magician’s corner: 9. performance metrics for machine learning models. In Radi-ology: Artificial Intelligence (Vol. 3, Issue 3). Radiological Society of North America Inc. https://doi.org/10.1148/ryai.2021200126

Yacouby Amazon Alexa, R., & Axman Amazon Alexa, D. (n.d.). Probabilistic Extension of Precision, Recall, and F1 Score for More Thorough Evaluation of Classification Models.

Trivusi. (2022, July 16). Metriks Evaluasi Sistem Menggunakan Confusion Matrix. Trivusi. Retrieved July 19, 2023, from https://www.trivusi.web.id/2022/04/evaluasi-sistem-dengan-confusion-matrix.html

Islam, T., Assaduzzaman, M., Mia, K., Shaha, S. P., Saha, A., Razzak, M. A., Dhar, A., Sarker, T., & Alvy, A. I. (2023). Isolated sign language recognition using hidden transfer learning. International Journal of Computer Science and Information Technology Research, 11, 28–38. https://doi.org/10.5281/zenodo.7615847