Indian Sign Language Understanding Through Deep Transfer Learning and Vision Models
DOI:
https://doi.org/10.5281/zenodo.15745216Keywords:
Indian Sign Language (ISL), Sign Language Recognition, Hindi Sign Language (HSL), EfficientNetB0, Deep Learning, Transfer Learning, Gesture ClassificationAbstract
Communication barriers remain a significant challenge for individuals with hearing and speech impairments, especially in regions where sign language literacy among the general population is limited. The gap in accessible communication tools for the deaf and hard-of-hearing population remains a pressing issue, particularly in countries like India, Pakistan and Bangladesh, where public awareness and proficiency in sign language are minimal. This paper introduces EfficientSign-ISL, a robust and lightweight deep learning model for recognizing Indian Sign Language (ISL) gestures. The model is built upon the EfficientNetB0 architecture, which employs compound scaling to optimize accuracy and computational efficiency. To achieve this, we collected data from various individuals’ work, personal gatherings and augmented a dataset of several ISL gestures of 10 distinct classes. To verify our methodology's efficacy, we thoroughly compared well-known transfer learning models, such as ResNet50, MobileNetV2, and InceptionV3. With an excellent overall accuracy of 99.38%, an F1-score of 96.96%, a precision of 97.54%, and a recall of 97.10%, our suggested model fared better than these baselines. These results demonstrate the model’s strong classification capability while maintaining low complexity, making it highly suitable for mobile applications or edge device deployment.References
Powell, D. S. W., & McCoy, R. G. (2025). Hearing care within diabetes care: Strategies for direct inclusion. Clinical Diabetes, cd250002.
Batts, S., Pham, N., Tearney, G., & Stankovic, K. M. (2025). The state of high-resolution imaging of the human inner ear: A look into the black box. Advanced Science, e00556.
Penobsm, D. M., & Gomwalk, N. V. (2025). Effects of sign language interpreters’ services on academic performance of students with hearing impairment University of Jos. BW Academic Journal.
Dritsas, E., Trigka, M., Troussas, C., & Mylonas, P. (2025). Multi-modal interaction, interfaces, and communication: A survey. Multimodal Technologies and Interaction, 9(1), 6.
Saleed, F. M., Shabbir, M., Poudyal, S., & Amalanathan, G. M. (2025). A systematic review of Indian sign language detection systems: Evaluating the evolution, techniques, and challenges in ISL recognition technology. In Tools for Promoting Independent Living Skills in Individuals with Disabilities (pp. 297–326).
Rastogi, U., Mahapatra, R. P., & Kumar, S. (2025). Advancements in machine learning techniques for hand gesture-based sign language recognition: A comprehensive review. Archives of Computational Methods in Engineering, 1–38.
Khan, A., Jin, S., Lee, G.-H., Arzu, G. E., Nguyen, T. N., Dang, L. M., Choi, W., & Moon, H. (2025). Deep learning approaches for continuous sign language recognition: A comprehensive review. IEEE Access.
Navin, N., Farid, F. A., Rakin, R. Z., Tanzim, S. S., Rahman, M., Rahman, S., Uddin, J., & Karim, H. A. (2025). Bilingual sign language recognition: A YOLOv11-based model for Bangla and English alphabets. Journal of Imaging, 11(5), 134.
Rubaiyeat, H. A., Youssouf, N., Hasan, M. K., & Mahmud, H. (2025). BDSLW401: Transformer-based word-level Bangla sign language recognition using relative quantization encoding (RQE). arXiv Preprint arXiv:2503.02360.
Ahmed, S., Hossain, M., Amit, A. A., Tahsin, M., Mahmud, M., & Kaiser, M. S. (2025). Real-time Bangla sign language detection and recognition using YOLOv10. In International Conference on Trends in Computational and Cognitive Engineering (pp. 383–400). Springer.
Rafi, A. M., Nawal, N., Bayev, N. S. N., Nima, L., Shahnaz, C., & Fattah, S. A. (2019). Image-based Bengali sign language alphabet recognition for deaf and dumb community. In 2019 IEEE Global Humanitarian Technology Conference (GHTC) (pp. 1–7). IEEE.
Ahammad, K., Shawon, J. A. B., Chakraborty, P., Islam, M. J., & Islam, S. (2021). Recognizing Bengali sign language gestures for digits in real time using convolutional neural network. International Journal of Computer Science and Information Security (IJCSIS), 19(1).
Patel, U., & Ambekar, A. G. (2017). Moment based sign language recognition for Indian languages. In 2017 International Conference on Computing, Communication, Control and Automation (ICCUBEA) (pp. 1–6). IEEE.
Biyani, D., Doohan, N. V., Rode, M., & Jain, D. (2023). Real time sign language recognition using YOLOv5. In 2023 IEEE World Conference on Applied Intelligence and Computing (AIC) (pp. 582–588). IEEE.
Singh, A., Singh, S., & Mittal, A. (2023). A novel deep learning approach for recognition of Hindi vowels of Indian sign language. In International Conference on Systems, Control and Automation (pp. 463–474). Springer.
Hamza, H. M., & Wali, A. (2023). Pakistan sign language recognition: Leveraging deep learning models with limited dataset. Machine Vision and Applications, 34(5), 71.
Mirza, M. S., Munaf, S. M., Azim, F., Ali, S., & Khan, S. J. (2022). Vision-based Pakistani sign language recognition using bag-of-words and support vector machines. Scientific Reports, 12(1), 21325.
Prova, N. N. I. (2024). Improved solar panel efficiency through dust detection using the InceptionV3 transfer learning model. In 2024 8th International Conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud) (I-SMAC) (pp. 260–268). IEEE.
Prova, N. N. I. (2024). Enhancing fish disease classification in Bangladeshi aquaculture through transfer learning, and LIME interpretability techniques. In 2024 4th International Conference on Sustainable Expert Systems (ICSES) (pp. 1157–1163). IEEE.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Vishal Kumar Jaiswal
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
