Vol. 2 No. 1 (2024): Issue - 01
Articles

iSignNet: A Novel Bidirectional Hybrid Sign Language Translation Framework Leveraging IoT and NLP for Intelligent Communication

DOWNLOAD PDF
  • Faizal Hussain,
  • MD Omar Mukhtar,
  • Kushal V,
  • Vivek R Madabhavi,
  • Chaithra M H,
  • Syed Muzamil Basha
Faizal Hussain
School of Computer Science and Engineering, REVA University, Bengaluru, India
MD Omar Mukhtar
School of Computer Science and Engineering, REVA University, Bengaluru, India
Kushal V
School of Computer Science and Engineering, REVA University, Bengaluru, India
Vivek R Madabhavi
School of Computer Science and Engineering, REVA University, Bengaluru, India
Chaithra M H
School of Computer Science and Engineering, REVA University, Bengaluru, India
Syed Muzamil Basha
School of Computer Science and Engineering, REVA University, Bengaluru, India
DOI: https://doi.org/10.5281/zenodo.14557864

Published 2024-12-26

Keywords

  • Sign to text,
  • Smart Gloves,
  • Arduino UNO R3,
  • Flex Sensor,
  • Speech Impaired,
  • Speech Recognition
    • ...More
    Less

How to Cite

Faizal Hussain, MD Omar Mukhtar, Kushal V, Vivek R Madabhavi, Chaithra M H, & Syed Muzamil Basha. (2024). iSignNet: A Novel Bidirectional Hybrid Sign Language Translation Framework Leveraging IoT and NLP for Intelligent Communication . Milestone Transactions on Medical Technometrics, 2(1), 64–78. https://doi.org/10.5281/zenodo.14557864

Abstract

Effective communication is essential, yet individuals with speech, hearing, or multiple impairments often face challenges, especially when others do not understand sign language. This paper introduces a system that translates sign language into text and speech, facilitating easier communication. The system includes five flex sensors attached to each finger to detect gestures, which are converted into binary signals processed by an Arduino Uno R3 microcontroller. These signals are displayed as text on a 16x2 LCD screen, supporting 32 gestures per phase, with multiple phases possible. Additionally, a DF Mini Player speaker module vocalizes the translated text for enhanced accessibility. The system also includes software that allows users to input text or speech. If a pre-recorded Blender 3D animation exists for a word, it is shown; if not, the system breaks the word down into characters and displays animations for each one. For sentences, it tokenizes and removes stop-words, showing animations for each word. If no animation exists, it displays each character individually. This hardware-software integration offers a cost-effective solution for those with speech and hearing impairments, using flex sensors to reduce costs and enhance precision, effectively bridging communication gaps.
DOWNLOAD PDF

References

  1. Haas, J. D., & Brownlie IV, T. (2001). Iron deficiency and reduced work capacity: a critical review of the research to determine a causal relationship. The Journal of nutrition, 131(2), 676S-690S.
  2. Kozuki, N., Lee, A. C., & Katz, J. (2012). Child Health Epidemiology Reference G. Moderate to severe, but not mild, maternal anemia is associated with increased risk of small-for-gestational-age outcomes. J Nutr, 142(2), 358-62.
  3. Steer, P. J. (2000). Maternal hemoglobin concentration and birth weight. The American journal of clinical nutrition, 71(5), 1285S-1287S.
  4. Amin, M. N., & Habib, M. A. (2015). Comparison of different classification techniques using WEKA for hematological data. American Journal of Engineering Research, 4(3), 55-61.
  5. Dogan, S., & Turkoglu, I. (2008). Iron-deficiency anemia detection from hematology parameters by using decision trees. International Journal of Science & Technology, 3(1), 85-92.
  6. Abdullah, M., & Al-Asmari, S. (2016). Anemia types prediction based on data mining classification algorithms. In Communication, management and information technology (pp. 629-636). CRC Press.
  7. Veluchamy, M., Perumal, K., & Ponuchamy, T. (2012). Feature extraction and classification of blood cells using artificial neural network. American journal of applied sciences, 9(5), 615.
  8. Bashir, S., Qamar, U., Khan, F. H., & Javed, M. Y. (2014, December). An efficient rule-based classification of Diabetes using ID3, C4. 5, & CART ensembles. In 2014 12th International Conference on Frontiers of Information Technology (pp. 226-231). IEEE.
  9. Sathiyamoorthi, V., Ilavarasi, A. K., Murugeswari, K., Ahmed, S. T., Devi, B. A., & Kalipindi, M. (2021). A deep convolutional neural network based computer aided diagnosis system for the prediction of Alzheimer's disease in MRI images. Measurement, 171, 108838.
  10. Swamy, R., Ahmed, S. T., Thanuja, K., Ashwini, S., Siddiqha, S., & Fathima, A. (2021, January). Diagnosing the level of Glaucoma from Fundus Image Using Empirical Wavelet Transform. In Proceedings of the First International Conference on Advanced Scientific Innovation in Science, Engineering and Technology, ICASISET 2020, 16-17 May 2020, Chennai, India.
  11. El-kenawy, E. S. M., Eid, M. M., & Ibrahim, A. (2021). Anemia estimation for covid-19 patients using a machine learning model. Journal of Computer Science and Information Systems, 17(11), 2535-1451.
  12. Sreedhar Kumar, S., Ahmed, S. T., & NishaBhai, V. B. (2019). Type of supervised text classification system for unstructured text comments using probability theory technique. International Journal of Recent Technology and Engineering (IJRTE), 8(10).