banner
Home > Open Journal Systems

Indian sign language recognition and search results

Sandeep Musale, Kalyani Gargate, Vaishnavi Gulavani, Samruddhi Kadam, Shweta Kothawade

Abstract


Sign language is a medium of communication for people with hearing and speaking impairment. It uses gestures to convey messages. The proposed system focuses on using sign language in search engines and helping specially-abled people get the information they are looking for. Here, we are using Marathi sign language. Translation systems for Indian sign languages are not much simple and popular as American sign language. Marathi language consists of words with individual letters formed of two letter = Swara + Vyanjan (Mulakshar). Every Vyanjan or Swara individually has a unique sign which can be represented as image or video with still frames. Any letter formed of both Swara and Vyanjan is represented with hand gesture signing the Vyanjan as above and with movement of signed gesture in shape of Swara in Devnagari script. Such letters are represented with videos containing motion and frames in particular sequence. Further the predicted term can be searched on google using the sign search. The proposed system includes three important steps: 1) hand detection; 2) sign recognition using neural networks; 3) fetching search results. Overall, the system has great potential to help individuals with hearing and speaking impairment to access information on the internet through the use of sign language. It is a promising application of machine learning and deep learning techniques.


Keywords


Swar; Vyanjan; Mulakshar; Devnagari; neural networks; Indian sign language

Full Text:

PDF

References


1. Katoch S, Singh V, Tiwary US. Indian sign language recognition system using SURF with SVM and CNN. Array 2022; 14: 100141. doi: 10.1016/j.array.2022.100141

2. Huang J, Zhou W, Li H, Li W. Sign language recognition using 3D convolutional neural networks. In: Proceedings of the 2015 IEEE International Conference on Multimedia and Expo (ICME); 29 June–3 July 2015; Turin, Italy.

3. Haldera A, Tayadeb A. Real-time vernacular sign language recognition using mediapipe and machine learning. International Journal of Research Publication and Reviews 2021; 2(5): 9–17.

4. Mali D, Limkar N, Mali S. Indian sign language recognition using SVM classifier. In: Proceedings of the International Conference on Communication and Information Processing (ICCIP); 18 May 2019.

5. Patil R, Patil1 V, Bahuguna A, Datkhile G. Indian sign language recognition using convolutional neural network. In: ITM Web of Conferences, Proceedings of the International Conference on Automation, Computing and Communication 2021 (ICACC-2021); 14–15 July 2021; Nerul, India. EDP Sciences; 2021. Volume 40.

6. Rani RS, Rumana R, Prema R. A review paper on sign language recognition for the deaf and dumb. International Journal of Engineering Research & Technology (IJERT) 2021; 10(10). doi: 10.17577/IJERTV10IS100129

7. Sood D. Sign language recognition using deep learning. International Journal for Research in Applied Science & Engineering Technology (IJRASET) 2022; 10(Ⅲ). doi: 10.22214/ijraset.2022.40627

8. Darekar AA, Pawar NB, Pawar RD, et al. Marathi sign language recognition for physically disabled people. International Journal of Advanced Research in Science, Communication and Technology 2022; 2(7). doi: 10.48175/IJARSCT-4435

9. Shinde A, Kagalkar RM. Advanced Marathi sign language recognition using computer vision. International Journal of Computer Applications 2015; 118(13): 1–7. doi: 10.5120/20802-3485

10. Subramanian B, Olimov B, Naik SM, et al. An integrated mediapipe-optimized GRU model for Indian sign language recognition. Scientific Reports 2022; 12(1): 11964. doi: 10.1038/s41598-022-15998-7

11. Daroya R, Peralta D, Naval P. Alphabet sign language image classification using deep learning. In: Proceedings of the TENCON 2018–2018 IEEE Region 10 Conference; 28–31 October 2018; Jeju, Korea.

12. Patravali1 SD, Wayakule JM, Katre AD. Skin segmentation using YCBCR and RGB color models. International Journal of Advanced Research in Computer Science and Software Engineering 2014; 4(7): 341–346.

13. Mohamed SS, Tahir NM, Adnan R. Background modelling and background subtraction performance for object detection. In: Proceedings of the 2010 6th International Colloquium on Signal Processing & Its Applications; 21–23 May 2010; Malacca, Malaysia.




DOI: https://doi.org/10.32629/jai.v6i3.1000

Refbacks

  • There are currently no refbacks.


Copyright (c) 2023 Sandeep Musale, Kalyani Gargate, Vaishnavi Gulavani, Samruddhi Kadam, Shweta Kothawade

License URL: https://creativecommons.org/licenses/by-nc/4.0