Sign Language Recognition using Convolutional Neural Networks in Machine Learning
Main Article Content
Abstract
Sign language is a way or means of communication used by individuals with speaking and hearing impairments. It is one of the essential means of communication for such individuals to stay connected with the rest of the world and to express their ideas, needs or beliefs. There is a great need for an efficient and cost-effective real-time translation software or tool in the modern-day world to understand what the disabled individual is trying to express with accuracy. The proposed system is a real-time translation software or tool used for the conversion of hand gestures into natural languages such as English used by people for communication. The translated data will interpret the alphabet or number associated with the sign shown to the live camera feed. The software proposed in this project is created using Python, NumPy, OpenCV, LabelImg and TensorFlow. The image or video obtained from the camera device will be processed using convolutional neural networks (CNN). The CNN model is pre-trained with a large dataset from open sources or using a custom dataset on sign language gestures. Based on the recognition rate and prediction analysis from the CNN model, the provided image or video will be classified as the respective Alphabet or number from the American Sign Language Set. This helps the individuals to understand the sign language used by disabled individuals with ease.
Â
Â
Downloads
Article Details
COPYRIGHT
Submission of a manuscript implies: that the work described has not been published before, that it is not under consideration for publication elsewhere; that if and when the manuscript is accepted for publication, the authors agree to automatic transfer of the copyright to the publisher.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work
- The journal allows the author(s) to retain publishing rights without restrictions.
- The journal allows the author(s) to hold the copyright without restrictions.
References
N. Adaloglou, T. Chatzis, I. Papastratis, A. Stergioulas, Georgios Th. Papadopoulos, V. Zacharopoulou , G.J. Xydopoulos , Kl Atzakas2 , D. Papazachariou ,P. Daras, “A Comprehensive Study on Sign Language Recognition Methodsâ€, arXiv:2007.12530v1 [cs.CV] 24 Jul 2020
S. Bachani, S. Dixit, R. Chadha, A. Bagul, “Sign Language Recognition Using Neural Networkâ€, International Research Journal of Engineering and Technology (IRJET), Volume: 07 Issue: 04, 2020
L.K.S. Tolentino, Ronnie O.S Juan, August C. Thio-ac, M.A.B. Pamahoy, J.R.R. Forteza, Xavier J.O. Garcia, “Static Sign Language Recognition Using Deep Learningâ€, International Journal of Machine Learning and Computing, Vol. 9, No. 6, December 2019
R.G Rajan, M.J Leo, “A Comprehensive Analysis on Sign Language Recognition Systemâ€, International Journal of Recent Technology and Engineering (IJRTE), Volume-7, Issue-6, March 2019
S.Javed, Ghousia Banu S, J.A.S Kani and A. Rahman,†Wireless Glove for Hand Gesture Acknowledgment: Sign Language to Discourse Change Framework in Territorial Dialectâ€, Juniper publications, Robotics and Automation Engineering Journal, Volume 3 Issue 2 June 2018
Suharjitoa, R. Anderson, F. Wiryana, M.C Ariesta, G.P Kusuma, “Sign Language Recognition Application Systems for Deaf-Mute People: A Review Based On Input-Process-Outputâ€, 2nd International Conference on Computer Science and Computational Intelligence 2017, ICCSCI 2017, 13-14 October , Bali, Indonesia, 2017
Mrs.Dipali Rojasara Dr.Nehal G Chitaliya, “Indian Sign Language Recognition –A Surveyâ€, International Journal of Engineering Research & Technology (IJERT) Vol. 2 Issue 10, October – 2013
S.S Kumar, V.L. Iyangar, “Sign Language Recognition Using Machine Learningâ€, International Journal of Pure and Applied Mathematics Volume 119 No. 10 2018
Aditya Das, Shantanu Gawde, Khyati Suratwala, Dhananjay Kalbande, "Sign Language Recognition Using Deep Learning on Custom Processed Static Gesture Images", Smart City and Emerging Technology (ICSCET) 2018 International Conference on, pp. 1-6, 2018.
Shadman Shahriar, Ashraf Siddiquee, Tanveerul Islam, Abesh Ghosh, Rajat Chakraborty, Asir Intisar Khan, Celia Shahnaz, Shaikh Anowarul Fattah, "Real-Time American Sign Language Recognition Using Skin Segmentation and Image Category Classification with Convolutional Neural Network and Deep Learning", Region 10 Conference TENCON 2018 - 2018 IEEE, pp. 1168-1171, 2018.
Yuto Hirota, Tetsuya Oda, Nobuki Saito, Aoto Hirata, Masaharu Hirota, Kengo Katayama, "Proposal and Experimental Results of a DNN Based Real-Time Recognition Method for Ohsone Style Fingerspelling in Static Characters Environment", Consumer Electronics (GCCE) 2020 IEEE 9th Global Conference on, pp. 476-477, 2020.
KP Nimisha, Agnes Jacob, "A Brief Review of the Recent Trends in Sign Language Recognition", Communication and Signal Processing (ICCSP) 2020 International Conference on, pp. 186-190, 2020.
H. Muthu Mariappan and V. Gomathi, "Real-Time Recognition of Indian Sign Language," 2019 International Conference on Computational Intelligence in Data Science (ICCIDS), 2019, pp. 1-6, doi: 10.1109/ICCIDS.2019.8862125.
G. A. Rao, K. Syamala, P. V. V. Kishore and A. S. C. S. Sastry, "Deep convolutional neural networks for sign language recognition," 2018 Conference on Signal Processing And Communication Engineering Systems (SPACES), 2018, pp. 194-197, doi: 10.1109/SPACES.2018.8316344.
Ashwinkumar.U.M and Dr. Anandakumar K.R, "Predicting Early Detection of cardiac and Diabetes symptoms using Data mining techniques", International conference on computer Design and Engineering, vol.49, 2012.
Khalil Bousbai, Mostefa Merah, "A Comparative Study of Hand Gestures Recognition Based on MobileNetV2 and ConvNet Models", Image and Signal Processing and their Applications (ISPA) 2019 6th International Conference on, pp. 1-6, 2019.