Sign Language Recognition using Convolutional Neural Networks in Machine Learning

Main Article Content

Aishwarya Girish Menon
Anusha S.N
Arshia George
A. Abhishek
Gopinath. R


Sign language is a way or means of communication used by individuals with speaking and hearing impairments. It is one of the essential means of communication for such individuals to stay connected with the rest of the world and to express their ideas, needs or beliefs. There is a great need for an efficient and cost-effective real-time translation software or tool in the modern-day world to understand what the disabled individual is trying to express with accuracy. The proposed system is a real-time translation software or tool used for the conversion of hand gestures into natural languages such as English used by people for communication. The translated data will interpret the alphabet or number associated with the sign shown to the live camera feed. The software proposed in this project is created using Python, NumPy, OpenCV, LabelImg and TensorFlow. The image or video obtained from the camera device will be processed using convolutional neural networks (CNN). The CNN model is pre-trained with a large dataset from open sources or using a custom dataset on sign language gestures. Based on the recognition rate and prediction analysis from the CNN model, the provided image or video will be classified as the respective Alphabet or number from the American Sign Language Set. This helps the individuals to understand the sign language used by disabled individuals with ease.




Download data is not yet available.

Article Details



N. Adaloglou, T. Chatzis, I. Papastratis, A. Stergioulas, Georgios Th. Papadopoulos, V. Zacharopoulou , G.J. Xydopoulos , Kl Atzakas2 , D. Papazachariou ,P. Daras, “A Comprehensive Study on Sign Language Recognition Methodsâ€, arXiv:2007.12530v1 [cs.CV] 24 Jul 2020

S. Bachani, S. Dixit, R. Chadha, A. Bagul, “Sign Language Recognition Using Neural Networkâ€, International Research Journal of Engineering and Technology (IRJET), Volume: 07 Issue: 04, 2020

L.K.S. Tolentino, Ronnie O.S Juan, August C. Thio-ac, M.A.B. Pamahoy, J.R.R. Forteza, Xavier J.O. Garcia, “Static Sign Language Recognition Using Deep Learningâ€, International Journal of Machine Learning and Computing, Vol. 9, No. 6, December 2019

R.G Rajan, M.J Leo, “A Comprehensive Analysis on Sign Language Recognition Systemâ€, International Journal of Recent Technology and Engineering (IJRTE), Volume-7, Issue-6, March 2019

S.Javed, Ghousia Banu S, J.A.S Kani and A. Rahman,†Wireless Glove for Hand Gesture Acknowledgment: Sign Language to Discourse Change Framework in Territorial Dialectâ€, Juniper publications, Robotics and Automation Engineering Journal, Volume 3 Issue 2 June 2018

Suharjitoa, R. Anderson, F. Wiryana, M.C Ariesta, G.P Kusuma, “Sign Language Recognition Application Systems for Deaf-Mute People: A Review Based On Input-Process-Outputâ€, 2nd International Conference on Computer Science and Computational Intelligence 2017, ICCSCI 2017, 13-14 October , Bali, Indonesia, 2017

Mrs.Dipali Rojasara Dr.Nehal G Chitaliya, “Indian Sign Language Recognition –A Surveyâ€, International Journal of Engineering Research & Technology (IJERT) Vol. 2 Issue 10, October – 2013

S.S Kumar, V.L. Iyangar, “Sign Language Recognition Using Machine Learningâ€, International Journal of Pure and Applied Mathematics Volume 119 No. 10 2018

Aditya Das, Shantanu Gawde, Khyati Suratwala, Dhananjay Kalbande, "Sign Language Recognition Using Deep Learning on Custom Processed Static Gesture Images", Smart City and Emerging Technology (ICSCET) 2018 International Conference on, pp. 1-6, 2018.

Shadman Shahriar, Ashraf Siddiquee, Tanveerul Islam, Abesh Ghosh, Rajat Chakraborty, Asir Intisar Khan, Celia Shahnaz, Shaikh Anowarul Fattah, "Real-Time American Sign Language Recognition Using Skin Segmentation and Image Category Classification with Convolutional Neural Network and Deep Learning", Region 10 Conference TENCON 2018 - 2018 IEEE, pp. 1168-1171, 2018.

Yuto Hirota, Tetsuya Oda, Nobuki Saito, Aoto Hirata, Masaharu Hirota, Kengo Katayama, "Proposal and Experimental Results of a DNN Based Real-Time Recognition Method for Ohsone Style Fingerspelling in Static Characters Environment", Consumer Electronics (GCCE) 2020 IEEE 9th Global Conference on, pp. 476-477, 2020.

KP Nimisha, Agnes Jacob, "A Brief Review of the Recent Trends in Sign Language Recognition", Communication and Signal Processing (ICCSP) 2020 International Conference on, pp. 186-190, 2020.

H. Muthu Mariappan and V. Gomathi, "Real-Time Recognition of Indian Sign Language," 2019 International Conference on Computational Intelligence in Data Science (ICCIDS), 2019, pp. 1-6, doi: 10.1109/ICCIDS.2019.8862125.

G. A. Rao, K. Syamala, P. V. V. Kishore and A. S. C. S. Sastry, "Deep convolutional neural networks for sign language recognition," 2018 Conference on Signal Processing And Communication Engineering Systems (SPACES), 2018, pp. 194-197, doi: 10.1109/SPACES.2018.8316344.

Ashwinkumar.U.M and Dr. Anandakumar K.R, "Predicting Early Detection of cardiac and Diabetes symptoms using Data mining techniques", International conference on computer Design and Engineering, vol.49, 2012.

Khalil Bousbai, Mostefa Merah, "A Comparative Study of Hand Gestures Recognition Based on MobileNetV2 and ConvNet Models", Image and Signal Processing and their Applications (ISPA) 2019 6th International Conference on, pp. 1-6, 2019.