Addressing Emergency Communication Challenges: Deep Learning Solutions for the Speech and Hearing- Impaired

Abstract

Emergency communication plays a very important role in ensuring that help reaches people promptly and safely in case of any emergency. However, the biggest problem faced by those who cannot speak and hear properly is to convey their message clearly and understand others. In this regard, the proposed research work focuses on recognizing emergency gestures made in the Indian Sign Language. It recognizes 14 categories of emergency gestures for various medical-related words. Two types of novel deep learning methods are used in the process to increase recognition efficiency such as the hybrid architecture of 3D Convolutional Neural Networks and Long Short-Term Memory networks and TimeSformer with DenseNet pre-trained network. For the evaluation of both models, two specially developed benchmark datasets have been used such as ISL_CSLTR and INCLUDE. The average accuracy obtained in the experiment using the TimeSformer architecture is 97% while for the hybrid approach is 91%.