Annotation of Facial Expressions (Navarasa) in Videos Using Deep Learning

Abstract

Facial expression plays a significant role in understanding human behavior, alongside other relevant elements such as body posture, gait, and hand gestures. In the context of Indian Classical Dance (ICD), these elements work together to convey the stories depicted by characters from Indian mythology. Accompanying songs and shlokas evoke emotions, reflected in performers' facial expressions known as the performance. In recent years, computer vision techniques have garnered Navarasa, or "nine emotions". Recognizing these expressions is vital for appreciating significant attention due to their applications in the active research field of emotion recognition in humans. This paper aims to develop a deep-learning algorithm to detect facial expressions, thereby enhancing the comprehension of ICD performances. Our approach decodes the meanings of Navarasas represented in images and videos by recognizing and annotating these emotions as performed. By utilizing image processing and transfer learning techniques, we achieved an accuracy of 92% in classifying and annotating the Navarasas.