Online Handwritten Physics Expression Recognition Using a CRNN-LSTM Approach

Abstract

Optical Character Recognition (OCR) is crucial for identifying and extracting information from scientific documents. However, recognizing online handwritten physics expressions remains a relatively unexplored area of research. In the existing literature, most of the researchers worked on CNN classification to identify mathematical expressions. The traditional Convolution Neural Network (CNN) has limitations for extracting sequential data. This study proposes a CRNN (Convolution Recurrent Neural Network)-LSTM (Long Short-Term Memory) model that uses sequence a to-sequence approach for recognition of online handwritten physics expressions to enhance the feature extraction process for the sequential data which overcomes the limitations of CNN. A dataset was constructed from 100 users using a Java-based interface, covering five commonly used types of physics expressions: (1) Electric flux; (2) Maxwell’s equations; (3) Inductance; (4) Pointing vector, and (5) Moment of inertia. This dataset identifies 25 unique Physics symbols. The Online Handwritten Physics Expression (OHPE) dataset comprises a total of 500 physics expressions, including operators, special symbols, alphabets, and numbers. The proposed architecture of CRNN-LSTM architecture consists of convolution layers followed by LSTM layers for sequence processing. CTC (Connectionist Temporal Classification) loss function is used to train the model, which is particularly used to predict the sequence of symbols within the physics expressions. The performance evaluation has been carried out and the accuracy of symbol level and expression level prediction is reported. In the final stage the trained model is evaluated to make the predictions on the unseen test set images with the recognition accuracy of 96.36% at the expression level and 98.10% at the symbol level.