Hybrid Deep Learning Models for Text Classification: Performance Evaluation of TriDistilBERT and BiGRU Architectures
- 1 Computers and Systems Department, Electronics Research Institute, Cairo, 12622, Egypt
Abstract
Text can be a valuable source of information, but its unstructured nature makes analysis challenging and time-consuming. Machine Learning (ML) algorithms can efficiently analyze and structure text, enabling organizations to automate processes and uncover insights that support better decision-making. This study focuses on applying ML to a classification problem using two datasets. Four deep learning models are introduced, combining Bi and Tri-layer hybrids of BERT and DistilBERT with a Bidirectional Gated Recurrent Unit (BiGRU) algorithm. These methods aim to enhance accuracy while examining the impact of hybridizing BERT and DistilBERT layers with BiGRU. The proposed models were evaluated against standalone BERT and DistilBERT approaches. Among them, the TriDistilBERT with BiGRU architecture achieved the highest accuracy, delivering 91.6% for the WASSA-17 dataset and 99.6% for the BBC dataset.
DOI: https://doi.org/10.3844/jcssp.2025.1983.1992
Copyright: © 2025 Amira Samy Talaat. This is an open access article distributed under the terms of the
Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
- 60 Views
- 8 Downloads
- 0 Citations
Download
Keywords
- Artificial Intelligence
- Intelligent Systems
- Machine Learning System
- Machine Learning Application
- Deep Learning
- BERT Model
- DistilBERT
- BiGRU
- Text Classification
- Sentiment Classification