Improving Sentiment Analysis in Persian text based on combination of Stacked Auto-Encoder and Transformer-BiLSTM-CNN

Abstract :

The expansion of the internet and the increasing amount of user-generated textual opinions on various topics have made sentiment analysis a crucial tool for understanding public sentiment towards different subjects. These insights are invaluable for businesses, policymakers, and society as a whole, but manually analyzing such a volume of data is costly and impractical. This study leverages automated and deep learning approaches by combining a Stacked Autoencoder (SAE) for feature extraction and a Transformer-BiLSTM-CNN model for sentiment classification, specifically designed for the Persian language. ParsBert, the Persian version of BERT, was used for data preprocessing. This combined approach demonstrated improved performance in key evaluation metrics such as accuracy, precision, recall, and F1 score, outperforming comparative models like Transformer-BiLSTM-CNN, SAE-LSTM, and CNN. Results on datasets including user reviews from the Taghcheh and Digikala platforms and Persian tweets affirm the effectiveness of this hybrid model.

References:

[1] S. Tam, R. BenSaid and Ö. Ö. Tanriöver, "A ConvBiLSTM Deep Learning Model-Based Approach for Twitter Sentiment Classification," IEEE Access, vol. 9, Mar. 2021
[2] F. Huang, X. Li, C. Yuan, S. Zhang, J. Zhang and S. Qiao, "Attention-Emotion-Enhanced Convolutional LSTM for Sentiment Analysis," IEEE Transactions on Neural Networks and Learning Systems, vol. 33, Sept. 2022
[3] L. Zhang, S. Wang and B. Liu, "Deep learning for sentiment analysis: A survey," wires data mining and knowledge discovery, vol. 8, Aug. 2018
[4] M. Dong, Y. Li, X. Tang, J. Xu, S. Bi and Y. Cai, " Variable Convolution and Pooling Convolutional Neural Network for Text Sentiment Classification," IEEE Access, vol. 8, jan. 2020
[5] L. Yue, W. Chen, X. Li, W. Zuo and M. Yin, "A survey of sentiment analysis in social media," Knowledge and Information Systems, vol. 60, July. 2018
[6] M. Shukla and A. Kumar, "An Experimental Analysis of Deep Neural Network Based Classifiers for Sentiment Analysis Task," IEEE Access, vol. 11, Apr. 2023
[7] M. Birjali, M. Kasri and A. B. Hssane, "A comprehensive survey on sentiment analysis: Approaches, challenges and trends," Knowledge-Based Systems, vol. 226, Aug. 2021
[8] M. U. Salur and I. Aydin, "A Novel Hybrid Deep Learning Model for Sentiment Classification," IEEE Access, vol. 8, Mar. 2020
[9] D. W. O. Otter, J. R. Medina and J. K. Kalita, " A Survey of the Usages of Deep Learning for Natural Language Processing," IEEE Transactions on Neural Networks and Learning Systems, vol. 32, Apr. 2020
[10] F. A. Acheampong, H. N. Mensah and W. Chen, "Transformer models for text‑based emotion detection: a review of BERT‑based approaches," Artificial Intelligence Review, vol. 54, Feb. 2021
[11] K. Berahmand, F. Daneshfar, E. S. Salehi, Y. Li and Y. Xu, " Autoencoders and their applications in machine learning: a survey," Artificial Intelligence Review, vol. 57, Feb. 2024
[12] I. Kanwal, F. Wahid, S. Ali, A. UrRehman, A. Alkhayyat and A. Al-Radaei, "Sentiment Analysis Using Hybrid Model of Stacked Auto-Encoder-Based Feature Extraction and Long Short Term Memory-Based Classification Approach," IEEE Access, vol. 11, Sept. 2023
[13] G. M. Shafiq, T. Hamza, M. F. Alrahmawy and R. El-Deeb, "Enhancing Arabic Aspect-Based Sentiment Analysis Using End-to-End Model," IEEE Access, vol. 11, Dec. 2023
[14] M. Aljebreen, B. Alabduallah, M. M. Asiri, A. S. Salama, M. Assiri and S. S. Ibrahim, "Moth Flame Optimization With Hybrid Deep Learning Based Sentiment Classification Toward ChatGPT on Twitter," IEEE Access, vol. 11, Sept. 2023
[15] I. Carvalho, H. G. Oliveira and C. Silva, "The Importance of Context for Sentiment Analysis in Dialogues," IEEE Access, vol. 11, Aug. 2023
[16] K. L. Tan, C. P. Lee, K. M. Lim and K. S. M. Anbananthen, "Sentiment Analysis With Ensemble Hybrid Deep Learning Model," IEEE Access, vol. 10, Sept. 2022