Neural network-based sentiment classification of Russian sentences into four classes

M. A. Kosterin ; I. V. Paramonov

Modelirovanie i analiz informacionnyh sistem, Tome 29 (2022) no. 2, pp. 116-133.

Résumé

The paper is devoted to the classification of Russian sentences into four classes: positive, negative, mixed, and neutral. Unlike the majority of modern study in this area, the mixed sentiment class is introduced. Mixed sentiment sentences contain positive and negative sentiments simultaneously. To solve the problem, the following tools were applied: the attention-based LSTM neural network, the dual attention-based GRU neural network, the BERT neural network with several modifications of the output layer to provide classification into four classes. The experimental comparison of the efficiency of various neural networks were performed on three corpora of Russian sentences. Two of them consist of users' reviews: one with wear reviews and another with hotel reviews. The third corpus contains news from Russian media. The highest weighted F-measure in experiments (0.90) was achieved when using BERT on the wear reviews corpus, as well as the highest weighted F-measure for positive and negative sentences (0.92 and 0.93, respectively). The best classification results for neutral and mixed sentences were achieved on the news corpus. For them F-measure was 0.72 and 0.58, respectively. As a result of experiments, the significant superiority of the BERT transfer network was demonstrated in comparison with older neural networks LTSM and GRU, especially for classification of sentences with weakly expressed sentiments. The error analysis showed that “adjacent” (positive/negative and mixed) classes are worse classified with BERT than “opposite” classes (positive and negative, neutral and mixed).

Détail

Keywords: sentiment analysis, neural network-based classifier, natural language processing.
Mots-clés : BERT

Bibliographie

[1] C. Potts, Z. Wu, A. Geiger, and D. Kiela, Dynasent: A dynamic benchmark for sentiment analysis, 2020, arXiv: 2012.15349 [cs.CL]

[2] F. Hamborg and K. Donnay, “NewsMTSC: a dataset for (multi-) target-dependent sentiment classification in political news articles”, Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics, Association for Computational Linguistics (ACL), 2021, 1663–1675

[3] B. Liu, “Sentiment analysis and opinion mining”, Synthesis lectures on human language technologies, 5:1 (2012), 1–167 | DOI | MR

[4] O. Habimana, Y. Li, R. Li, X. Gu, and G. Yu, “Sentiment analysis using deep learning approaches: an overview”, Science China Information Sciences, 63:1 (2020), 1–36 | DOI | MR

[5] S. Smetanin and M. Komarov, “Deep transfer learning baselines for sentiment analysis in Russian”, Information Processing Management, 58:3 (2021), 102484 | DOI

[6] A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, and I. Sutskever, Language models are unsupervised multitask learners, Technical report, OpenAI, 2019

[7] Z. Yang, Z. Dai, Y. Yang, J. Carbonell, R. R. Salakhutdinov, and Q. V. Le, “XLNet: Generalized autoregressive pretraining for language understanding”, Advances in neural information processing systems, 32 (2019), 5754–5764

[8] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding, 2018, arXiv: 1810.04805v2 [cs.CL]

[9] N. Kalchbrenner, E. Grefenstette, and P. Blunsom, A convolutional neural network for modelling sentences, 2014, arXiv: 1404.2188 [cs.CL]

[10] I. Paramonov and A. Poletaev, “Adaptation of Semantic Rule-Based Sentiment Analysis Approach for Russian Language”, Proceedings of 30th Conference of Open Innovations Association FRUCT, IEEE, 2021, 155–164 | DOI

[11] K. Kenyon-Dean, E. Ahmed, S. Fujimoto, J. Georges-Filteau, C. Glasz, B. Kaur, A. Lalande, S. Bhanderi, R. Belfer, N. Kanagasabai, et al., “Sentiment analysis: It's complicated!”, Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, v. 1, Long Papers, 2018, 1886–1895

[12] X. Tan, Y. Cai, and C. Zhu, “Recognizing conflict opinions in aspect-level sentiment classification with dual attention networks”, Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP, 2019, 3426–3431 | MR

[13] M. Soleymani, D. Garcia, B. Jou, B. Schuller, S.-F. Chang, and M. Pantic, “A survey of multimodal sentiment analysis”, Image and Vision Computing, 65 (2017), 3–14 | DOI

[14] L. A. M. Oberländer and R. Klinger, “An analysis of annotated corpora for emotion classification in text”, Proceedings of the 27th International Conference on Computational Linguistics, 2018, 2104–2119

[15] A. Radford, R. Jozefowicz, and I. Sutskever, Learning to generate reviews and discovering sentiment, 2017, arXiv: 1704.01444v2 [cs.LG] | Zbl

[16] Y. Wang, M. Huang, L. Zhao, and X. Zhu, “Attention-based LSTM for aspect-level sentiment classification”, Proceedings of the 2016 conference on empirical methods in natural language processing, 2016, 606–615 | DOI

[17] P. Chen, Z. Sun, L. Bing, and W. Yang, “Recurrent attention network on memory for aspect sentiment analysis”, Proceedings of the 2017 conference on empirical methods in natural language processing, 2017, 452–461 | DOI

[18] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need”, Advances in neural information processing systems, 2017, 5998–6008

[19] R. Socher, A. Perelygin, J. Wu, J. Chuang, C. D. Manning, A. Y. Ng, and C. Potts, “Recursive deep models for semantic compositionality over a sentiment treebank”, Proceedings of the 2013 conference on empirical methods in natural language processing, 2013, 1631–1642

[20] S. Smetanin and M. Komarov, “Sentiment Analysis of Product Reviews in Russian using Convolutional Neural Networks”, IEEE 21st Conference on Business Informatics, CBI, v. 1, 2019, 482–486