Application of deep neural networks for automatic irony detection in Russian texts

M. A. Kosterin   ; I. V. Paramonov

Modelirovanie i analiz informacionnyh sistem, Tome 31 (2024) no. 1, pp. 90-101

Résumé

The paper examines automatic methods for classifying Russian-language sentences into two classes: ironic and non-ironic. The discussed methods can be divided into three categories: classifiers based on language model embeddings, classifiers using sentiment information, and classifiers with embeddings trained to detect irony. The components of classifiers are neural networks such as BERT, RoBERTa, BiLSTM, CNN, as well as an attention mechanism and fully connected layers. The irony detection experiments were carried out using two corpora of Russian sentences: the first corpus is composed of journalistic texts from the OpenCorpora open corpus, the second corpus is an extension of the first one and is supplemented with ironic sentences from the Wiktionary resource. The best results were demonstrated by a group of classifiers based on embeddings of language models with the maximum F-measure of 0.84, achieved by a combination of RoBERTa, BiLSTM, an attention mechanism and a pair of fully connected layers in experiments on the extended corpus. In general, using the extended corpus produced results that were 2–5% higher than those of the basic corpus. The achieved results are the best for the problem under consideration in the case of the Russian language and are comparable to the best one for English.

Détail

Keywords: irony detection, sarcasm detection, neural network-based classifier, deep learning, natural language processing
Mots-clés : BERT.

Bibliographie

[1] M. Kosterin, I. Paramonov, and N. Lagutina, “Automatic Irony and Sarcasm Detection in Russian Sentences: Baseline Methods”, 33rd Conference of Open Innovations Association FRUCT, 2023, 148–154 | DOI

[2] D. Šandor and M. B. Babac, “Sarcasm detection in online comments using machine learning”, Information Discovery and Delivery, 2023 | DOI

[3] R. A. Potamias, G. Siolas, and A.-G. Stafylopatis, “A transformer-based approach to irony and sarcasm detection”, Neural Computing and Applications, 32 (2020), 17309–17320 | DOI

[4] C. Van Hee, E. Lefever, and V. Hoste, “Semeval-2018 task 3: Irony detection in English tweets”, Proceedings of The 12th International Workshop on Semantic Evaluation, 2018, 39–50 | DOI

[5] M. Khodak, N. Saunshi, and K. Vodrahalli, A large self-annotated corpus for sarcasm, 2017, arXiv: 1704.05579 [cs.CL]

[6] E. Riloff, A. Qadir, P. Surve, L. De Silva, N. Gilbert, and R. Huang, “Sarcasm as contrast between a positive sentiment and negative situation”, Proceedings of the 2013 conference on empirical methods in natural language processing, 2013, 704–714

[7] S. Zhang, X. Zhang, J. Chan, and P. Rosso, “Irony detection via sentiment-based transfer learning”, Information Processing Management, 56:5 (2019), 1633–1644 | DOI | MR

[8] D. Hazarika, S. Poria, S. Gorantla, E. Cambria, R. Zimmermann, and R. Mihalcea, Cascade: Contextual sarcasm detection in online discussion forums, 2018, arXiv: 1805.06413 [cs.CL]

[9] T. Zefirova and N. Loukachevitch, “Irony and sarcasm expression in Twitter”, EPiC Series in Language and Linguistics, 4 (2019), 45–49 | DOI

[10] A. A. Gurin and T. A. Zhukov, “Avtomaticheskoe opredelenie sarkazma v tekstakh na russkom yazyke”, Tsyfrovaya ekonomika, 1(22) (2023), 44–53

[11] A. D. Yacoub, S. Slim, and A. Aboutabl, “A Survey of Sentiment Analysis and Sarcasm Detection: Challenges, Techniques, and Trends”, International journal of electrical and computer engineering systems, 15:1 (2024), 69–78 | DOI

[12] Y. Kuratov and M. Arkhipov, Adaptation of deep bidirectional multilingual transformers for Russian language, 2019, arXiv: 1905.07213 [cs.CL]

[13] D. Zmitrovich et al., A family of pretrained transformer language models for Russian, 2023, arXiv: 2309.10931 [cs.CL]

[14] C. Zhou, C. Sun, Z. Liu, and F. Lau, A C-LSTM neural network for text classification, 2015, arXiv: 1511.08630 [cs.CL]

[15] A. Rogers, A. Romanov, A. Rumshisky, S. Volkova, M. Gronas, and A. Gribov, “RuSentiment: An enriched sentiment analysis dataset for social media in Russian”, Proceedings of the 27th international conference on computational linguistics, 2018, 755–763