Geodesic
Modern approaches to detect and classify comment toxicity using neural networks
Modelirovanie i analiz informacionnyh sistem, Tome 27 (2020) no. 1, pp. 48-61.

Voir la notice de l'article provenant de la source Math-Net.Ru

The growth of popularity of online platforms which allow users to communicate with each other, share opinions about various events, and leave comments boosted the development of natural language processing algorithms. Tens of millions of messages per day are published by users of a particular social network need to be analyzed in real time for moderation in order to prevent the spread of various illegal or offensive information, threats and other types of toxic comments. Of course, such a large amount of information can be processed quite quickly only automatically. That is why there is a need to and a way to teach computers to “understand” a text written by humans. It is a non-trivial task even if the word “understand” here means only “to classify”. The rapid evolution of machine learning technologies has led to ubiquitous implementation of new algorithms. A lot of tasks, which for many years were considered almost impossible to solve, are now quite successfully solved using deep learning technologies. This article considers algorithms built using deep learning technologies and neural networks which can successfully solve the problem of detection and classification of toxic comments. In addition, the article presents the results of the developed algorithms, as well as the results of the ensemble of all considered algorithms on a large training set collected and tagged by Google and Jigsaw.
Keywords: toxicity, Natural Language Processing, deep learning, word embedding, GloVe, recurrent neural networks, convolutional neural networks, CNN, LSTM
Mots-clés : NLP, FastText, GRU. 
@article{MAIS_2020_27_1_a3,
     author = {S. V. Morzhov},
     title = {Modern approaches to detect and classify comment toxicity using neural networks},
     journal = {Modelirovanie i analiz informacionnyh sistem},
     pages = {48--61},
     publisher = {mathdoc},
     volume = {27},
     number = {1},
     year = {2020},
     language = {ru},
     url = {http://geodesic.mathdoc.fr/item/MAIS_2020_27_1_a3/}
}
TY  - JOUR
AU  - S. V. Morzhov
TI  - Modern approaches to detect and classify comment toxicity using neural networks
JO  - Modelirovanie i analiz informacionnyh sistem
PY  - 2020
SP  - 48
EP  - 61
VL  - 27
IS  - 1
PB  - mathdoc
UR  - http://geodesic.mathdoc.fr/item/MAIS_2020_27_1_a3/
LA  - ru
ID  - MAIS_2020_27_1_a3
ER  - 
%0 Journal Article
%A S. V. Morzhov
%T Modern approaches to detect and classify comment toxicity using neural networks
%J Modelirovanie i analiz informacionnyh sistem
%D 2020
%P 48-61
%V 27
%N 1
%I mathdoc
%U http://geodesic.mathdoc.fr/item/MAIS_2020_27_1_a3/
%G ru
%F MAIS_2020_27_1_a3
S. V. Morzhov. Modern approaches to detect and classify comment toxicity using neural networks. Modelirovanie i analiz informacionnyh sistem, Tome 27 (2020) no. 1, pp. 48-61. http://geodesic.mathdoc.fr/item/MAIS_2020_27_1_a3/

[1] Toxic comment classification challenge, https://www.kaggle.com/c/jigsaw-toxic-comment-classification-challenge/overview

[2] S. V. Georgakopoulos, S. K. Tasoulis, A. G. Vrahatis, V. P. Plagianakos, “Convolutional neural networks for toxic comment classification”, Proceedings of the 10th hellenic conference on artificial intelligence, 2018, 1–6, arXiv: 1802.09957 | MR

[3] M. Kohli, E. Kuehler, J. Palowitch, Paying attention to toxic comments online https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1184/reports/6856482.pdf

[4] T. Chu, J. K., M. Wang, Comment abuse classification with deep learning https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1174/reports/2762092.pdf

[5] K. Khieu, N. Narwal, Detecting and classifying toxic comments https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1184/reports/6837517.pdf

[6] S. Hochreiter, J. Schmidhuber, “Long short-term memory”, Neural computation, 9:8 (1997), 1735–1780

[7] K. Cho, B. Van Merriënboer, C. Gulcehre, D. Bahdanau, F. Bougares, H. Schwenk, Y. Bengio, Learning phrase representations using RNN encoder-decoder for statistical machine translation, 2014, arXiv: 1406.1078

[8] J. Pennington, R. Socher, C. Manning, “Glove: global vectors for word representation”, Proceedings of the 2014 conference on empirical methods in natural language processing, EMNLP, 2014, 1532–1543

[9] A. Joulin, E. Grave, P. Bojanowski, T. Mikolov, “Bag of tricks for efficient text classification”, Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics, v. 2, 2017, 427–431

[10] J. Chung, C. Gulcehre, K. Cho, Y. Bengio, Empirical evaluation of gated recurrent neural networks on sequence modeling, 2014, arXiv: 1412.3555

[11] D. Bahdanau, K. Cho, Y. Bengio, Neural machine translation by jointly learning to align and translate, 2014, arXiv: 1409.0473

[12] Z. Yang, D. Yang, C. Dyer, X. He, A. Smola, E. Hovy, “Hierarchical attention networks for document classification”, Proceedings of NAACL-HLT, 2016, 1480–1489 https://www.cs.cmu.edu/h̃ovy/papers/16HLT-hierarchical-attention-networks.pdf

[13] M. Hughes, I. Li, S. Kotoulas, T. Suzumura, “Medical text classification using convolutional neural networks”, Stud. Health Technol. Inform., 235 (2017), 246–50

[14] K. Kowsari, K. Jafari Meimandi, M. Heidarysafa, S. Mendu, L. Barnes, D. Brown, “Text classification algorithms: a survey”, Information, 10:4 (2019), 150