Geodesic
Blending of predictions boosts understanding for multimodal advertisements
Zapiski Nauchnykh Seminarov POMI, Investigations on applied mathematics and informatics. Part II–1, Tome 529 (2023), pp. 176-196

Voir la notice de l'article provenant de la source Math-Net.Ru

The advertising industry employs several content modalities to deliver implied messages: images, videos, text, music, and all of them combined. “Decoding” a message implied by multimodal content often requires both text and visual components. We study the tasks of multimodal symbolism prediction, topic detection, and sentiment type classification. Motivated by the difference in parts of the message conveyed by two modalities in advertisements, we train separate models for images and texts and significantly improve upon current state of the art by blending image- and text-based predictions (with OCR-extracted text), providing a comprehensive experimental validation of our approach. 
@article{ZNSL_2023_529_a11,
     author = {A. Alekseev and A. Savchenko and E. Tutubalina and E. Myasnikov and S. Nikolenko},
     title = {Blending of predictions boosts understanding for multimodal advertisements},
     journal = {Zapiski Nauchnykh Seminarov POMI},
     pages = {176--196},
     publisher = {mathdoc},
     volume = {529},
     year = {2023},
     language = {en},
     url = {http://geodesic.mathdoc.fr/item/ZNSL_2023_529_a11/}
}
TY  - JOUR
AU  - A. Alekseev
AU  - A. Savchenko
AU  - E. Tutubalina
AU  - E. Myasnikov
AU  - S. Nikolenko
TI  - Blending of predictions boosts understanding for multimodal advertisements
JO  - Zapiski Nauchnykh Seminarov POMI
PY  - 2023
SP  - 176
EP  - 196
VL  - 529
PB  - mathdoc
UR  - http://geodesic.mathdoc.fr/item/ZNSL_2023_529_a11/
LA  - en
ID  - ZNSL_2023_529_a11
ER  - 
%0 Journal Article
%A A. Alekseev
%A A. Savchenko
%A E. Tutubalina
%A E. Myasnikov
%A S. Nikolenko
%T Blending of predictions boosts understanding for multimodal advertisements
%J Zapiski Nauchnykh Seminarov POMI
%D 2023
%P 176-196
%V 529
%I mathdoc
%U http://geodesic.mathdoc.fr/item/ZNSL_2023_529_a11/
%G en
%F ZNSL_2023_529_a11
A. Alekseev; A. Savchenko; E. Tutubalina; E. Myasnikov; S. Nikolenko. Blending of predictions boosts understanding for multimodal advertisements. Zapiski Nauchnykh Seminarov POMI, Investigations on applied mathematics and informatics. Part II–1, Tome 529 (2023), pp. 176-196. http://geodesic.mathdoc.fr/item/ZNSL_2023_529_a11/