Geodesic
Sentiment Polarity Analysis of Love Letters: Evaluation of TextBlob, Vader, Flair, and Hugging Face Transformer
Computer Science and Information Systems, Tome 21 (2024) no. 4.

Voir la notice de l'article provenant de la source Computer Science and Information Systems website

Sentiment analysis is the task of computationally identifying and quantifying the emotions and opinions expressed in text. However, existing sentiment analysis tools, while increasingly sophisticated, face challenges when applied to complex and personal domains such as love letters. This study investigates the performance and accuracy of four popular Python libraries for sentiment analysis (TextBlob, Vader, Flair, and Hugging Face Transformer) in determining the polarity and intensity of sentiments in love letters. A corpus of 300 love letters was collected and randomly sampled to provide 500 sentences for analysis. Due to the lack of labelled data, human experts participated in evaluating the quality and accuracy of the sentiment annotations.Inter-rater agreements were computed among four judges across randomly sampled sentence lots in two distinct blind rounds. The results reveal varying degrees of effectiveness and agreement among sentiment analysis tools (TextBlob, Vader, Flair, and Hugging Face) and human judges, with Cohen’s Kappa values showing low to moderate agreement (ranging from 0.09 to 0.77), and each tool demonstrating unique strengths—Vader excelling in sentiment intensity and Flair with Hugging Face better at contextual nuances—in handling the emotional complexity of the texts. The study also highlights limitations and proposes some custom metrics for evaluating sentiment analysis tools in the context of love letters, such as tenderness index, passion quotient, nostalgia score, and others. The findings contribute to the emerging field of sentiment analysis and provide insights for developing natural language models better suited for personal and emotionally charged domains.
Keywords: Sentiment Analysis, Polarity Analysis, Natural Language Processing 
@article{CSIS_2024_21_4_a12,
     author = {Gaganpreet Kaur and Amandeep Kaur and Meenu Khurana and Robertas Dama\v{s}evi\v{c}ius},
     title = {Sentiment {Polarity} {Analysis} of {Love} {Letters:} {Evaluation} of {TextBlob,} {Vader,} {Flair,} and {Hugging} {Face} {Transformer}},
     journal = {Computer Science and Information Systems},
     publisher = {mathdoc},
     volume = {21},
     number = {4},
     year = {2024},
     url = {http://geodesic.mathdoc.fr/item/CSIS_2024_21_4_a12/}
}
TY  - JOUR
AU  - Gaganpreet Kaur
AU  - Amandeep Kaur
AU  - Meenu Khurana
AU  - Robertas Damaševičius
TI  - Sentiment Polarity Analysis of Love Letters: Evaluation of TextBlob, Vader, Flair, and Hugging Face Transformer
JO  - Computer Science and Information Systems
PY  - 2024
VL  - 21
IS  - 4
PB  - mathdoc
UR  - http://geodesic.mathdoc.fr/item/CSIS_2024_21_4_a12/
ID  - CSIS_2024_21_4_a12
ER  - 
%0 Journal Article
%A Gaganpreet Kaur
%A Amandeep Kaur
%A Meenu Khurana
%A Robertas Damaševičius
%T Sentiment Polarity Analysis of Love Letters: Evaluation of TextBlob, Vader, Flair, and Hugging Face Transformer
%J Computer Science and Information Systems
%D 2024
%V 21
%N 4
%I mathdoc
%U http://geodesic.mathdoc.fr/item/CSIS_2024_21_4_a12/
%F CSIS_2024_21_4_a12
Gaganpreet Kaur; Amandeep Kaur; Meenu Khurana; Robertas Damaševičius. Sentiment Polarity Analysis of Love Letters: Evaluation of TextBlob, Vader, Flair, and Hugging Face Transformer. Computer Science and Information Systems, Tome 21 (2024) no. 4. http://geodesic.mathdoc.fr/item/CSIS_2024_21_4_a12/