Geodesic
Improving RAG with LoRA finetuning for persona text generation
Zapiski Nauchnykh Seminarov POMI, Investigations on applied mathematics and informatics. Part IV, Tome 540 (2024), pp. 162-177

Voir la notice de l'article provenant de la source Math-Net.Ru

We address the challenge of maintaining consistency in Retrieval-Augmented Generation (RAG) systems for persona text generation when databases are subject to rapid updates and conventional large language model (LLM) fine-tuning is inadequate. We propose an approach that enhances an existing RAG system used for persona-based information retrieval in dialogue agents through the application of Low-Rank Adaptation fine-tuning on synthetic data. We find that this method improves the system's logic and correctness by 5% on SSA scores and ensures that generated content remains more coherent and contextually relevant. 
@article{ZNSL_2024_540_a8,
     author = {V. Pavliukevich and A. Zherdeva and O. Makhnytkina and D. Dyrmovskiy},
     title = {Improving {RAG} with {LoRA} finetuning for persona text generation},
     journal = {Zapiski Nauchnykh Seminarov POMI},
     pages = {162--177},
     publisher = {mathdoc},
     volume = {540},
     year = {2024},
     language = {en},
     url = {http://geodesic.mathdoc.fr/item/ZNSL_2024_540_a8/}
}
TY  - JOUR
AU  - V. Pavliukevich
AU  - A. Zherdeva
AU  - O. Makhnytkina
AU  - D. Dyrmovskiy
TI  - Improving RAG with LoRA finetuning for persona text generation
JO  - Zapiski Nauchnykh Seminarov POMI
PY  - 2024
SP  - 162
EP  - 177
VL  - 540
PB  - mathdoc
UR  - http://geodesic.mathdoc.fr/item/ZNSL_2024_540_a8/
LA  - en
ID  - ZNSL_2024_540_a8
ER  - 
%0 Journal Article
%A V. Pavliukevich
%A A. Zherdeva
%A O. Makhnytkina
%A D. Dyrmovskiy
%T Improving RAG with LoRA finetuning for persona text generation
%J Zapiski Nauchnykh Seminarov POMI
%D 2024
%P 162-177
%V 540
%I mathdoc
%U http://geodesic.mathdoc.fr/item/ZNSL_2024_540_a8/
%G en
%F ZNSL_2024_540_a8
V. Pavliukevich; A. Zherdeva; O. Makhnytkina; D. Dyrmovskiy. Improving RAG with LoRA finetuning for persona text generation. Zapiski Nauchnykh Seminarov POMI, Investigations on applied mathematics and informatics. Part IV, Tome 540 (2024), pp. 162-177. http://geodesic.mathdoc.fr/item/ZNSL_2024_540_a8/