Geodesic
Automatic grading of answers in knowledge control for “Definition” and “Description” question types
Učënye zapiski Kazanskogo universiteta. Seriâ Fiziko-matematičeskie nauki, Uchenye Zapiski Kazanskogo Universiteta. Seriya Fiziko-Matematicheskie Nauki, Tome 166 (2024) no. 4, pp. 580-593 Cet article a éte moissonné depuis la source Math-Net.Ru

Voir la notice du chapitre de livre

The progress in developing an effective automatic knowledge control system is directly associated with creating and implementing a software module for grading answers to test questions formulated in natural language. Previously, an experimental prototype of such a system was designed, and a study was performed where short answers to basic question types provided by students were examined via a pragmatically oriented question-answer text processing algorithm, considering its outputs and exposing flaws. This article introduces the next iteration of the algorithm tailored to handle more complex question types that requires the identification of relations such as “Definition” and “Description.” The key features of the enhanced algorithm were outlined, with a particular focus on the problem of segmenting answers into meaningful chunks, a task for which processing methods have already been found. The results of an experiment based on the developed prototype with obtaining answers from students and a thorough analysis of the instances of the system’s incorrect behavior were discussed.
Keywords: natural language processing, automatic answer grading, e-learning. 
@article{UZKU_2024_166_4_a8,
     author = {N. A. Prokopyev},
     title = {Automatic grading of answers in knowledge control for {{\textquotedblleft}Definition{\textquotedblright}} and {{\textquotedblleft}Description{\textquotedblright}} question types},
     journal = {U\v{c}\"enye zapiski Kazanskogo universiteta. Seri\^a Fiziko-matemati\v{c}eskie nauki},
     pages = {580--593},
     year = {2024},
     volume = {166},
     number = {4},
     language = {ru},
     url = {http://geodesic.mathdoc.fr/item/UZKU_2024_166_4_a8/}
}
TY  - JOUR
AU  - N. A. Prokopyev
TI  - Automatic grading of answers in knowledge control for “Definition” and “Description” question types
JO  - Učënye zapiski Kazanskogo universiteta. Seriâ Fiziko-matematičeskie nauki
PY  - 2024
SP  - 580
EP  - 593
VL  - 166
IS  - 4
UR  - http://geodesic.mathdoc.fr/item/UZKU_2024_166_4_a8/
LA  - ru
ID  - UZKU_2024_166_4_a8
ER  - 
%0 Journal Article
%A N. A. Prokopyev
%T Automatic grading of answers in knowledge control for “Definition” and “Description” question types
%J Učënye zapiski Kazanskogo universiteta. Seriâ Fiziko-matematičeskie nauki
%D 2024
%P 580-593
%V 166
%N 4
%U http://geodesic.mathdoc.fr/item/UZKU_2024_166_4_a8/
%G ru
%F UZKU_2024_166_4_a8
N. A. Prokopyev. Automatic grading of answers in knowledge control for “Definition” and “Description” question types. Učënye zapiski Kazanskogo universiteta. Seriâ Fiziko-matematičeskie nauki, Uchenye Zapiski Kazanskogo Universiteta. Seriya Fiziko-Matematicheskie Nauki, Tome 166 (2024) no. 4, pp. 580-593. http://geodesic.mathdoc.fr/item/UZKU_2024_166_4_a8/

[1] Khasianov A., Suleymanov D., Marchenko A., “Three agent platform approach for digital education environment”, INTED2017 Proc., 2017, 8580–8587 | DOI

[2] Suleymanov D., Prokopyev N., “Development of prototype of natural language answer processor for e-learning”, Artificial Intelligence, RCAI 2020, Proc. 18th Russ. Conf., Lecture Notes in Computer Science, 12412, Springer, Cham, 2020, 448–459 | DOI

[3] Suleymanov Dzh., Prokopyev N., “Experimental prototype of the pragmatically oriented e-assessment algorithm for automatic natural language answer grading”, EDULEARN20 Proc., 2020, 1917–1924 | DOI

[4] Suleymanov D.Sh. A system for semantic processing of natural language responses, Ontol. Proekt., 2014, no. 1 (11), 65–77 (In Russian)

[5] Burrows S., Gurevych I., Stein B., “The eras and trends of automatic short answer grading”, Int. J. Artif. Intell. Educ., 25 (2015), 60–117 | DOI

[6] Mishunin O.B., Savinov A.P., Firstov D.I., “Current progress and development trends of systems for automatic grading of free answers in natural language”, Sovrem. Naukoemkie Tekhnol. Tekh. Nauki, 2016, no. 1, 38–44 (In Russian)

[7] Merzlyakov D.A., “Generation of regular expressions for automatic grading of open-ended tests”, Student Research Forum - 2013: Proc. V Int. Student Sci. Conf., 2013 (In Russian)

[8] Ziai R., Rudzewitz B., “CoMiC: Exploring text segmentation and similarity in the English entrance exams task”, CLEF2015 Working Notes, CEUR Workshop Proceedings, 1391, eds. Cappellato L., Ferro N., Jones G.J.F., Juan E.S., 2015

[9] Pribadi F.S., Permanasari A.E., Adji T.B., “Short answer scoring system using automatic reference answer generation and geometric average normalized-longest common subsequence (GAN-LCS)”, Educ. Inf. Technol., 23:6 (2018), 2855–2866 | DOI

[10] Xu L.-X., Wang N., Xu L., Li L.-Y., “Research of automated assessment of subjective tests based on domain ontology”, Multidisciplinary Social Networks Research, MISNC 2014, Proc. Int. Conf., Communications in Computer and Information Science, 473, Springer, Berlin–Heidelberg, 2014, 445–453 | DOI

[11] Kozhevnikov V.A., Sabinin O.Yu., “Automatic system for processing answers to open-ended questions in Russian”, Inf. Telekommun. Upr., 11:3 (2018), 57–72 (In Russian) | DOI