Shapley weights of test items

M. M. Lutsenko ; N. V. Shadrinceva

Vestnik Sankt-Peterburgskogo universiteta. Prikladnaâ matematika, informatika, processy upravleniâ, Tome 13 (2017) no. 3, pp. 300-312

Résumé

One of the problems of the theory of testing is the problem of determining the weights of the test item. For most tests, the weights of assignments are considered to be equal, but as the complexity of test design and increasing the number of test items, the need for determining the complexity of the test increases. This is especially important when comparing students to solve different tests or in cases when the time for the solution to all the test items is not enough. In addition, the resolution of the test increases significantly if different test items are attributed to different weights. In the work for a course that has a hierarchical structure and is known for the duration of the development of its sections, we built a “sum of attainments” cooperative game in which players are the parts of the course, and the value of the characteristic function on a coalition is the time necessary for a student to study the coalition of the parts. We have found simple formulas for the calculation of the Shapley value as constructed in the game and so expanded the class of cooperative games for which the Shapley value is calculated analytically. The basis of the proof lies in the decomposition of the characteristic function in a sum of the dual unanimity games and using the properties of a cone in the set of partially ordered elements (parts of course). The component of the Shapley value is the average time of learning the part of the course and it may be used as the weight of the item for a final test. The theory of duality (complementarity), developed for cooperative games permits us to link “the sum of attainments game”, “the airport game” and “the peer group game”. Examples of the calculation of weights for test items has been prepared by the use of a particular model term course. Refs 16. Figs 2.

Détail

Keywords: TU game, Shapley value, analytical game solution, teaching test, weight of item, partial ordered set of players.

Bibliographie

[1] Littlechild S. C., Owen G., “A simple expression for the Shapley value in a special case”, Management Sci., 20 (1973), 370–372 | DOI | Zbl

[2] Crocker L., Algina J., Introduction to Classical and Modern Test Theory, Cengage Learning, Boston, 2006, 527 pp.

[3] Lutsenko M. M., Shadrintseva N. V., “Cooperative and frequency approaches to assigning weights of the test”, Proceeding of Saint Petersburg University of Railways, 2014, no. 3(40), 150–156 (In Russian)

[4] Lutsenko M. M., Shadrintseva N. V., “A cooperative approach to assigning the weights of the tasks in the pedagogical test”, Mathematical game theory and its applications, 6:4 (2014), 37–67 (In Russian)

[5] Neiman U. M., Khlebnikov V. A., Introduction to the theory of modelling and parameterization of pedagogical tests, Vysshaja shkola Publ., M., 2000, 168 pp. (In Russian)

[6] Win J. Van der Linden, R. K. Hambleton (eds.), Handbook of Modern Item Response Theory, Springer-Verlag, New York, 1997, 510 pp. | MR

[7] Lutsenko M. M., “Game-theoretic approach to the assessment of testing accuracy”, Mathematical game theory and its applications, 1:4 (2009), 63–77 (In Russian)

[8] Lutsenko M. M., Shadrintseva N. V., “The Shapley and Banzaf values for different courses of study”, The Sixth Intern. conference “Game Theory and Management”, Abstracts, St. Petersburg State University, St. Petersburg, 2012, 165–169

[9] Lutsenko M. M., Shadrintseva N. V., “Shapley value in testing”, Proceedings of the second Intern. conference “Game theory and management application”, “Game theory, operations research and applications”, Institute of public enterprise, Hyderabad, India, 2012, 47–49

[10] Mashler M., Solan E., Zamir S., Game theory, Cambridge University Press, Cambridge, USA, 2013, 1005 pp. | MR

[11] Petrosiyan L. A., Zenkevich N. A., Shevkoplias E. V., Game theory, Textbook, BHV-Petersburg Publ., Saint Petersburg, 2012, 432 pp. (In Russian)

[12] Shapley L. S., “A value for $N$-person games”, Contributions to the Theory of Games II, eds. H. W. Kuhn, A. W. Tucker, Princeton University Press, Princeton, 1953, 307–317 | MR

[13] Brânzei R., Fragnelli V., Tijs S., “Tree-connected peer group situations and peer group games”, Mathematical Methods of Operations Research, 55 (2002), 93–106 | DOI | MR

[14] Gilles R. P., Owen G., Brink R. van den, “Games with permission structures: The Cunjuctive Approach”, Intern. Journal of Game Theory, 20:3 (1992), 277–294 | DOI | MR

[15] Rasmusen E., Games and information, An Introduction to Game theory, Blackwell, Oxford, UK–Cambridge, USA, 1989, 560 pp. | MR | Zbl

[16] Ni D., Wang Y., “Sharing a polluted river”, Games and Economic Behavior, 60 (2007), 176–186 | DOI | MR | Zbl