The maximum entropy principle in decision theory

A. N. Prokaev

Vestnik Sankt-Peterburgskogo universiteta. Prikladnaâ matematika, informatika, processy upravleniâ, Tome 20 (2024) no. 2, pp. 154-169

Résumé

Traditionally, the principle of maximum entropy is used to find unknown distribution of random variables. In decision theory, this principle is used primarily in situations of uncertainty regarding the probability distribution of hypotheses about the “state of the environment,” where the environment is understood as a set of parameters that influence the result of the decision made. This paper considers the use of the principle of maximum entropy for a different purpose, namely for the purpose of optimal distribution of resources of various types. A proof of theorems is given that make it possible to create algorithms for solving various problems of resource allocation based on the principle of maximum entropy, as well as examples of solving demonstrative problems.

Détail

Keywords: information theory, decision theory, search theory, expected utility theory, utility function, maximum entropy principle.

Bibliographie

[1] Jaynes E. T., “Information theory and statistical mechanics”, Physical Review. Series II, 1957, no. 4, 620–630 | MR | Zbl

[2] Jaynes E. T., “Information theory and statistical mechanics. II”, Physical Review. Series II, 1957, no. 2, 171–190 | MR | Zbl

[3] Shannon C. E., “A mathematical theory of communication”, Bell System Techn. Journal, 27:4 (1948), 623–656 | DOI | MR | Zbl

[4] Jaynes E. T., Where do we stand on maximum entropy?, Maximum Entropy Formalism, MIT Press, Cambridge, 1978, 15–118 | MR

[5] Jaynes E. T., “The relation of Bayesian and maximum entropy methods”, Maximum entropy and Bayesian methods in science and engineering, v. 1, Kluwer Academic Publ, Dordrecht, Netherlands, 1988, 25–29 | DOI | MR

[6] Dias P. M., Shimony A., “A critique of Jaynes' maximum entropy principle”, Advances in Applied Mathematics, 2 (1981), 172–211 | DOI | MR | Zbl

[7] Mohammad-Djafari A., Maximum entropy and Bayesian inference: where do we stand and where do we go?, Bayesian inference and maximum entropy methods in science and engineering, 872:11 (2006), 3–14 | DOI | MR

[8] Grendar M. jr, Grendar M., “Maximum entropy: Clearing up mysteries”, Entropy, 3:2 (2001), 58–63 | DOI | Zbl

[9] Neapolitan R. E., Jiang X., “A note of caution on maximizing entropy”, Entropy, 16:7 (2014), 4004–4014 | DOI

[10] De Martino A., De Martino D., “An introduction to the maximum entropy approach and its application to inference problems in biology”, Heliyon, 4:4 (2018), 314–344 | DOI | MR

[11] Neumann J. von, Morgenstern O., Theory of games and economic behavior, Princeton University Press, Princeton, 1953, 707 pp. | MR | Zbl

[12] Ackoff R. L., Sasieni M. W., Fundamentals of operations research, John Wiley Sons, Inc. Publ., New York, 1967, 455 pp. | MR | MR

[13] Trukhaev R. I., Models of decision making in conditions of uncertainty, Nauka Publ, M., 1981, 257 pp. (In Russian) | MR

[14] Koroliov O. L., Kussiy M. Y., Sigal A. V., Application of entropy in modeling decision-making processes in economics, ODSHAK Publ, Simferopol, 2013, 148 pp. (In Russian)

[15] Mikhno V. N., “Maximum entropy model for forming an investment portfolio”, Vestnik of Tver State University. Applied Mathematics, 2017, no. 1, 45–55 (In Russian)

[16] Wilson A. G., Entropy in urban and regional modelling, Pion Ltd, London, 1978, 248 pp. | MR

[17] Geraskin M. I., Mathematical economics, Textbook, Samara State Aerospace University Publ, Samara, 2011, 176 pp. (In Russian)

[18] Markowitz H. M., “Portfolio selection”, The Journal of Finance, 7:1 (1952), 77–91 https://edisciplinas.usp.br/mod/resource/view.php?id=1413077

[19] Prokaev A. N., “The maximum entropy principle in search theory”, Vestnik of Saint Petersburg University. Applied Mathematics. Computer Science. Control Processes, 19:1 (2023), 27–42 (In Russian) | DOI | MR

[20] Prokaev A. N., “Theoretical fundamentals for search problems solving using maximum entropy method”, Vestnik of Saint Petersburg University. Applied Mathematics. Computer Science. Control Processes, 19:3 (2023), 348–368 (In Russian) | DOI | MR

[21] Lisitsa M. I., Assessing the profitability and risk of financial investments, Textbook, University of Interparliamentary Assembly of EurAsES Publ, St. Petersburg, 2018, 226 pp. (In Russian)

[22] Dobrina M. V., “Utility functions and their application in modeling portfolio decisions”, Modern economy: problems and solutions, 2017, no. 8(92), 64–76 (In Russian) | DOI

[23] Bakulin A. G., Index investing, 2021, 82 pp. (accessed: January 19, 2024)

[24] Jaynes E. T., “Entropy and search theory”, Maximum-entropy and Bayesian methods in inverse problems, Fundamental theories of physics, 14, Springer, Dordrecht, Netherlands, 1985, 1–18 | MR