Geodesic
On the infuence of the cental trend on the nature of the density distribution of maximum entropy in machine learning
Vestnik Sankt-Peterburgskogo universiteta. Prikladnaâ matematika, informatika, processy upravleniâ, Tome 19 (2023) no. 2, pp. 176-184 Cet article a éte moissonné depuis la source Math-Net.Ru

Voir la notice de l'article

The principle of maximum entropy (ME) has a number of advantages that allow it to be used in machine learning. The density distribution of maximum entropy (WEO) requires solving the problem of calculus of variations on the a priori distribution, where the central tendency can be used as a parameter. In Lebesgue space, the central tendency is described by the generalized Gelder average. The paper shows the evolution of the density of the ME distribution depending on the given norm of the average. The minimum Kulback — Leibler divergence between the WEO and the a prior density is achieved at the harmonic mean, which is effective in reducing the dimensionality of the training sample. At the same time, this leads to a deterioration in the function of loss in the conditions of machine learning by precedents.
Keywords: maximum entropy principle, maximum entropy distribution, central trend, generalized average, machine learning. 
@article{VSPUI_2023_19_2_a3,
     author = {A. V. Kvasnov and A. A. Baranenko and E. Yu. Butyrsky and U. P. Zaranik},
     title = {On the infuence of the cental trend on the nature of the density distribution of maximum entropy in machine learning},
     journal = {Vestnik Sankt-Peterburgskogo universiteta. Prikladna\^a matematika, informatika, processy upravleni\^a},
     pages = {176--184},
     year = {2023},
     volume = {19},
     number = {2},
     language = {ru},
     url = {http://geodesic.mathdoc.fr/item/VSPUI_2023_19_2_a3/}
}
TY  - JOUR
AU  - A. V. Kvasnov
AU  - A. A. Baranenko
AU  - E. Yu. Butyrsky
AU  - U. P. Zaranik
TI  - On the infuence of the cental trend on the nature of the density distribution of maximum entropy in machine learning
JO  - Vestnik Sankt-Peterburgskogo universiteta. Prikladnaâ matematika, informatika, processy upravleniâ
PY  - 2023
SP  - 176
EP  - 184
VL  - 19
IS  - 2
UR  - http://geodesic.mathdoc.fr/item/VSPUI_2023_19_2_a3/
LA  - ru
ID  - VSPUI_2023_19_2_a3
ER  - 
%0 Journal Article
%A A. V. Kvasnov
%A A. A. Baranenko
%A E. Yu. Butyrsky
%A U. P. Zaranik
%T On the infuence of the cental trend on the nature of the density distribution of maximum entropy in machine learning
%J Vestnik Sankt-Peterburgskogo universiteta. Prikladnaâ matematika, informatika, processy upravleniâ
%D 2023
%P 176-184
%V 19
%N 2
%U http://geodesic.mathdoc.fr/item/VSPUI_2023_19_2_a3/
%G ru
%F VSPUI_2023_19_2_a3
A. V. Kvasnov; A. A. Baranenko; E. Yu. Butyrsky; U. P. Zaranik. On the infuence of the cental trend on the nature of the density distribution of maximum entropy in machine learning. Vestnik Sankt-Peterburgskogo universiteta. Prikladnaâ matematika, informatika, processy upravleniâ, Tome 19 (2023) no. 2, pp. 176-184. http://geodesic.mathdoc.fr/item/VSPUI_2023_19_2_a3/

[1] Jaynes E. T., “Prior probabilities”, IEEE Transactions on Systems Science and Cybernetics, 4 (1968), 227–251 | DOI

[2] Jaynes E. T., “Information theory and statistical mechanics”, Physical Review, 106:4 (1957), 620–630 | DOI | MR | Zbl

[3] Popkov Y. S., “Asymptotic efficiency of maximum entropy estimates”, Reports of the Russian Academia of Sciences. Mathematics. Computer Science. Control Processes, 493 (2020), 104–107 (In Russian) | DOI | Zbl

[4] Popkov Y. S., Dubnov Y. A., Popkov A. Y., “Entropy-randomized projection”, Automation and Telemechanics, 82:3 (2021), 149–168 (In Russian) | MR

[5] Popkov Yu. S., “Randomization and entropy in machine learning and data processing”, Reports of the Russian Academy of Sciences. Mathematics. Computer Science. Control Processes, 504:1 (2022), 3–27 (In Russian)

[6] Baggenstoss P. M., “Uniform Manifold Sampling (UMS): Sampling the maximum entropy PDF”, IEEE Transactions on Signal Processing, 65:1 (2017), 2455–2470 | DOI | MR | Zbl

[7] Mak S., Xie Y., “Maximum entropy low-rank matrix recovery”, IEEE Journal of Selected Topics in Signal Processing, 12:5 (2018), 886–901 | DOI

[8] Mazuelas S., Shen Y., Perez A., “Generalized maximum entropy for supervised classification”, IEEE Transactions on Information Theory, 68:4 (2022), 2530–2550 | DOI | MR | Zbl

[9] Richards M. A., Scheer J. A., Scheer J., Holm W. A., Principles of modern radar: basic principles, $1^{\rm st}$ ed., Institution of Engineering and Technology, Atlanta, 2010, 962 pp.

[10] Vorontsov K. V., “Combinatorial bounds for learning performance”, Reports of the Russian Academia of Sciences, 394:2 (2004), 175–178 (In Russian) | MR | Zbl

[11] Niven R. K., Andresen B., “Jaynes' maximum etntropy principle, riemannian metrics and generalised least action bound”, Statistical Mechanics, 2010, 283–317 | Zbl

[12] Kvasnov A. V., “Application of bayesian programming in recognition and classification of radar emission sources”, Radio Engineering, 84:3(5) (2020), 5–14 (In Russian)

[13] Kvasnov A. V., “Enhance of information completeness of classifier in remote sensing tasks of aerial point objects”, Journal of Radio Electronics, 11 (2021), 1–10 (In Russian)

[14] Kvasnov A. V., “Enhance of information completeness of classifier in remote sensing tasks of aerial point objects”, Sensors and Systems, 262:3 (2022), 9–14 (In Russian)

[15] Kvasnov A. V., “Accuracy of the coordinates for underlying surface based on photogrammetric measurements of the images with an oblique projection”, Sensor Systems, 36:3 (2022), 262–274 (In Russian)

[16] Marcial M. C. N., Santillan J. R., “A maximum entropy approach for mapping falcata plantations in sen-tinel-2 imagery”, IEEE region 10 Conference (TENCON), 2020, 596–601 | DOI

[17] Yin C., Xi J., Wang J., “The research of text classification technology based on improved maximum entropy model”, First International Conference on Computational Intelligence Theory, Systems and Applications (CCITSA), 2015, 142–145 | DOI

[18] Pan W., “Feature selection algorithm based on maximum information coefficient”, IEEE $5^{\rm th}$ Advanced Information Technology, Electronic and Automation Control Conference (IAEAC), 2021, 2600–2603 | DOI | MR