Geodesic
Customization of J.~Bather UCB strategy for a Gaussian multi-armed bandit
Matematičeskaâ teoriâ igr i eë priloženiâ, Tome 14 (2022) no. 2, pp. 3-30

Voir la notice de l'article provenant de la source Math-Net.Ru

We consider the customization of the UCB strategy, which was first proposed by J. Bather for Bernoulli two-armed bandit, to the case of a Gaussian multi-armed bandit describing batch data processing. This optimal control problem has classical interpretation as a game with nature, in which the payment function of the player is the expected loss of total income caused by incomplete information. The goal is stated in minimax setting. For the considered game with nature, we present an invariant description of the control with a horizon equal to one, which allows to perform computations in two ways: using Monte-Carlo simulations and analytically by dynamic programming technique. For various configurations of the considered game with nature, we have found saddle points, which characterize the optimal control and the worst-case distribution of the parameters of the multi-armed bandit.
Keywords: multi-armed bandit problem, Gaussian multi-armed bandit, minimax approach, UCB rule, dynamic programming.
Mots-clés : invariant description, Monte-Carlo simulations 
@article{MGTA_2022_14_2_a0,
     author = {Sergey V. Garbar and Alexander V. Kolnogorov},
     title = {Customization of {J.~Bather} {UCB} strategy for a {Gaussian} multi-armed bandit},
     journal = {Matemati\v{c}eska\^a teori\^a igr i e\"e prilo\v{z}eni\^a},
     pages = {3--30},
     publisher = {mathdoc},
     volume = {14},
     number = {2},
     year = {2022},
     language = {ru},
     url = {http://geodesic.mathdoc.fr/item/MGTA_2022_14_2_a0/}
}
TY  - JOUR
AU  - Sergey V. Garbar
AU  - Alexander V. Kolnogorov
TI  - Customization of J.~Bather UCB strategy for a Gaussian multi-armed bandit
JO  - Matematičeskaâ teoriâ igr i eë priloženiâ
PY  - 2022
SP  - 3
EP  - 30
VL  - 14
IS  - 2
PB  - mathdoc
UR  - http://geodesic.mathdoc.fr/item/MGTA_2022_14_2_a0/
LA  - ru
ID  - MGTA_2022_14_2_a0
ER  - 
%0 Journal Article
%A Sergey V. Garbar
%A Alexander V. Kolnogorov
%T Customization of J.~Bather UCB strategy for a Gaussian multi-armed bandit
%J Matematičeskaâ teoriâ igr i eë priloženiâ
%D 2022
%P 3-30
%V 14
%N 2
%I mathdoc
%U http://geodesic.mathdoc.fr/item/MGTA_2022_14_2_a0/
%G ru
%F MGTA_2022_14_2_a0
Sergey V. Garbar; Alexander V. Kolnogorov. Customization of J.~Bather UCB strategy for a Gaussian multi-armed bandit. Matematičeskaâ teoriâ igr i eë priloženiâ, Tome 14 (2022) no. 2, pp. 3-30. http://geodesic.mathdoc.fr/item/MGTA_2022_14_2_a0/