The structure of optimal trajectories of a discrete deterministic scheme with discounting

V. D. Matveenko

Diskretnaya Matematika, Tome 10 (1998) no. 3, pp. 100-114

Résumé

We investigate the family of problems on finding $$ \max_{i_1,\ldots,i_{T-1}}\sum_{k=0}^{T-1}\beta^ku(i_k,i_{k+1}) $$ for $i_0=j_0$, $i_T=j_T$, where $\beta$ is the discount factor ($\beta>0$, $\beta\ne1$); $i_k$, $k=0,1,\ldots,T$, are elements of a given finite set; $u$ is a function taking values in the space $\mathbb R\cup\{-\infty\}$. The number of steps $T$ and the boundary states $j_0,j_T$ are considered as parameters. We give a description of the structure of the optimal trajectories for sufficiently large number of steps $T$. A theorem on a representation of the value function is proved. A sufficient condition is given under which a given contour is not included into any optimal trajectory regardless of the value of $\beta$.