Projects
Markov Decision Process and Risk Project
May 2019  April 2020
FAPESP grant 2018/112369
Abstract:
Markov Decision Processes (MDPs) are widely used to solve sequential decisionmaking problems. The most used criterion in this type of problem to find a solution is to minimize the expected total cost. However, this approach does not take into account cost variability (that is, fluctuations around the average), which can significantly affect the performance of the solution. MDPs that deal with these types of problems are called risksensitive MDPs. Among the risksensitive MDPs we have: (i) MDPs that use as an optimization criterion the expected exponential utility; (ii) MDPs whose objective is to maximize the probability that the accumulated cost is not greater than a given limit provided by the user, called MDP with limited budget; (iii) MDPs whose criteria include the CVaR metric, a robust way of measuring risk commonly used in the financial area, called CVaR MDPs; and (iv) MDPs whose criterion uses the average of the total cost in conjunction with the CVaR criterion, called meanCVAR MDPs. This research project intends to work with MDPs with limited budget, CVar MDPs and meanCVaR MDPs. The main objective is to propose exact and approximate algorithms based on dynamic programming to solve these risksensitive MDPs.
Coordinator: Karina Valdivia Delgado.
Project page
PROPAT Project (part of Eclipse Project at IMEUSP): An Eclipse plugin for teaching programming.
Funded by IBM from 2003 to 2006. Coordinator: Leliane Nunes de
Barros. As student of Leliane Nunes de Barros.
Probabilistic Logic Project : Fundamentals and Applications. Funded
by FAPESP from 2008 to 2013, Process: 2008/039955. .Coordinator:
Marcelo Finger. As student of Leliane Nunes de Barros.
