A Two-Level Hierarchical Markov Decision Model with Considering Interaction between Levels

Decision in reality often have the characteristic of hierarchy because of the hierarchy of an organization’s structure. In this paper, we propose a two-level hierarchic Markov decision model that considers the interactions of agents in different levels and different time scales of levels. A backward induction algo-rithm is given for the model to solve the optimal policy of finite stage hierarchic decision problem. The proposed model and its algorithm are illustrated with an example about two-level hierar-chical decision problem of infrastructure maintenance. The optimal policy of the example is solved and the impacts of interactions between levels on decision making are analyzed.