Approximate dynamic programming for network recovery problems with stochastic demand

Immediately after a disruption, in order to minimize the negative impact inflicted on the society, its imperative to re-establish the interdicted critical services enabled by the infrastructure networks. In this paper, we study the stochastic network recovery problem that tackles the planning of restoration activities (considering limited resources) on interdicted infrastructure network links so that the pre-disruption critical service flows can be re-established as quickly as possible. As an illustrative case study, we consider a disaster scenario on a road infrastructure network that obstructs the flow of relief-aid commodities and search-and-rescue teams between critical service providing facilities and locations in need of these critical services. As in the case of many realistic applications, we consider the amount of demand for critical services as stochastic. First, we present a Markov decision process (MDP) formulation for the stochastic road network recovery problem (SRNRP), then we propose an approximate dynamic programming (ADP) approach to heuristically solve SRNRP. We develop basis functions to capture the important complex network interactions that can be used to approximate cost-to-go values for the MDP states. We conduct computational experiments on a set of small-scale randomly generated instances and demonstrate that the ADP approach provides near-optimal results regardless of the demand distribution and network topology. In order to develop a practical approach suitable for solving real world sized instances, we propose a framework where we first develop an ADP model and derive a policy on a spatially aggregated network of large scale instance. Next, we show the performance of this policy through computational testing on the large scale disaggregated network. Moreover, we provide managerial insights by assessing the importance of each basis function in the ADP model contributing to the recovery policies. We test this approach on a case study based on the Boston road infrastructure network. We observe that, as the urgency of re-establishing services increases or the resources become more scarce, the information gained from the network characteristics and short-term decisions should be the main driving factors to derive recovery policies. The results of all experiments strongly evidence the significance of utilizing the inherent network interactions and attributes to generate basis function sets for ADP models that yield high-quality recovery policies.