A predictive Markov decision process for optimizing inspection and maintenance strategies of partially observable multi-state systems

Optimizing both inspection and maintenance strategies for multi-state systems is challenging, especially when the inspected conditions contain uncertainties. One classic approach for addressing this issue is the Partially Observable Markov Decision Process (POMDP). However, the POMDP often considers the system is periodically inspected, resulting in a waste of inspection resources (cost and manpower) in the early stage of the system. To predictively optimize the inspection strategies, we formulate a new model-Predictive Markov Decision Process (PMDP). It extends the POMDP by embedding the Forward algorithm for inspection timing prediction and the Baum-Welch algorithm for model parameters estimation. Therefore, it could harvest the inspection information for predicting the successive inspection timing in an online updating scheme based on the new observation. In this manner, maintenance actions can take place at the predicted inspection timing that reduces unnecessary inspections. The PMDP manifests the power of predictive maintenance. As illustrated by the case study, the PMDP outperforms the POMDP under routine inspection by saving 26.3% of the cost on average.