MEDICAL TREATMENTS USING PARTIALLY OBSERVABLE MARKOV DECISION PROCESSES

Partially observable Markov decision processes (POMDPs) have recently been suggested as a suitable model to formalizing the planning of clinical patient management. The condition of a patient is a function of many unobservable physical characteristics. A POMDP is a generalization of a Markov decision process that allows for the incomplete state information. In this paper, we discuss a specialization of POMDPs that is tailored to a frequently reoccurring type of management problem, where our goal is to determine the best timing for testing and treatment decisions for patients with ischemic heart disease (IHD). We analyze the model for structural results and find that under certain assumptions conditions which guarantee an optimal policy. We develop a solution procedure utilizing these structural properties. It is described how this solution method can be used for patients with ischemic heart disease (IHD).