Discrete-time Markovian jump linear systems

The paper considers a problem of optimal control of a linear system with the parameters dependent on the states of a Markov chain. The cost criterion is quadratic in the controls and states of the system. The criterion parameters also depend on the states of the Markov chain. Two models of observation of the Markov chain are adopted - delay for one step and no delay. It is shown that under appropriate mean square detectability and stabilizability conditions the infinite horizon optimal control problem for the general case of Markovian jump linear quadratic systems has a unique mean square stabilizing solution. Necessary and sufficient conditions are gives to determine if a system is mean square stabilizable.