Algebraic decompositions of DP problems with linear dynamics

We consider Dynamic Programming (DP) problems in which the dynamics are linear, the cost is a function of the state, and the state-space is finite dimensional but defined over an arbitrary field. Starting from the natural decomposition of the state-space into a direct sum of invariant subspaces consistent with the rational canonical form, and assuming the cost functions exhibit an additive structure compatible with this decomposition, we extract from the original problem two distinct families of smaller DP problems associated with the invariant subspaces. Each family constitutes a decomposition of the original problem when the optimal policy and value function can be reconstructed from the optimal policies and value functions of the smaller problems. We derive necessary and sufficient conditions for these decompositions to exist, propose a readily verifiable sufficient condition for the first decomposition, and establish a hierarchy relating the two notions of decomposition. (C) 2015 Elsevier B.V. All rights reserved.