This paper addresses the question of existence of stationary Markov perfect equilibria in a class of dynamic games that includes many known bargaining models and models of coalition formation. General sufficient conditions for existence of equilibria are currently lacking in a number of interesting environments, e.g., models with non-convexities, consumption lower bounds, or an evolving state variable. The main result establishes existence of equilibrium under compactness and continuity conditions, without the structure of convexity or strict comprehensiveness used in the extant literature. The proof requires a precise selection of voting equilibria following different proposals using a generalization of Fatou's lemma. (C) 2016 Elsevier Inc. All rights reserved.