As more and more organisations move from on-premise to on-demand software solutions in order to minimise overhead costs, the Enterprise Resource Planning (ERP) providers begin to offer their well-established best business processes as cloud based IT solutions. As a result, ERP software previously reserved for large organisations mainly due to heavy costs associated with the initial purchase of the software, its licensing costs and subsequent configuration and maintenance, begins to open up to Small and Medium Enterprises (SMEs). With more and more Software as a Service (SaaS) ERP solutions on the market, SMEs are beginning to consider and implement these solutions as opposed to in-house developed applications, in a bid to offer better customer relationship, gain new markets and increase profits, amongst other benefits. As this study reveals, while deployment model is a key advantage in enabling organisations to minimise costs, certain critical factors as well as some barriers need to be addressed to enable organisations to successfully implement these systems. This paper aims to identify and evaluate common problems experienced in the implementation as well as post-implementation stage of Software as Service ERP systems by Small and Medium Enterprises. Qualitative data was collected from five European and US companies, which implemented SaaS ERP, by means of a set of semi-formal interviews and online questionnaire.