Methods for Live Testing of Cloud Services

Service providers use cloud to reduce the cost of their services and speed up their time to market. There is a huge gap between the environment where the services are developed and first tested, and the environment where they operate. Live testing is defined as testing a service in its production environment without causing any intolerable disruption to its usage. It is considered as one of the solutions to overcome the impact of the difference between the development environment and the production environment on the reliability of the test results. Test interferences are a major challenge for live testing as they may lead to the system mishandling the production traffic. Existing solutions to alleviate the risk of interferences have limited applicability. In this paper we propose a set of test methods, applicable in different situations, for live testing to improve the reliability of test results without causing any intolerable disturbance to the cloud services. To reduce the complexity of these test methods we define the concept of boundary environments and a set of coverage criteria to aim at during live testing.