Some Lessons Learned in Conducting Software Engineering Surveys in China

Component-Based Software Engineering (CBSE) with Open Source Software and Commercial-Off-the-Shelf (COTS) components, Open Source Software (OSS) based development, and Software Outsourcing (SO) are becoming increasingly important for the Chinese software industry. It is therefore necessary to establish pragmatic and possibly nation-specific guidelines for Chinese software companies regarding the use of CBSE, OSS, and SO. Such guidelines should be based on insights from actual practice, which are in our case, obtained through surveys. A European state-of-the-practice survey on COTS- and OSS-oriented CBSE was conducted in Germany, Italy, and Norway in 2004-2005. We repeated similar surveys in China, with an extended survey on OSS and SO. We encountered many difficulties in conducting the surveys, but in most cases managed to find working solutions. We report on the lessons learned while conducting these surveys. In particular, we address issues relating to sampling, contacting respondents, data collection, and data validation. The main lessons are: 1) it was necessary to cooperate with a third-party organization with close relations to Chinese software companies; 2) it was necessary to assign researchers to this third-party organization to facilitate data collection and to control the quality of the data collected; and 3) an email survey, after an initial telephone call to establish contact, was the best method for getting questionnaires completed by Chinese respondents.