HOW EFFECTIVE ARE SOFTWARE ENGINEERING METHODS

For 25 years, software engineers have sought methods which they hope can provide a technological fix for the ''software crisis.'' Proponents of specific methods claim that their use leads to significantly improved quality and productivity. Such claims are rarely, if ever, backed up by hard evidence. We show that often, where real empirical evidence does exist, the results are counter to the views of the so-called experts. We examine the impact on the software industry of continuing to place our trust in unproven, and often revolutionary, methods. The very poor state of the art of empirical assessment in software engineering can be explained in part by inappropriate or inadequate use of measurement. Numerous empirical studies are flawed because of their poor experimental design and lack of adherence to proper measurement principles.