Context/Background: The Mission Design and Navigation Software (MDN) Group at the Jet Propulsion Laboratory (JPL) develops and continuously maintains software systems critical for NASA deep space missions. Given limited budgets, staffing resources, and a time critical need for repair or enhancement, there is an ever-present temptation to sacrifice quality for higher productivity or slip release target to ensure better quality. We have learned that poor management of this increases risk of mission failure. As a result, our process must be both highly productive and maintain high quality (e.g. reliability, maintainability, usability). Inspired by the "quality is free" paradigm, we have instituted a set of "Rapid Release" maintenance process policies and measures aimed to continually manage productivity and quality. Six Rapid Release polices were established from well-known engineering principles and best practices to address specific issues of concern encountered in the development phase. However, due to the critically of our systems, we must have objective assurance that our developers are following the six policies and that they are demonstratively effective in addressing the areas of concern. Goal: Investigate if Rapid Release as currently implemented is effective in achieving effects and impacts as expected from principles and best practice beliefs. Additionally, determine practical methods to assure compliance and performance to Rapid Release policies and determine if any adjustments to policy or practice is needed. Method: We have over 15 years of reliable and accurate quality and productivity process data for Monte, a critical system currently in continual operation and maintenance. Time series cross-correlation analyses on this data is used to compare process productivity and quality characteristics pre- and post-implementation of Rapid Release. Results: We find strong evidence, that for Monte: (1) there is continual risk due to productivity and quality tradeoffs, (2) the majority of the Rapid Release policies are being complied with, and (3) the policies have been effective in managing this risk. Conclusions: High productivity and high quality in maintenance of our critical systems requires more than implementing policy based on belief. The process must be monitored to assure that policies are adhered to and are effective in producing the results desired.
