Performance Effects of Measurement and Analysis: Perspectives from CMMI High Maturity Organizations and Appraisers

This report describes results from two recent surveys conducted by the Software Engineering Institute (SEI) to collect information about the measurement and analysis activities of software systems development organizations. Representatives of organizations appraised at maturity levels 4 and 5 completed the survey in 2008. Using a variant of the same questionnaire in 2009, certified high maturity lead appraisers described the organizations that they had most recently coached or appraised for the achievement of similar high maturity levels. The replies to both surveys were generally consistent even though the two groups are often thought to be quite different. The results of the surveys suggest that the organizations understood and used CMMI-based process performance modeling and related aspects of measurement and analysis a great deal. Both the organizational respondents in 2008 and the appraisers in 2009 reported that process performance models were useful for the organizations. The respondents in both surveys also judged process performance modeling to be more valuable in organizations that understood and used measurement and analysis activities more frequently and provided organizational resources and management support. In addition, results from the 2009 survey of lead appraisers indicate that organizations that achieved their appraised high maturity level goals also found measurement and analysis activities more useful than those organizations that did not achieve their targets.