An Empirical Comparison of Field Defect Modeling Methods

In this study, we report empirical results from forecasting field defect rates and predicting the number of field defects for a large commercial software system. We find that we are not able to accurately forecast field defect rates using a combined time-based and metrics-based approach, as judged by the Theil forecasting statistic. We suggest possible conditions that may have contributed to the poor results. Next, we use metrics-based approaches to predict the number of field defects within the six months after deployment. We find that the simple ratios method produce more accurate predictions than more complex metrics-based methods. Our results are steps toward quantitatively managing the risks associated with software field defects.