Debugging the Performance of Configurable Software Systems: A Human-Centered White-box Approach

Most of today’s software systems are configurable. The flexibility to customize these systems, however, comes with the cost of increased complexity. Understanding

how configuration options and their interactions affect performance, in terms of execution time, and often directly correlated energy consumption and operational costs,is challenging, due to the large configuration spaces of these systems. For this reason, developers often struggle to debug and maintain their systems when surprising performance behaviors occur. While there are numerous performance and program debugging techniques that developers could use to debug their systems, there is limited empirical evidence of how useful the techniques are to help developers debug the performance of configurable software systems; the techniques typically solve a specific technical challenge that is usually evaluated in terms of accuracy, not usability. Hence, we could only, at best, speculate which techniques might support developers’ needs to debug unexpected performance behaviors in configurable software systems. In this dissertation, we take a human-centered approach to identify solutionsto support developers’ actual needs in the process of debugging the performance of configurable software systems. Specifically, we identify white-box analyses and techniques that can be tailored to provide relevant performance-behavior information for developers to understand how configuration options and their interactions cause performance issues. In our human-centered research design, we first conduct an exploratory user study to identify the information needs that developers have when debugging the performance of configurable software systems. Afterwards, we identify the program analyses and techniques that can be tailored to support those needs. In this process, we note the limitations of existing performance-influence modeling techniques, and present and evaluate a white-box approach that overcomes those limitations. Afterwards, we describe how we design and implement information providers, tailoring the white-box analyses that we identified, to support developers’ needs; namely, global and local performance-influence models, CPU profiling, and program slicing. Finally, we conduct two users studies to validate and confirm that our designed information providers support the needs that developers have and speed up the process of debugging the performance of complex configurable software systems.

The contributions in this dissertation help reduce the energy consumption and operational costs of running configurable software systems by providing developers with tool support to help them debug and maintain their systems.