Program Comprehension as Fact Finding

Little is known about how developers think about design during code modification tasks or how experienced developers’ design knowledge helps them work more effectively. We performed a lab study in which thirteen developers worked for 3 hours understanding the design of a 54 KLOC open source application. Participants had from 0 to 10.5 years of industry experience and were grouped into three “experts” and ten “novices.” We observed that participants spent their time seeking, learning, critiquing, explaining, proposing, and implementing facts about the code such as “getFoldLevel has effects”. These facts served numerous roles, such as suggesting changes, constraining changes, and predicting the amount of additional investigation necessary to make a change. Differences between experts and novices included that the experts explained the root cause of the design problem and made changes to address it, while novice changes addressed only the symptoms. Experts did not read more methods but also did not visit some methods novices wasted time understanding. Experts talked about code in terms of abstractions such as “caching” while novices more often described code statement by statement. Experts were able to implement a change faster than novices. Experts perceived problems novices did not and were able to explain facts novices could not. These findings have interesting implications for future tools.