Meta-Information to Support Sensemaking by Developers

 Software development requires developers to juggle and balance many information seeking and understanding tasks. From understanding how a bug was introduced,  to choosing what application programming interface (API) method to use to resolve  the bug, to determining how to properly integrate this change, even the smallest  implementation tasks can lead to many questions. These questions may range from  hard-to-answer questions about the rationale behind the original code to common  questions such as how to use an API. Software development, in contrast to other  sensemaking domains, has the unique property that many information artifacts are  created at different points during the development process (e.g., output data). Once  this challenging sensemaking is done, this rich thought history is often lost given  the high cost of externalizing these details, despite potentially being useful to future  developers.  

In this thesis, I explore different systems and methods for authoring and using  this rich thought history as meta-information about code. Specifically, I have developed systems for annotating to support developers’ natural sensemaking when un derstanding information-dense sources such as software documentation and source  code. I then demonstrated how this meta-information can be captured and harnessed for new tasks, including for assessing the trustworthiness of documentation,  for creating long-form documentation as stories about the code, for capturing design  rationale and provenance data of code, and for supporting developer implementation tasks such as debugging.  

This thesis begins by exploring methods for externalizing developers’ thoughts  in a form that is lightweight yet contextualized. We explore annotating as a method  for simultaneously allowing developers to offload their mental processes, while presenting that information in-context for later developers to utilize. We developed two  prototype annotation systems, Adamite and Catseye, which showed the promise of  annotating for assisting developers both in overcoming issues with using developer  documentation and debugging code. The dynamic nature of code and its connection to annotated materials introduced unique design challenges in which information can quickly become outdated and disconnected, leading both to changes in the  Catseye user interface and to the insight that leveraging the (lack of) connectivity  between annotation and documentation can support other documentation-related  tasks, which inspired Sodalite. The final two systems, Meta-Manager and MMAI,  explore capturing other forms of already-authored meta-information, such as edit  traces and log data, for question-answering support, with MMAI utilizing large language models to make that querying possible in natural language.  

The series of work introduced in this thesis points to the need to treat users’  thoughts and intents as first-class entities and that meta-information is a way of  preserving that information. I show that developers’ thought histories can be represented in the form of code-related meta-information and, through proper tooling,  can be used by later developers to accelerate their sensemaking of code.