Conducting Browser-Based User Studies in Software Engineering

User studies of software engineers are an invaluable part of the evaluation of new tools in academic software engineering (SE). Unfortunately, significant barriers to the efficient execution of these user studies still exist. Researchers in software engineering and related fields continue to struggle with recruiting, orchestration, experimental design, and IRB approval (Buse 2011). In this paper we address these challenges by comparing them to problem domains already identified and addressed in the behavioral sciences. We present (1) a survey of the current frameworks for administering online experiments in behavioral science and (2) a pair of software engineering-specific tools extending jsPsych, a front-end framework from behavioral science.