Analyzing, Aligning, Assessing: A portable framework for corpus-based writing pedagogy at scale

The teaching of writing in disciplinary content areas reinforces learning objectives and  encourages students to engage in active problem-solving applicable to future professional or academic settings. Within Writing Across the Curriculum (WAC) and Writing in the Disciplines (WID) scholarship,  subject areas like civil or mechanical engineering have received thorough attention. However, the  presently expanding field of statistics and data science has yet to receive explicit attention in WAC/WID,  technical and professional communication, and rhetorical genre studies more broadly. Statistical analyses  appear in a variety of interdisciplinary and professional contexts, but relatively recent calls for curricular  shifts to promote data science initiatives have widened the responsibilities of statistics departments and  instructors. Recent scholarship has nevertheless outlined the dearth of writing-intensive data science  classes across various programs in higher education, highlighting a variety of factors, including the  additional labor burden placed on instructors of technical courses that are often composed of 70 to 150  students. Responding to these conversations, the research presented in my dissertation is the product of  an ongoing collaboration between the Department of English and the Department of Statistics and Data  Science at Carnegie Mellon University.   

My dissertation presents the first large scale study of its kind to describe writing in the discipline  of statistics and to use those descriptive results to pilot a technology-enhanced learning intervention using  DocuScope Write & Audit. First, I collected and analyzed a corpus of over 1,200 texts, consisting of  published statistics research and authentic student writing samples from six courses offered by CMU’s  Department of Statistics & Data Science. Using corpus linguistics methods, I explicitly name the rhetorical  and linguistic features of client-facing and expert-facing statistics reports. Using the more granular results  from the corpus analysis, our interdisciplinary team designed learning materials for voluntary revision  workshops offered to students of 36-200: Reasoning with Data. These workshops asked participants (n =  30) to revise their papers using DocuScope: Write & Audit, a student-facing tool that can visualize task specific features in real-time as students write and revise. This exploratory study showed promising  results in the comparison of pre- and post-workshop survey responses: significant positive increases in  subscales of self-efficacy and beliefs about the importance of writing content after using the tool. To  better contextualize the survey results, I pair them with an analysis of think-aloud interviews. This   dissertation project contributes to corpus linguistics studies, WAC/WID research, genre studies, and  areas of formative writing assessment. Ultimately, though, I sketch a portable framework for analyzing  and teaching disciplinary writing at scale, one which I plan to refine and reapply in future collaborations.