Detecting the Noteworthiness of Utterances in Human Meetings

Our goal is to make note-taking easier in  meetings by automatically detecting 

noteworthy utterances in verbal exchanges and suggesting them to meeting  participants for inclusion in their notes.  To show feasibility of such a process we  conducted a Wizard of Oz study where  the Wizard picked automatically transcribed utterances that he judged as noteworthy, and suggested their contents to the participants as notes. Over 9 meetings, participants accepted 35% of these suggestions. Further, 41.5% of their notes at the end of the meeting contained Wizard-suggested text. Next, in order to perform noteworthiness detection automatically, we annotated a set of 6 meetings with a 3-level noteworthiness annotation scheme, which is a break from the binary “in summary”/ “not in summary” labeling typically used in speech summarization. We report Kappa of 0.44 for the 3-way classification, and 0.58 when two of the 3 labels are merged into one. Finally, we trained an SVM classifier on this annotated data; this classifier’s performance lies between that of trivial baselines and  inter-annotator agreement.