LSCOM Lexicon Definitions and Annotations (Version 1.0)

The DTO sponsored LSCOM workshop is developing an expanded multimedia concept lexicon on the order of 1000. Concepts related to events, objects, locations, people, and programs have been selected following a multi-step process involving input solicitation, expert critiquing, comparison with related ontologies, and performance evaluation. Participants of the process include representatives from intelligence community users, ontology specialists, and multimedia analytics researchers. In addition, each concept has been qualitatively assessed according to some criteria, such as utility (usefulness), observability (by humans), and feasibility (by automatic detection). An annotation process was completed in late 2005 by student annotators at Columbia University and CMU, over the entire development set of TRECVID 2005 videos. Human subjects judge the presence or absence of each concept in the key frame of each subshot, resulting in a total of 61901 labels for each concept. Out of the 834 initial selected concepts, the first version of LSCOM annotations consists of 449 unique concepts (39 LSCOM-Lite concepts included) over the entire TRECVID 2005 development set (61901 subshots).