file.pdf (871.06 kB)
Download file

Gestural Communication over Video Stream: Supporting Multimodal Interaction for Remote Collaborative Physical Tasks

Download (871.06 kB)
journal contribution
posted on 01.01.2003, 00:00 authored by Jiazhi Ou, Susan R. Fussell, Xilin Chen, Leslie D. Setlock, Jie Yang
We present a system integrating gesture and live video to support collaboration on physical tasks. The architecture combines network IP cameras, desktop PCs, and tablet PCs to allow a remote helper to draw on a video feed of a workspace as he/she provides task instructions. A gesture recognition component enables the system both to normalize freehand drawings to facilitate communication with remote partners and to use pen-based input as a camera control device. Results of a preliminary user study suggest that our gesture over video communication system enhances task performance over traditional video-only systems. Implications for the design of multimodal systems to support collaborative physical tasks are also discussed.


Publisher Statement

“© ACM, (2003). This is the author’s version of the work. It is posted here by permission of ACM for your personal use. Not for redistribution. The definitive version was published in Proceedings of the 5th international conference on Multimodal interfaces. {1-58113-621-8 (2003)}