Re-envisioning the Keyboard as a Spatial User Interface

Current screen reader technology presents the content of web pages to users as a one-dimensional stream of data. However, as web pages themselves are two-dimensional, we believe this output method is lossy since it removes the visuospatial relationships between elements. Keyboard surface interaction is a growing field within human computer interaction that studies the repurposing of keyboards for nontraditional applications. We compare a keyboard surface interaction system (Fingers) designed to preserve the visuospatial relationships between web page elements to a traditional screen reader (VoiceOver) to learn which has better interaction effectiveness measured in terms of time spent and interactions per task on four simulated shopping tasks. We find no consistent difference in effectiveness between the two conditions. Participants consistently have more interactions per task with Fingers than with VoiceOver, but it is unclear whether this positively or negatively impacts their time. We find that learning a new mental model of a keyboard is challenging for participants. We suggest further research into the applicability of keyboard surface interaction as an alternative form of web accessibility, but that future studies take a longitudinal nature to help participants learn the mental model required and overcome the novelty effect.