Sensory Perception & Interaction Research Group

We examine how haptic feedback could enable an implicit human-computer interaction, in the context of an audio stream listening use case where a device monitors a user's electrodermal activity for orienting responses to external interruptions. When such a response is detected, our previously developed system automatically places a bookmark in the audio stream for later resumption of listening. Here, we investigate two uses of haptic feedback to support this implicit interaction and mitigate effects of noisy (false-positive) bookmarking: (a) low-attention notification when a bookmark is placed, and (b) focused-attention display of bookmarks during resumptive navigation. Results show that haptic notification of bookmark placement, when paired with visual display of bookmark location, significant improves navigation time. Solely visual or haptic display of bookmarks elicited equivalent navigation time; however, only the inclusion of haptic display significantly increased accuracy. Participants preferred haptic notification over no notification at interruption time, and combined haptic and visual display of bookmarks to support navigation to their interrupted location at resumption time. Our contributions include an approach to handling noisy data in implicit HCI, an implementation of haptic notifications that signal implicit system behavior, and discussion of user mental models that may be active in this context.