Using Visual Attention for Intelligent Multimodal UI

Ken Pfeuffer slides Mon, 12.03.2018

User interfaces can benefit from the information of the user’s visual attention, to optimise the UI and offer users gaze controls. In this talk, I will discuss various projects that we conducted to improve the current gaze UI integration on a range between implicit toward explicit usage of the eye modality. For example, we implicitly used eye gaze to understand visual search patterns on mobile devices, which helps to build precise user performance models. In other projects, we explicitly used eye gaze to provide users shortcuts to system controls, remote menus, and out of reach UI elements.