Extending the Input Space of Smartwatches

Smartwatches provide quick and easy access to information. Due to their wearable nature, users can perceive the information while being stationary or on the go. The main drawback of smartwatches, however, is the limited input possibility. They use similar input methods as smartphones but thereby suffer from a smaller form factor. To extend the input space of smartwatches, we present GestureSleeve, a sleeve made out of touch enabled textile. It is capable of detecting different gestures such as stroke based gestures or taps. With these gestures, the user can control various smartwatch applications. Exploring the performance of the GestureSleeve approach, we conducted a user study with a running application as use case. In this study, we show that input using the GestureSleeve outperforms touch input on the smartwatch. In the future the GestureSleeve can be integrated into regular clothing and be used for controlling various smart devices.

Implicit Navigation

Pedestrian navigation systems require users to perceive, interpret, and react to navigation information. This can tax cognition as navigation information competes with information from the real world. We propose actuated navigation, anew kind of pedestrian navigation in which the user does not need to attend to the navigation task at all. An actuation signal is directly sent to the human motor system to influence walking direction. To achieve this goal we stimulate the sartorius muscle using electrical muscle stimulation. The rotation occurs during the swing phase of the leg and can easily be counteracted. The user therefore stays in control. We discuss the properties of actuated navigation and present a lab study on identifying basic parameters of the technique as well as an outdoor study in a park. The results show that our approach changes a user’s walking direction by about 16°/m on average and that the system can successfully steer users in a park with crowded areas, distractions, obstacles, and uneven ground.

Emotion Actuation

The human body reveals emotional and bodily states through measurable signals, such as body language and electroencephalography. However, such manifestations are difficult to communicate to others remotely. We propose EmotionActuator, a proof-of-concept system to investigate the transmission of emotional states in which the recipient performs emotional gestures to understand and interpret the state of the sender. We call this kind of communication embodied emotional feedback, and present a prototype implementation. To realize our concept we chose four emotional states: amused, sad, angry, and neutral. We designed EmotionActuator through a series of studies to assess emotional classification via EEG, and create an EMS gesture set by comparing composed gestures from the literature to sign-language gestures. Through a final study with the end-to-end prototype interviews revealed that participants like implicit sharing of emotions and find the embodied output to be immersive, but want to have control over shared emotions and with whom.This work contributes a proof of concept system and set of design recommendations for designing embodied emotional feedback systems.

Implicit Authentication using Wearable Devices

Secure user identification is important for the increasing number of eyewear computers but limited input capabilities pose significant usability challenges for established knowledge-based schemes, such as a passwords or PINs. We present SkullConduct, a biometric system that uses bone conduction of sound through the user’s skull as well as a microphone readily integrated into many of these devices, such as Google Glass. At the core of SkullConduct is a method to analyze the characteristic frequency response created by the user’s skull using a combination of Mel Frequency Cepstral Coefficient (MFCC) features as well as a computationally light-weight 1NN classifier. We report on a controlled experiment with 10 participants that shows that this frequency response is person-specific and stable — even when taking off and putting on the device multiple times — and thus serves as a robust biometric. We show that our method can identify users with 97.0% accuracy and authenticate them with an equal error rate of 6.9%, thereby bringing biometric user identification to eyewear computers equipped with bone conduction technology.

A Novel Mobile Authentication Method

Touch-enabled user interfaces have become ubiquitous on portable devices. At the same time, authentication using touch input is problematic, since finger smudge traces may allow attackers to reconstruct passwords. We present SmudgeSafe, an authentication system that uses random geometric image transformations, such as translation, rotation, scaling, shearing, and flipping, to increase the security of cued-recall graphical passwords. We describe the design space of these transformations and report on two user studies: A lab-based security study involving 20 participants in attacking user-defined passwords, using high quality pictures of real smudge traces captured on a mobile phone display; and an in-the-field usability study with 374 participants who generated more than 130,000 logins on a mobile phone implementation of SmudgeSafe. Results show that SmudgeSafe significantly increases security compared to authentication schemes based on PINs and lock patterns, and exhibits very high learnability, efficiency, and memorability.

Exploring User Behavior in the Wild

Common user authentication methods on smartphones, such as lock patterns, PINs, or passwords, impose a trade-off between security and password memorability. Image-based passwords were proposed as a secure and usable alternative. As of today, however, it remains unclear how such schemes are used in the wild. We present the first study to investigate how image-based passwords are used over long periods of time in the real world. Our analyses are based on data from 2318 unique devices collected over more than one year using a custom application released in the Android Play store. We present an in-depth analysis of what kind of images users select, how they define their passwords, and how secure these passwords are. Our findings provide valuable insights into real-world use of image-based passwords and inform the design of future graphical authentication schemes.

Exploiting Thermal Reflection for Interactive Systems

Thermal cameras have recently drawn the attention of HCI researchers as a new sensory system enabling novel interactive systems. They are robust to illumination changes and make it easy to separate human bodies from the image background. Far-infrared radiation, however, has another characteristic that distinguishes thermal cameras from their RGB or depth counterparts, namely thermal reflection. Common surfaces reflect thermal radiation differently than visual light and can be perfect thermal mirrors. In this paper, we show that through thermal reflection, thermal cameras can sense the space beyond their direct field-of-view. A thermal camera can sense areas besides and even behind its field-of-view through thermal reflection. We investigate how thermal reflection can increase the interaction space of projected surfaces using camera-projection systems. We moreover discuss the reflection characteristics of common surfaces in our vicinity in both the visual and thermal radiation bands. Using a proof-of-concept prototype, we demonstrate the increased interaction space for hand-held camera-projection system. Furthermore, we depict a number of promising application examples that can benefit from the thermal reflection characteristics of surfaces.

Modeling Distant Pointing for Compensating Systematic Displacements

Distant pointing at objects and persons is a highly expressive gesture that is widely used in human communication. Point- ing is also used to control a range of interactive systems. For determining where a user is pointing at, different ray casting methods have been proposed. In this paper we assess how accurately humans point over distance and how to improve it. Participants pointed at projected targets on a wall display from 2m and 3m while standing and sitting. Testing three common ray casting methods, we found that even with the most accurate one the average error is 61.3cm. We found that all tested ray casting methods are affected by systematic dis- placements. Therefore, we trained a polynomial to compen- sate this displacement. We show that using a user-, pose-, and distant-independent quartic polynomial can reduce the aver- age error by 37.3%.

Cognitive Effects of Interactive Public Display Applications

Many public displays are nowadays equipped with different types of sensors. Such displays allow engaging and persistent user experiences to be created, e.g., in the form of gesture-controlled games or content exploration using direct touch at the display. However, as digital displays replace traditional posters and billboards, display owners are reluctant to deploy interactive content, but rather adapt traditional, non-interactive content. The main reason is, that the benefit of such interactive deployments is not obvious. Our hypothesis is that interactivity has a cognitive effect on users and therefore increases the ability to remember what they have seen on the screen — which is beneficial both for the display owner and the user. In this paper we systematically investigate the impact of interactive content on public displays on the users’ cognition in different situations. Our findings indicate that overall memorability is positively affected as users interact. Based on these findings we discuss design implications for interactive public displays.

What People Really Remember – Understanding Cognitive Effects When Interacting with Large Displays

This paper investigates how common interaction techniques for large displays impact on recall in learning tasks. Our work is motivated by results of prior research in different areas that attribute a positive effect of interactivity to cognition. We present findings from a controlled lab experiment with 32 participants comparing mobile phone-based interaction, touch interaction and full-body interaction to a non-interactive baseline. In contrast to prior findings, our results reveal that more movement can negatively influence recall. In particular we show that designers are facing an immanent trade-off between creating immersive and engaging user experiences and memorable content.

Using Eye-Tracking to Support Interaction with Layered 3D Interfaces on Stereoscopic Displays

In this paper, we investigate the concept of gaze-based in- teraction with 3D user interfaces. We currently see stereo vision displays becoming ubiquitous, particularly as auto- stereoscopy enables the perception of 3D content without the use of glasses. As a result, application areas for 3D beyond entertainment in cinema or at home emerge, including work settings, mobile phones, public displays, and cars. At the same time, eye tracking is hitting the consumer market with low-cost devices. We envision eye trackers in the future to be integrated with consumer devices (laptops, mobile phones, displays), hence allowing the user’s gaze to be analyzed and used as input for interactive applications. A particular chal- lenge when applying this concept to 3D displays is that cur- rent eye trackers provide the gaze point in 2D only (x and y coordinates). In this paper, we compare the performance of two methods that use the eye’s physiology for calculating the gaze point in 3D space, hence enabling gaze-based inter- action with stereoscopic content. Furthermore, we provide a comparison of gaze interaction in 2D and 3D with regard to user experience and performance. Our results show that with current technology, eye tracking on stereoscopic displays is possible with similar performance as on standard 2D screens.

Evaluating Stereoscopic 3D for Automotive User Interfaces in a Real-World Driving Study

This paper reports on the use of in-car 3D displays in a real-world driving scenario. Today, stereoscopic displays are becoming ubiquitous in many domains such as mobile phones or TVs. Instead of using 3D for entertainment, we explore the 3D effect as a mean to spatially structure user interface (UI) elements. To evaluate potentials and drawbacks of in-car 3D displays we mounted an autostereoscopic display as instrument cluster in a vehicle and conducted a real-world driving study with 15 experts in automotive UI design. The results show that the 3D effect increases the perceived quality of the UI and enhances the presentation of spatial information (e.g., navigation cues) compared to 2D. However, the effect should be used well-considered to avoid spatial clutter which can increase the system’s complexity.

User Defined Exhibitions – Exploring Possibilities to Involve Visitors in the Design of Museum Exhibitions

When connecting the physical experience of museums and exhibitions with relevant digital information, creating engaging exhibitions is a challenge shared by both curators and interaction designers. There is a wealth of artifacts scattered across museums, their archives, and remote locations. Additionally, online repositories hold a multitude of digital cultural heritage content about these artifacts. The challenge of creating an exhibition lies in selecting related artifacts, combining them with available or proprietary information, and arranging them in a way that serves an intended story line. Common practice dictates the creation to be limited to a set of curators working together. We propose a visitor-centered design approach including a physical interaction space where visitors can browse archived objects and put together their own exhibitions with regards to a unique or given story line. We conducted an initial field study and report findings regarding the likes and dislikes of visitors.

Parallel Exhibits: Combining Physical and Virtual Exhibits

People have a special fascination for original physical objects, their texture, and visible history. However, the digitiza- tion of exhibits and the use of these data is a current challenge for museums. We believe that museums need to capitalize on the affordances of physical exhibits to help users navigate their more extensive virtual collections. Although lacking materiality, virtual objects have other advantages: They can easily be manipulated, rearranged, duplicated, and moved. This offers new op- portunities for visitors to engage with museum collections and the curatorial process in a creative way. In this paper, we propose a concept designed to make use of existing digital content in combination with physical exhibits in museums, which we call Parallel Exhibits.
Parallel Exhibits is a system that enables museum visitors to interact with traditional museum collections and virtual objects at the same time. It is an interactive exhibition space where visitors and curators enter a design dialogue mediated by tech- nology. Curators display a selection of physical objects and invite visitors to complete the exhibition with virtual objects from the museum’s collections or elsewhere. The ever-changing display can be augmented with digital text labels and messages. We implemented Parallel Exhibits as a web application, which bears the advantage of easily running the application on different platforms. We tested the system both in a museum, using an interactive table and a projection wall, and as part of an online sur- vey reaching a broader audience. In the field study we observed that visitors like to share their ideas and thoughts while using the table. The results of the online survey indicate that visitors like to contribute to exhibitions. In this paper, we describe the technical design of Parallel Exhibits, as well as the outcomes of the on-site study and online survey.

FrameBox

The core idea behind the FrameBox is to allow users to work with a large variety of materials, including paper, foil, and 3D mockups created with a laser printer, and to spatially position the different elements. Hence, we designed a cubic box made of acrylic glass with a number of slots that represent the different depth layers and allow for positioning UI elements on the z-axis in discrete steps. Within each slot, UI elements can be easily moved in the x-direction. Positioning on the y-axis can be achieved by means of paper-clips. In accordance to the specified requirements, we built a series of FrameBoxes for different application areas. One FrameBox was aimed for the design of automotive UIs and two for the design of mobile phones UIs (one for landscape and one for portrait mode).

Download the *.svg layouts here.

MirrorBox

As a second prototyping tool, we designed the MirrorBox. We use a number of semi-transparent mirrors in the front and a surface-coated mirror in the back to allow users to see the mirrored image of a UI element projected from below. The mirrors are aligned one after another on top of a light source. Foils can be used to design UI elements, which are then sliced between the mirrors and the light source to make them visible to the user in front of the MirrorBox.

Let me catch this! Experiencing Interactive 3D Cinema through Collecting Content with a Mobile Phone

The entertainment industry is going through a transforma- tion, and technology development is affecting howwe can en- joy and interact with the entertainment media content in new ways. In our work, we explore how to enable interaction with content in the context of 3D cinemas by means of a mobile phone. Hence, viewers can use their personal devices to re- trieve, for example, information on the artist of the soundtrack currently playing or a discount coupon on the watch the main actor is wearing. We are particularly interested in the user ex- perience of the interactive 3D cinema concept, and how dif- ferent interactive elements and interaction techniques are per- ceived. We report on the development of a prototype applica- tion utilizing smart phones and on an evaluation in a cinema context with 20 participants. Results emphasize that design- ing for interactive cinema experiences should drive for holis- tic and positive user experiences. Interactive content should be tied together with the actual video content, but integrated into contexts where it does not conflict with the immersive experience with the movie.

Midair Displays: Free-Floating Pervasive Displays

Due to advances in technology, displays could replace literally any surface in the future, includingwalls, windows, and ceilings. At the same time, midair remains a relatively unexplored domain for the use of displays as of today, particularly in public space. Neverthe- less, we see large potential in the ability to make displays appear at any possible point in space, both indoors and outdoors. Such dis- plays, that we call midair displays, could control large crowds in emergency situations, they could be used during sports for naviga- tion and feedback on performance, or as group displays. We see midair displays as a complementary technology to wearable dis- plays. In contrast to statically deployed displays they allow infor- mation to be brought to the user anytime and anywhere.We explore the concept of midair displays and show that with current technol- ogy, e.g., copter drones, such displays can be easily built. A study on the readability of such displays showcases the potential and fea- sibility of the concept and provides early insights.

An Interactive Curtain for Media Usage in the Shower

Access to digital information became almost ubiquitous. There are only few situations left where digital media cannot be ac- cessed. Showering is probably the only regular and common activity that does not allow to access and interact with digital media. Based on a large-scale survey, we identified potential applications that users want to use in the shower and designed a system that augments the user’s showering experience to pro- vide pervasive media access. We developed a projection-based system that augments shower curtains from the back side and recognizes user input using a thermal camera. Through a user study in a running shower, we collected feedback from poten- tial users and evaluated different algorithms to recognize touch input on a shower curtain. Our results show that participants are enthusiastic about accessing and controlling media using an interactive shower curtain. Furthermore, we identified two algorithms that are robust enough to be used in challenging environments such as a shower.

As a first analysis (see citation below) we used this data to analyze the differences in the workload in terms of road type as well as especially important parts of the route such as exits and on-ramps. Furthermore, we investigated the correlation between the objective assessed and subjective measured data.

Citation / First Paper

A first publication about this data collection project has been published at AutomotiveUI ’13:

Important Note and Copyright

Disclaimer: This data is property of the hciLab Group, Institute for Visualization and Interactive Systems, Unversity of Stuttgart, Germany. If you use this dataset, you agree to the license information available in the license file license.txt (and accompanying files) delivered with the dataset. If you would like to use the data set but need a different license for your specific use case, please contact us! You may use this data for scientific, non-commercial purposes, provided that you give credit to the owners when publishing any work based on this data as defined in the provided license files. We would also be very interested to hear back from you if you use our data in any way and are happy to answer any questions or address any remarks related to it.

hciLab Driving Dataset: Copyright © 2012-2013 hciLab, Institute for Visualization and Interactive Systems (VIS), University of Stuttgart, Pfaffenwaldring 5a, 70569 Stuttgart, Germany

This hciLab Driving Dataset is made available under the Open Database License (ODbL 1.0): http://opendatacommons.org/licenses/odbl/1.0/. Any rights in individual contents of the database are licensed under the Database Contents License (DbCL 1.0): http://opendatacommons.org/licenses/dbcl/1.0/ – See more at: http://opendatacommons.org/licenses/odbl/#sthash.TfyXJQAv.dpuf

You are free to copy, distribute, use, modify, transform, build upon, and produce derived works from our data as long as you attribute any use of the data, or works produced from the data, in the manner specified in the licenses. Read the full ODbL 1.0 and DbCL 1.0 license texts for the exact terms that apply. The licenses are courtesy of the Open Knowledge Foundation.

Download the Dataset

The dataset is available as a ZIP file (37.9 MB): hcilab Driving Dataset