Author: Sven Mayer

The final part of the afternoon block activity will focus on responsible research, ethics, and privacy. As a community, we expect this to be a key focus issue for the future, with physiological interaction and technology presenting several potentially invasive challenges. We intend to use the Legal and Moral-IT cards on tables to provoke discussion.

Either submitting authors or invited speakers will be invited to co-facilitate this activity, where they have already considered aspects of this theme.

We will switch the focus from physiological evaluation to a multi-user, social perspective of sharing biosignals.

Here, participants who selected the topic of sharing biosignals will pitch their presentations on their submitted position paper – case study. Groups previously formed in Round 4 will discuss how biodata can be represented, either individually or as an aggregate from multiple users, offering potential privacy solutions.

Second, groups will discuss the centrality of visual representations in biofeedback designs and explore the potential of other sensory modalities. Participants will highlight the challenges and opportunities in mapping and representing signals for multiple users, presenting the outcome of discussions by emphasizing the need for symmetrical data representation and its impact on user interpretation.

Each group will develop a shared definition of how each biosignal is physiologically interpreted, and to ensure a breadth of discussion, list what each signal is mapped in their HCI research to which construct and as input for interaction. The definition and application areas will be presented and discussed by a representative of each group.

We will ask each group to return to the afternoon session with a considered set of biosignal measures that they employ in their research and which challenges they face regarding the construct validity.

Pedro Lopes, University of Chicago

Keynote title: Integrating interactive devices with the user’s body & brain

Keynote Synopsis

When we look back to the early days of computing, user and device were distant, often located in separate rooms. Then, in the ’70s, personal computers “moved in” with users. In the ’90s, mobile devices moved computing into users’ pockets. Recently, wearables brought computing into constant physical contact with the user’s skin. These transitions proved useful: moving closer to users allowed interactive devices to sense more of their users and act more personal. The main question that drives my research is: what is the next interface paradigm that supersedes wearable devices?

I propose that the next generation of interfaces will be defined by how devices integrate with the user’s biological senses and actuators. For the past years, my lab has been exploring how this body-device integration allows to engineer interactive devices that intentionally borrow parts of the body for input and output, rather than adding more technology to the body.

The first key advantage of body-device integration is that puts forward a new generation of miniaturized devices; allowing us to circumvent traditional physical constraints. For instance, in the case of our devices based on electrical muscle stimulation, they illustrate how to create realistic haptic feedback (e.g., forces in VR/AR) while circumventing the constraints imposed by robotic exoskeletons, which need to balance their output power against the size of their motors and batteries. Taking this further, we successfully applied this body-device integration approach to other sensory modalities. For instance, we engineered a device that delivers chemicals to the user to render temperature sensations without the need to rely on cumbersome thermal actuators. Our approach to miniaturizing devices is especially useful to advance mobile interactions, such as in virtual or augmented reality, where users have a desire to remain untethered.

A second key advantage is that integrating devices with the user’s body allows for new interactions to emerge without encumbering the user’s hands. Using our approach, we demonstrated how to create tactile sensations in the user’s fingerpads without putting anything directly on the fingerpads—instead we intercept the fingerpad nerves from the back of the user’s hand. This allows users to benefit from haptic feedback (e.g., for guidance in VR/AR) without encumbering their dexterity. Taking this further, we demonstrated that using brain stimulation we can achieve haptic feedback on all four limbs without wearing any hardware (e.g., feeling forces & tactile sensations on both hands and feet)—opening up a new way to achieve haptics by directly stimulation the source (brain) rather than the endpoints (limbs).

A third facet is that our integrated devices enable new physical modes of reasoning with computers, going beyond just symbolic thinking. For example, we have engineered a set of devices that control the user’s muscles to provide tacit information to the user, such as a muscle-stimulation devices to learn new skills (piano or sign-language), or our wearable devices to allow users to control information using their bodies and without the need for screens (e.g., using their lips, muscles or feet for both input and output).

A fourth key aspect that we found while integrating devices with the user’s body is that we can endow users with new physical abilities. We engineered a device that allows users to locate odor sources by “smelling in stereo” as well as a device that physically accelerates one’s reaction time using muscle stimulation, which can steer users to safety or even catch a falling object that they would normally miss.

While this integration between human and computer is beneficial (e.g., faster reaction time, realistic simulations in VR/AR, or faster skill acquisition), it also requires tackling new challenges, such as improving the precision of how we safely stimulate the body or the question of agency: do we feel in control when our body is integrated with an interface? Together with our colleagues in neuroscience, we have been measuring how our brain encodes agency to improve the design of this new type of integrated interfaces. We found that, even in the extreme case of our interfaces that electrically control the user’s muscles, it is possible to improve the sense of agency. More importantly, we found that it is only by preserving the user’s sense of agency that these integrated devices provide benefits even after the user takes them out.

Finally, I believe that these bodily-integrated devices are the natural succession to wearable interfaces and allow us to investigate how interfaces will connect to our bodies in a more direct and personal way.

We will focus on important challenges for Open Science practices and create a physical post-it mindmap or on a Miro Board.

Francesco Chiossi will provide an introductory talk (10 minutes) on open guiding principles, where data should be Findable, Accessible, Interoperable, and Reusable (FAIR).

This allows participants to have a shared background in the topic.

Then, participants will engage in 3 rounds of medium to small group discussions (4-8 participants per group) in the form of roundtables. The main goal of each group is to identify how each group member follows or does not follow FAIR guidelines in their work. One moderator per group will be chosen to report to all other participants on the discussion outcomes.

The workshop will begin with an introduction by the organizers and a short warm-up speed-dating activity. This speed-dating activity will help attendees to focus on the workshop interest and engage with the other participants. Participants will be asked to introduce themselves and explain their motivation to participate in the workshop. They will also be asked to briefly introduce the work (e.g., method or application). The ultimate aims of Round 1 are a) to explicate the scope of the workshop and the expertise in the room, b) to highlight the variety of expertise, and c) to end up with mixed groups around the tables.