Congratulations to Frau Doktor Dagmar Kern for a great PhD defense (No. 5)

Dagmar Kern has successfully defended her PhD on “Supporting the Development Process of Multimodal and Natural Automotive User Interfaces” in Essen. External examiner was Antonio Krüger from University of Saarbrücken. Her dissertation will be available online soon. The core contribution of the thesis is the investigation of how to improve a user centered design process for automotive user interfaces. In order to systematically assess user interface designs in cars she developed a design space (inspired by Card et al [5]). In various cases studies she create novel in-car user interfaces and explored experimentally the implications on driver distraction.

Dagmar started working with me as a student of Media Informatics at the LMU Munich in 2005, then jointed my group at Fraunhofer IAIS/BIT in Bonn and move in 2007 with the group to Essen. She was for a short research stay in Saarbrücken and Milton Keynes and was extremely productive over the last years – 18 publications she co-authored are listed in DBLP and here are some highlights of here research:

  • exploration of how to present navigation information (e.g. vibra tactile steering wheel) [1]
  • gazemarks – an approach to aid attention switching between the road and an in car display using eye gaze date [2]
  • a multi-touch steering wheel, that reduced driver distraction [3]
  • a design space for automotive user interfaces [4]

Additionally to the publications one of the side products of here thesis is the CARS open source driving simulator. It is a configurable low cost simulator that can be used to measure driver distraction, e.g. as an alternative to LCT.

Dagmar’s defense brought us back to Essen and it was great to meet many colleagues again. We finally managed to have a group photo taken with nearly all the team (Elba is missing in the Photo).

The doctoral hat may look strange to non-Germans but it has some funny tradition. It is hand crafted by the colleagues and each of the items on the hat tells a story – usually known to the group but in the best case hard to guess for outsiders. Besides others Dagmar’s hat included a scrap heap of cars, a giraffe, a personal vibration device, a yoyo, a railway station building side, and a steering wheel cover.

[1] Dagmar Kern, Paul Marshall, Eva Hornecker, Yvonne Rogers, and Albrecht Schmidt. 2009. Enhancing Navigation Information with Tactile Output Embedded into the Steering Wheel. InProceedings of the 7th International Conference on Pervasive Computing (Pervasive ’09). Springer-Verlag, Berlin, Heidelberg, 42-58. DOI=10.1007/978-3-642-01516-8_5 (free PDF)

[2] Dagmar Kern, Paul Marshall, and Albrecht Schmidt. 2010. Gazemarks: gaze-based visual placeholders to ease attention switching. In Proceedings of the 28th international conference on Human factors in computing systems (CHI ’10). ACM, New York, NY, USA, 2093-2102. DOI=10.1145/1753326.1753646 (free PDF)

[3] Tanja Döring, Dagmar Kern, Paul Marshall, Max Pfeiffer, Johannes Schöning, Volker Gruhn, and Albrecht Schmidt. 2011. Gestural interaction on the steering wheel: reducing the visual demand. In Proceedings of the 2011 annual conference on Human factors in computing systems (CHI ’11). ACM, New York, NY, USA, 483-492. DOI=10.1145/1978942.1979010 (free PDF)

[4] Dagmar Kern and Albrecht Schmidt. 2009. Design space for driver-based automotive user interfaces. In Proceedings of the 1st International Conference on Automotive User Interfaces and Interactive Vehicular Applications (AutomotiveUI ’09). ACM, New York, NY, USA, 3-10. DOI=10.1145/1620509.1620511 (free PDF)

[5] Stuart K. Card, Jock D. Mackinlay, and George G. Robertson. 1991. A morphological analysis of the design space of input devices. ACM Trans. Inf. Syst. 9, 2 (April 1991), 99-122. DOI=10.1145/123078.128726

Auto-UI 2012 in the US, looking for hosts for 2013

The next and 4rd international conference on Automotive User Interfaces and Vehicular Applications (AutoUI 2012) will be in Portsmouth, New Hampshire in the USA. The dates for the conference are 17-19 of October 2012. The first day for workshops and tutorials and 2 days for the main conference. Portsmouth is about an 1 hour drive from Boston and the timing is great (fall foliage – the photos of the colorful forests looked good 😉

The steering committee ( is inviting proposals for Auto-UI 2013 from the community of researchers in the field. The conference was 2009 in Essen (Germany), 2010 in Pittsburgh (USA), 2011 in Salzburg (Austria), and it will be in 2012 in Portsmouth (USA). Keeping this cycle between Europe and North America 2013 should be in Europe.

Paper and demo in Salzburg at Auto-UI-2011

At the automotive user interface conference in Salzburg we presented some of our research. Salzburg is a really nice place and Manfred and his team did a great job organizing the conference!

Based on the Bachelor Thesis of Stefan Schneegaß and some follow-up work we published a full paper [1] that describes a KLM-Model for the car and a prototyping tools that makes use of the model. In the model we look at the specific needs in the car, model rotary controllers, and cater for the limited attention while driving. The prototyping tool provides means to quickly estimate interaction times. It supports visual prototyping using images of the UI and tangible prototyping using Nic Villar´s VoodooIO. Looking forward to having Stefan on our team full-time 🙂

We additionally had a demo on a recently completed thesis by Michael Kienast. Here we looked at how speech and gestures can be combined for controlling functions, such as mirror adjustments or windscreen wipers, in the car. This multimodal approach combines the strength of gestural interaction and speech interaction [2].

The evening event of the conference was at Festung Hohensalzburg – with a magnificent view over the twon!

[1] Stefan Schneegaß, Bastian Pfleging, Dagmar Kern, Albrecht Schmidt. Support for modeling interaction with in-vehicle interfaces. (PDF) Proceedings of 3rd international conference on Automotive User Interfaces and Vehicular Applications 2011 ( Salzburg. 30.11-2.12.2011

[2] Bastian Pfleging, Michael Kienast, Albrecht Schmidt. DEMO: A Multimodal Interaction Style Combining Speech and Touch Interaction in Automotive Environments. Adjunct proceedings of 3rd international conference on Automotive User Interfaces and Vehicular Applications 2011 ( Salzburg. 30.11-2.12.2011

Bryan Reimer: Opening keynote at Auto-UI 2011 in Salzburg

Bryan started his keynote talk the automotive user interface conference ( in Salzburg with reminding us that having controversial discussions about the HMI in the car is not new. Quoting a newspaper article from the 1930s on the introduction of the radio in the car and its impact on the driver he picked an interesting example, that can be seen as the root of many issues we have now with infotainment systems in the car.

The central question he raised is: how to create user interface that fit human users? He made an important point: humans are not “designed” to drive at high speed in complex environments; perception has evolved for walking and running in natural environment. Additionally to the basic limitations of human cognition, there is a great variety of capabilities of drivers, their skills and cognitive ability (e.g. influence of age). A implication of the global change is demographics is that the average capabilities of a drivers will be reduced – basically as many older people will be drivers…

Over the last 100 years cars have changes significantly! Looking more closely Bryan argues that much of the chance happened in the last 10 years. There has been little change from the 1950s to the 1990s with regard to the car user interface.

It is apparent that secondary tasks are becoming more important to the user. Users will interact more while driving because the can. It is however not obvious that they are capable of it.

Even given these developments it is apparent that driving has become safer. Passive safety has been improved massively and this made driving much safer. There seems to be a drawback to this as well, as people may take greater risks as they feel safer. The next step is really to avoid accidence in the first place. Bryan argues that the interaction between driver, environment, and vehicles is very important in that. He suggests that we should make more of an effort to create systems that fit the drivers.

The Yerkes-Dodson Law helps to understand how to design systems that keep peoples attention in the optimal performance. He made an important point: there are certain issues that cannot be solved, e.g. if someone is tired we can do only very little – the driver will need to rest. We should make sure that we take these things into account when designing systems.

Visual distraction is an obvious factor and much discussed in the papers at the conference – but Bryan argued that “eyes on the road” is not equal to “mind on the road”. I think this is really a very important point. Ensuring that people keep their eyes on the road, seeing things is not enough. The big resulting question is how to keep or get people focused on the street and environment. It seems there is some more research to do…

The variety of interfaces and interaction metaphors build into cars opens more choices but at the same time creates problems, as people need to learn and understand them. A simple question such as: How do you switch the car off? may be hard to answer (Bryan had the example of a car with a push button starter, where you cannot remove the key). I think there are simple questions that can be learned from industry and production machines… add an emergency stop button and make it mandatory 😉

If you are interested more about Bryan’s work look at his webpage or his page at the MIT agelab or one of his recent publications [1] in the IEEE Pervasive Computing Magazine’s special issue on automotive computing, see [2] for an introduction to the special issue.

Sorry for the poor quality photos … back row and an iPhone…

[1] Joseph F. Coughlin, Bryan Reimer, and Bruce Mehler. 2011. Monitoring, Managing, and Motivating Driver Safety and Well-Being. IEEE Pervasive Computing 10, 3 (July 2011), 14-21. DOI=10.1109/MPRV.2011.54

[2] Albrecht Schmidt, Joseph Paradiso, and Brian Noble. 2011. Automotive Pervasive Computing. IEEE Pervasive Computing 10, 3 (July 2011), 12-13. DOI=10.1109/MPRV.2011.45

Automotive UI in Salzburg

Manfred Tscheligi opend the Automotive UI conference in Salzburg. The conference is now in its 3rd year after 2009 in Essen and 2010 in Pittsburgh. The conference is growing – there were well over 130 people registered 🙂

The proceedings of the conference series are online available at

3rd Auto-UI Proceedings 2011 (soon in the ACM DL)
2nd Auto-UI Proceedings 2010 (ACM DL)
1st Auto-UI Proceedings 2009 (ACM DL)

Our Paper and Note at CHI 2010

Over the last year we looked more closely into the potential of eye-gaze for implicit interaction. Gazemarks is an approach where the users’ gaze is continuously monitored and when leaving a screen or display the last active gaze area is determined and store [1]. When the user looks back at this display this region is highlighted. By this the time for attention switching between displays was in our study reduced from about 2000ms to about 700ms. See the slides or paper for details. This could make the difference that we enable people to safely read in the car… but before this more studies are needed 🙂

Together with Nokia Research Center in Finland we looked at how we can convey the basic message of an incoming SMS already with the notification tone [2]. Try the Emodetector application for yourself or see the previous post.

[1] Kern, D., Marshall, P., and Schmidt, A. 2010. Gazemarks: gaze-based visual placeholders to ease attention switching. In Proceedings of the 28th international Conference on Human Factors in Computing Systems (Atlanta, Georgia, USA, April 10 – 15, 2010). CHI ’10. ACM, New York, NY, 2093-2102. DOI=

[2] Sahami Shirazi, A., Sarjanoja, A., Alt, F., Schmidt, A., and Hkkilä, J. 2010. Understanding the impact of abstracted audio preview of SMS. In Proceedings of the 28th international Conference on Human Factors in Computing Systems (Atlanta, Georgia, USA, April 10 – 15, 2010). CHI ’10. ACM, New York, NY, 1735-1738. DOI=

PS: the social event was at the aquarium in Atlanta – amazing creatures! Again supprised how well the N95 camera works even under difficult light conditions…

CFP: 2nd Int. Conf. on Automotive User Interfaces and Interactive Vehicular Applications

The call for the 2nd International Conference on Automotive User Interfaces and Interactive Vehicular Applications is online – see – AutomotiveUI’10.

The conference will be in Pittsburgh on 11-12 Nov 2010.

Do not miss the submission deadline: 02 July 2010.

The call includes a wide range of topics, including:

  • new concepts for in-car user interfaces
  • multi-modal in-car user interfaces
  • in-car speech user interfaces
  • text input and output while driving
  • multimedia interfaces and in-car entertainment
  • evaluation of in-car user interfaces
  • methods and tools for automotive user interface research
  • development tools and methods for automotive user interfaces
  • automotive user interface frameworks and toolkits
  • detecting and estimating user intentions
  • detecting user distraction and estimating cognitive load
  • user interfaces for assistive functionality
  • biometrics and physiological sensors as a user interface component
  • using sensors and context for interactive experiences in the car
  • user interfaces for information access while driving
  • user interfaces for navigation systems
  • applications and user interfaces for inter-vehicle communication
  • in-car gaming

We enjoyed last years conference in Essen 🙂 There are several posts online in my blog and we had a conference report in IEEE Pervasive Magazine [1]. Last years proceedings are online in the ACM DL and on the 2009 conference website.

The full call for papers is online at:

[1] Albrecht Schmidt, Wolfgang Spiessl, Dagmar Kern, “Driving Automotive User Interface Research,” IEEE Pervasive Computing, vol. 9, no. 1, pp. 85-88, Jan.-Mar. 2010, doi:10.1109/MPRV.2010.3.

Article on Driving Automotive User Interface Research in IEEE Pervasive

We wrote an article on the automotive user interface conference that took place in Essen in September 2009. In the current issue of the IEEE Pervasive Computing Magazine this paper is published and gives an overview of the conference [1]. We hope our article highlights the key ideas that were presented at the conference.

Abstract. Cars offer an interesting but challenging microcosm for pervasive computing research and, in particular, for interaction with pervasive computing systems. Increasingly, researchers are looking at interactive applications in the car and investigating human-car interaction from a computer science-rather than an ergonomics or mechanical engineering-perspective. This article reports on the International Conference on Automotive User Interfaces and Interactive Vehicular Applications, wherein participants shared presentations on topics such as aesthetics, user interaction and distraction, safety, and driver monitoring.

[1] Albrecht Schmidt, Wolfgang Spiessl, Dagmar Kern, “Driving Automotive User Interface Research,” IEEE Pervasive Computing, vol. 9, no. 1, pp. 85-88, Jan.-Mar. 2010, doi:10.1109/MPRV.2010.3.

Auto-UI 2010 announced – 2009 Proceedings available in the ACM DL

The Auto-UI 2009 conference in Essen is over – and for us it was very enjoyable to have this many visitors at the University of Duisburg-Essen – see the photos. The conference facilitated good discussions and had a very constructive atmosphere. We should continue this exchange of ideas and there is always room for improvement… and that is why there is a Auto-UI conference 2010 in Pittsburgh, US – and there is interest beyond this in hosting the conference.

You can register to get information about the next conference on the Auto-UI webpage.

The proceedings are now online in the ACM DL and the linked on the program website.

Social event at Zeche Zollverein, Accidents are avoidable

In the evening we went to Zeche Zollverein – a world cultural heritage site called “the most beautiful coal mine in the world”. We got a guided tour and had dinner in the Kokerei.
It was interesting to see and learn about working conditions – which were really hard. 100 years ago it was common that live expectancy of the workers was less that 60, that there was typically one serious accident per day and that about 30 people died every year in the coal mine.

We find that nowadays inhumane and it would be in Germany (and many other countries) completely unacceptable. Coming back to cars … we accept that in order to have personal transportation it sees unavoidable to have accidents and that 4477 people were killed 2008 in traffic accidents in Germany (which was lower than all the years before). Perhaps in 100 years people will look back at us similar to how we look back at the working conditions in coal mines 100 years ago. And I think research in Automotive User Interfaces can help working towards safer individual traffic.