Media art, VIS Excursion to ZKM in Karlsruhe

This afternoon we (over 40 people from VIS and VISUS at the University of Stuttgart) went to Karlsruhe to visit the ZKM. We got guided tours to the panorama laboratory, the historic video laboratory, to the SoundARt exhibition and some parts of the regular exhibition. Additionally Prof. Gunzenhäuser gave a short introduction to the Zuse Z22 that is in on show there, too.

 The ZKM is a leading center for digital and media art that includes a museum for media art and modern art, several research institutes, and an art and design school. The approach is to bring media artists, works of art, research in media art and teaching in this field close together (within a single large building). The exhibitions include major media art works from the last 40 years.

The panorama laboratory is a 360 degree (minus a door) projection. Even though the resolution of the powerwall at VISUS [1] is higher and the presentation is in 3D, the360 degree 10 Megapixel panorama screen results in an exciting immersion. Without 3D, being surrounded by media creates a feeling of being in the middle of something that happens around you. Vivien described the sensation of movement similar to sitting in a train. The moment another train pulls out of the station you have a hard time to tell who is moving. I think such immersive environment could become very common once we will have digital display wallpaper.

The historic video laboratory is concerned with “rescuing” old artistic video material. We sometimes complain about the variety of video codecs, but looking at the many different formats for tapes and cassettes, this problem has a long tradition. Looking at historic split screen videos that were created using analog technologies one appreciates the virtues of digital video editing… Two are two amazing films by Zbigniew Rybczyński: Nowa Książka (New Book): and and Tango:

The current SoundArt exhibition is worthwhile. There are several indoor and outdoor installations on sounds. In the yard there is a monument built of speakers (in analogy to the oracle of Delphi) that you can call from anywhere (+49 721 81001818) and get 3 minutes of time to talk to whom even is in the vicinity of the installation. Another exhibit sonfied electron magnetic fields from different environments in an installation called the cloud.

[1] Powerwall at VISUS at the Univeristy of Stuttgart (6m by 2.20, 88 million pixel in, 44 million pixel per eye for 3D).

Doctoral Colloquium in Bommerholz

For the second time we organize a doctoral seminar for PhD students in CS from Bochum, Dortmund, Duisburg und Essen. The main purpose is to provide networking opportunities beyond the own subject area and to highlight to options for life after the PhD.

This year we had 4 invited speakers highlighting opportunities in academia, industry, and in SMEs:

The main take away message is be open with regard to career choices (bottom line: good students will be good and happy whatever they do after their PhD ;-). However if you are sure that you want to be in the management of a large enterprise than go for a top management consultancy job after your PhD and if you are sure academia in Germany is your only choice than go to one of the top Universities in the US as postdoc. This does not guarantee anything but puts you in the best position…

In the evening the tasks for the teams was to think 100 years ahead! This was inspired by the book “the world in 100 years” [2] and by a talk from Friedemann Mattern [2].

One essential reference on doing a PhD is [3] 🙂

[1] Arthur Brehmer (Herausgeber). Die Welt in 100 Jahren: Mit einer einführenden Essay “Zukunft von gestern” von Georg Ruppelt. Reprint der Auflage von 1910. (2010). ISBN-10: 3487083043.

[2] Friedemann Matter. Die Welt in 100 Jahren – Rückblick auf eine vergangene Zukunft. Kolloquium an der TU Darmstadt. 2006.

[3] PhD Comics.

Visiting TU-Berlin and T-Labs

We have a number of student projects that look at novel applications and novel application platforms on mobile phones. As Michael Rohs from T-Labs is also teaching a course on mobile HCI we thought it would be a good opportunity to meet and discuss some application ideas.

I gave a talk in Michael’s lecture discussing the concept of user interfaces beyond the desktop, context as enabling technology, and future applications in mobile, wearable and ubiquitous computing. We had an interesting discussion – and in the end it always comes down to privacy and impact on society. I see this as a very positive development as it shows that the students are not just techies but that they see the bigger picture – and the impact (be it good or bad) they may have with their developments. I mentioned to books that are interesting to read: the transparent society [1] and total recall [2].

In the afternoon we discussed two specific projects. One was an application for informal social while watching TV (based on a set iconic communication elements) that can be used to generate meta data on the program shown. The other is a platform that allows web developers to create distributed mobile applications making use of all the sensors on mobile phones. It is essential a platform an API that provides access to all functions on the phones available in S60 phones over a RESTful API, e.g. you can use a HTTP call to make a photo on someone’s phone. We hope to release some of the software soon.

In the coffee area at T-labs was a printout with the 10+1 innovation principles – could not resist to take a photo 😉 Seems innovation is really trival – just follow the 11 rules and you are there 😉

[1] David Brin. The Transparent Society: Will Technology Force Us to Choose Between Privacy and Freedom. Basic Books. 1999. ISBN-13: 978-0738201443. Amazon-link. Webpage:

[2] Gordon Bell, Jim Gemmell. Total Recall: How the E-Memory Revolution Will Change Everything. Dutton Adult. 2009. ISBN-13: 978-0525951346. Amazon-link. Webpage:

Seminar on Pervasive Display Networks in Dagstuhl

From the 4th to the 8th of January we (Nigel Davies, Antonio Krüger, Marc Langheinrich, Martin Strohbach, and I) organized a Seminar on Pervasive Display Networks in Dagstuhl. The timining was less not perfect but we go a really good team together.

Working on the topic we identify an interesting set of dimensions to classify public display, collected a large set of scenarios, and compiled a bibliography as well as a reading list. As soon as the material is publicly available I will post it.

The first time I have been to Dagstuhl was in September 2001 (the seminar was on during 9/11) on Ubiquitous Computing ( This previous seminar really helped to create a vibrant community. Perhaps Pervasive Display Networks will grow in a similar way.

Since then I have been many times to Dagstuhl – but this year I was skiing while there for the first time. There is a small “skiing resort” close by: Erbeskopf

PS: if you rent skis it is always good to remember what skis (e.g. color, make, size, etc.) one got 😉
I learned just taking the next pair of skis is sub-optimal…

TPC meetings in Atlanta, Palo Alto and Boston

December was filled with travel – twice to the US and several trips in Europe – and for the first time in two years I did not really get around to write my blog…

I am still wondering what technology we require that could make physical meetings less important. Video conference is getting better and I use it a lot – but it still does not facilitate a discussion between 30 or more people well. Besides the work that is in reviewing I really emjoy that part of my job – I find it really exciting to see so much (somehow) novel work in a very short time.

Academics often complain about a lot of travel – but sometimes we need a reality check. Walking around Atlanta airport and seeing the large number of soldiers I felt that I should not complain about my travels… At the same time I asked myself what we will have first: “remote only” wars or “remote only” critical business meeting.

In the first two weeks of December I had the privilege to be the CHI, Pervasive and PerCom program committee meeting. Having seen more than 100 papers being discussed made the effort in reviewing worthwhile. The overview of the field one gets is amazing. And with the insight view I am looking forward to three very interesting conference programs to come in 2010. By the way Geraldine Fitzpatrick (this years paper co-chair at CHI) has move to the Vienna University of Technology.

Besides the PC meetings there was some time to visit labs. In Palo Alto at Nokia Research we were shown a communication appliance that is designed to facilitate remote interaction and communication around a book. Looks interesting and they promised there will be a paper about this soon.

In Boston I went to the new MediaLab building (and met Leah Buechley and Joe Paradiso) – really exciting – research between boxes. Seeing some a Leah’s work motivated again to look more into wearables. If you are curious too, have a look at Lilypad Arduino [1] and at the 2008 CHI paper [2].

[1] Buechley, L. Lilypad Arduino | build something.

[2] Buechley, L., Eisenberg, M., Catchen, J., and Crockett, A. The Lilypad Arduino: using computational textiles to investigate engagement, aesthetics, and diversity in computer science education. In CHI ’08: Proceedings of the twenty-sixth annual SIGCHI conference on Human factors in computing systems (New York, NY, USA, 2008), ACM, pp. 423-432.

MUM 2009 in Cambridge, no technical solution for privacy

The 8th International Conference on Mobile and Ubiquitous Multimedia (MUM 2009) was held in Cambridge, UK. The conference is fairly specific and had an acceptance rate of about 33% – have a look at the table of content for an overview. Florian Michahelles presented our paper on a design space for ubiquitous product recommendation systems [1]. Our work contributes a comprehensive design space that outlines design options for product recommendation systems using mobile and ubiquitous technologies. We think that over the next years mobile recommendation systems have the potential to change the way we shop in the real world. It probably will be normal to have access in-depth information an price comparison while browsing in physical stores. The idea has been around for a while, e.g. the pocket bargain finder presented at the first ubicomp conference [2]. In Germany we see also a reaction of some electronics stores that asked users NOT to use a phone or camera in the shop.

The keynote on Tuesday morning was by Martin Rieser on the Art of Mobility. He blogs on this topic on
The examples he presented in his keynote concentrated on locative and pervasive media. He characterized locative media as media that by social interaction that is linked to a specific place. He raised the awareness that mapping is very important for our perception of the world, using several different subjective maps – I particular liked the map encoding travel times to London . A further interesting examples was a project by Christian Nold: Bio mapping – emotional mapping of journeys. QR or other bar code markers on cloth (large and on the outside) have a potential … I see this now.

In the afternoon was panel on “Security and Privacy: Is it only a matter of time before a massive loss of personal data or identity theft happens on a smart mobile platform?” with David Cleevely, Tim Kindberg, and Derek McAuley. I found the discussion very inspiring but in the end I doubt more and more that technical solutions will solve the problem. I think it is essential to consider the technological, social and legal framework in which we live. If I would need to live in a house that provides absolute safety (without a social and legal framework) it would be probably not a very nice place… hence I think here we need really interdisciplinary research in this domain.

[1] von Reischach, F., Michahelles, F., and Schmidt, A. 2009. The design space of ubiquitous product recommendation systems. In Proceedings of the 8th international Conference on Mobile and Ubiquitous Multimedia (Cambridge, United Kingdom, November 22 – 25, 2009). MUM ’09. ACM, New York, NY, 1-10. DOI=

[2] Brody, A. B. and Gottsman, E. J. 1999. Pocket Bargain Finder: A Handheld Device for Augmented Commerce. InProceedings of the 1st international Symposium on Handheld and Ubiquitous Computing (Karlsruhe, Germany, September 27 – 29, 1999). H. Gellersen, Ed. Lecture Notes In Computer Science, vol. 1707. Springer-Verlag, London, 44-51.

Keynote by Frits Grotenhuis at AMI-2009

In the opening keynote of AMI 2009 Frits Grotenhuis (who stepped in for Emil Aarts) looked back at the last 10 years ambient intelligence. In his talk he showed a number of examples of devices that Philips created in this time, including iCat, the Entertaible, Ambilight, and medical devices. He discussed briefly the forces in such developments between market-pull and technology-push and it became evident that many products in this domain are more technology push than market-pull.

I liked the reference back to the Electronic Poem at the 1958 Brussels World’s Fair, which must have been at that time an amazing large scale installation (creating “surround sound” with more than 200 speakers).

The update of the vision (title of the keynote was Ambient Intelligence 2.0: Towards Synergetic Prosperity) suggests a model where the human is in the center and the surrounded by the Mind (well being), Community (participation), Body (health), and Environment (responsibility). I found this did not offer many new insights as by now the human centered approach is widely accepted. Looking at the examples it seems that the vision is very much centering on the “rich” world’s problem… perhaps it is really hard to update a vision.

Finishing my term as External Examiner at Trinity College Dublin

Over the last three years I have been regularly to Dublin to act as external examiner for the Ubicomp MSc course. For me this was a good experience to see how serious The School of Computing at Trinity College takes external quality control and how well processes are managed. And besides the administrative part I saw a great many interesting MSc dissertations over the years. Even though the term has come to an end I hope to travel to Dublin in the future too – perhaps on Holiday to see more of the city (which I did not really manage …)

PS: recession seems to have hit Ireland – I have never seen such short queue in Dublin airport – and it is definatly not the selfservice machines that reduced the queue …

Summer party at the chair of ergonomics in Munich

On Friday afternoon I was at the summer party of the chair of ergonomics at the Technical University of Munich. Klaus Bengler, who took over the chair earlier this year and became professor, had in his talk 3 interesting points to take away:

  1. to assess more how much does bad ergonomics costs us (from health to missed sales)
  2. to quantify the value of ergonomics in real money in order to make it comparable with other factors in product design
  3. to include ergonomics as an integral part of the development process

From my Computer Science/HCI perspective I think (2) would be top of the list – as we have good approaches to (3) but need (2) to push it and as (1) is part of (2)… It would be great to have an argument based on economics. E.g. adding tactile feedback will costs x € and it will increase the value of the product by y € – if x>y do it – else don’t … still no idea how one would do this – but would be great if possible!

The researchers and students had prepared a large number of demos – mainly centered on ergonomics in the car, sports ergonomics, and mobile eye-tracking. Hard to pick a favorite but I liked the joystick control for steering a car a lot (there will be a paper on this at the automotive UI conference).

DFG Emmy Noether Meeting in Potsdam, Art, Ceilings

Meeting with other researchers that run or have run Emmy Noether research groups is very different from normal conferences and meetings. The participants are across all disciplines – from art history to zoology 😉 The meeting focuses mainly on strategic, political, personal, administrative and organizational issues when starting a research career. This year we had child care organized during the meeting and Vivien came with me to Potsdam.

On Saturday night I learned that we (our galaxy) will eventually collide with the Andromeda Galaxy (but after our sun is out fuel – so I do not worry too much). Vivien found this fascinating, too. Learning more about astrophysics (looks defiantly more complicated than the things I usually do) teaches me to worry less about the immediate usefulness and direct utility of research results – also in our domain.

I am fascinated how different research can be and at the same time how similar the enthusiasm is people have for their research. By now – being one of the old guys – I co-organized two workshops. One together with Dr. Hellfeier from DHV on how to negotiate for a professorship and one with Stefanie Scheu and Rainer Hirsch-Luipold on teaching and PhD-supervision.

I talked to Riko Jacob (CS at TU Munich) about teaching computer science in school and he showed me a picture of a tangible shortest path calculator (I took a photo of the photo ;-). Perhaps I have at some point time to play with the installation in Munich.

On Sunday morning we took the water taxi – direct from the hotel peer – to the central train station in Potsdam. Christian Scholl from Göttingen (he does Art History) took some time to show us around the castle Sans Souci. After our discussion I wondered if we should consider a joint seminar from computer science/media informatics and art history – in particular ideas related to ambient media, interactive facades, and robotic buildings would benefit from a more historic awareness. There is an interesting PhD thesis on ceiling displays [1] – for a shorter version see [2]. I met Martin Tomitsch at a Ubicomp DC and I was impressed with the idea and its grounding in history.

[1] Tomitsch M. (2008). Interactive Ceiling – Ambient Information Display for Architectural Environments. PhD Thesis, Vienna University of Technology, Austria.

[2] Tomitsch, M., Grechenig, T., Vande Moere, A. & Sheldon, R. (2008). Information Sky: Exploring Ceiling-based Data Representations. International Conference on Information Visualisation (IV08), London, UK, 100-105.

SEGA World – relaxing after the conference :-)

On the way back from the PC-dinner we needed to get an update on another aspect of Japanese technologies and so we went into SEGA World in Nara.

Many of the games are very similar to other toys around the world – shooter, sports games and racing games. Each time you use games in such a setting one is reminded of the power a physical controls and the concept of tangible interaction…

The photo maker however was very different from what I have seen before. Technically it is interesting and well engineered: you make photos in a well lit area, it removes the background, and then you can choose background, borders, frames etc. Marc’s Japanese helped us to get our pictures out of the machine – with more time an more Japanese reading skill we could have manipulated our pictures some more. It was interesting that the machine offered two options for output: paper and transfer to your mobile phone.

PS: remember not to play basketball against James and not to race against Antonio 😉

Some Interesting Papers and random Photos from Pervasive 2009

Pervasive 2009 had a really exciting program and provided a good overview of current research in pervasive and ubiquitous computing. Have a look at the proceedings of the pervasive 2009 conference. The Noh theater in Nara was a very special and enjoyable venue and it was organized perfectly – as one would expect when travelling to Japan.

The idea of having short and long papers together in the main track worked very well in my view. The number of demos and posters was much higher than in the years before – and that was great and very inspiring. Have a look at the photos for some of the posters and demos.
The program consisted of 20 full papers (18 pages) and 7 notes (8 pages) which were selected in a peer review process out of 147 submissions (113 full papers, 34 notes) which is a acceptance rate of 18%.

John Krumm presented his paper Realistic driving tips for location privacy – again having a good idea making the presentation interesting beyond its content (having review snippets in the footer of the slides – including a fake review). The paper explores the difficulties that arise when creating fake GPS tracks. He motivated that the probabilities need to be taken into account (e.g. you are usually on a road). I liked the approach and the paper is worthwhile to read. I think it could be interesting to compare the approach is not create the tracks but just share them between users (e.g. other people can use parts of my track as fake track and in return I get some tracks that I can use as fake tracks).

If you phone knows where you are you can use this information to control your heating system. This was the basic idea of the research presented by Stephen Intille. They explored using GPS location of the users to automate control of the heating / air condition control in a house. It seems there is quite some potential for saving energy with technology typically used in the US (one temperature control for the whole house). In Europe where heating systems typically offer finer control (e.g. room level) the potential is probably larger.

James Scott presented a paper that showed how you can use force gestures to interact with a device. In contrast to previous research (e.g. GUMMI) the approach works with a ridged device and could be used with current screen technologies.

What do you need to figure out who is holding and using the remote control? This question is addressed in the paper “Inferring Identity Using Accelerometers in Television Remote Controls” that was presented by Jeff Hightower. They looked at how well button press sequences and accelerometer data give you information about which person is using the device.

Geo-fencing: confining Wi-Fi Coverage to Physical Boundaries is an example of how to create technological solutions to fit a user’s conceptual model of the world. As people have experience with the physical world and they have mechanisms to negotiate and use space and hence linking technologies that have typically other characteristics (e.g. wireless radio coverage) to the known concept is really interesting.

Situvis, a tool for visualizing sensor data, was presented by Adrian Clear from Aaron’s group in Dublin. The software, papers and a video is available at: The basic idea is to have a parallel coordinate visualization of the different sensor information and to provide interaction mechanisms with the data.

Nathan Eagle presented the paper “Methodologies for continuous cellular tower data analysis”. He talked about the opportunities that arise when we have massive amounts of information from users – e.g. tracks from 200 million mobile phone user. It really is interesting that based on such methods we may get completely new insights into human behavior and social processes.

If you have seen a further interesting paper in the conference (and there are surely some) that I have missed feel free to give a link to them in the comments to this post.

Workshop on Pervasive Computing in Advertising

We got a good set of submission for our workshop and had about 20 participants who joined us in Nara to discuss how pervasive computing will shape advertising in the future. The papers and a selection of talks is online on the workshop website:

One question that was central to our discussion was: what is advertising and how is it different from information. It became quickly clear that there is a lot of information that has an influence on behavior and in particular shopping decisions and some of it is considered advertising but much is not. Hence it seems really interesting to imagine a world where advertising is replaced by information. One could image that replacing advertising by information (e.g. as it happens already in some domains such a hotel recommendations) would change the whole approach for creating product or providing services.

We have presented in the workshop our work on contextual mobile displays. The idea is that in the future we could have mobile displays (that replace current printed items, like bumper stickers, bags with printed logos, and t-shirts with prints) could become active and could act as contextual displays. Have a look at the paper for more details [1].

[1] Florian Alt, Albrecht Schmidt, Christoph Evers. Mobile Contextual display system. Pervasive Advertising Workshop at Pervasive 2009. (contact Florian Alt for a copy of the paper)

Japan – sightseeing (an less phone usage than expected)

To get cheaper flights we took a flight on Thursday/Friday to fly from Europe to Japan (never really understood the pricing model of flights). So we had two days off before the actual conference and many colleagues (who also took cheap flights) were also there. We went to do some sightseeing in Nara and Kyoto – which was great.

In Kyoto we got personal guides – students from a University in Kyoto – who offered to show us run and use this to practice their English. It was great for us as we got many insights we would have missed by ourselves and it was great to talk to some locals. Hopefully they enjoyed their time with us, too. In the evening we learned once more that the Japanese people are very social; we met the Nara Air Rescue team in a restaurant – and this was proof :-).

One thing that surprised me greatly was that very few people in Nara and Kyoto used their phone in public. On the train nearly nobody spoke on the phone, watch mobile TV or browsed the web. This is obviously very different from Tokyo. Overall Nara and Kyoto are very enjoyable and calming places. I hope to have at some point the time to spend more time in Japan (… when is my next sabbatical? 😉

For more photos see:

PS: an some people find a disco in the street…

Meeting on public display networks

Sunday night I travelled to Lugano for a meeting public display networks. I figured out that going there by night train is the best option – leaving midnight in Karlsruhe and arriving at 6am there. As I planned to sleep all the time my assumption was that the felt travel time would be zero. Made my plan without the rail company… the train was 2 hours late and I walked up and down for 2 hours in Karlsruhe at the track – and interestingly the problem would have been less annoying if public displays would provide the relevant information … The most annoying thing was passengers had no information if or when the train will come and no one could tell (neither was anyone at the station nor was anyone taking calls at the hotline).
The public display – really nice state of the art hardware – showed for 1 hour nothing and that it showed that the train is one hour late (was already more than 1 hour after the scheduled time) and finally the train arrived 2 hours late (the display still showing 1 hour delay). How hard can it be to provide this information? It seems with current approaches it is too hard…

On my way back I could observe a further example of short comings with content on public display. In the bus office they had a really nice 40-50 inch screen showing teletext of the departure. The problem was it was the teletext for the evening as the staff has to manually switch the pages. Here too it is very clear the information is available but current delivery systems are not well integrated.

In summary it is really a pity how poorly the public display infrastructures are used. It seems there are a lot of advances in the hardware but little on the content delivery, software and system side.

Visit to Newcastle University, digital jewelry

I went to see Chris Kray at Culture Lab at Newcastle University. Over the next months we will be working on a joined project on a new approach to creating and building interactive appliances. I am looking forward to spending some more time in Newcastle.

Chris showed me around their lab and I was truly impressed. Besides many interesting prototypes in various domains I have not seen this number of different ideas and implementations of table top systems and user interface in another place. For picture of me in the lab trying out a special vehicle see Chris’ blog.

Jayne Wallace showed me some of her digital jewelry. A few years back she wrote a very intersting article with the title “all the useless beauty” [1] that provides an interesting perspective on design and suggests beauty as a material in digital design. The approach she takes it to design deliberately for a single individual. The design fits their personality and their context. She created a communication device to connect two people in a very simple and yet powerful way [2]. A further example is a piece of jewelry that makes the environment change to provide some personal information – technically it is similar to the work we have started with encoding interest in the Bluetooth friendly names of phones [3] but her artefacts are much more pretty and emotionally exciting.

[1] Wallace, J. and Press, M. (2004) All this useless beauty The Design Journal Volume 7 Issue 2 (PDF)

[2] Jayne Wallace. Journeys. Intergeneration Project.

[3] Kern, D., Harding, M., Storz, O., Davis, N., and Schmidt, A. 2008. Shaping how advertisers see me: user views on implicit and explicit profile capture. In CHI ’08 Extended Abstracts on Human Factors in Computing Systems (Florence, Italy, April 05 – 10, 2008). CHI ’08. ACM, New York, NY, 3363-3368. DOI=

Mobile Boarding Pass, the whole process matters

Yesterday night I did an online check-in for my flight from Düsseldorf to Manchester. For convenience and curiosity I chose the mobile boarding pass. It is amazingly easy and it worked in principle very well. Only not everyone can work without paper yet. At some point in the process (after border control) I got a hand written “boarding pass” because this person needs to stamp it 😉 and we would probably have gotten into an argument if he tried to stamp my phone. There is some further room for improvement. The boarding pass shows besides the 2D barcode all the important information for the traveler – but you have to scroll to the bottom of the page to get the boarding number (which seems quite important for everyone else than the traveler – it was even on my handwritten boarding pass).

Visit to Nokia Research Center Tampere, SMS, Physiological sensors

This trip was my first time in Tampere (nice to see sometimes a new place). After arriving yesterday night I got a quick cultural refresher course. I even met a person who was giving today a presentation to the president of Kazakhstan (and someone made a copy using a phone – hope he got back OK to Helsinki after the great time in the bar).

In the morning I met a number of people in Jonna Hakkila’s group at the Nokia Research Center. The team has a great mix of backgrounds and it was really interesting to discuss the project, ranging from new UI concepts to new hardware platform – just half a days is much too short… When Ari was recently visiting us in Essen he and Ali started to implement a small piece of software that (hopefull) improves the experience when receiving an SMS (to Ali/Ari – the TODOs for the Beta-release we identified are: sound design, screen design with statistics and the exit button in the menu, recognizing Ok and oK, autostart on reboot, volume level controlable and respecting silent mode). In case you have not helped us with our research yet please fill in the questionnaire:

I gave a talk (see separate post on the next big thing) and had the chance to meet Jari Kangas. We discovered some common interest in using physiological sensing in the user interface context. I think the next steps in integrating physiological sensors into devices are smaller than expected. My expectation is that we rather detect simple events like “surprise” rather than complex emotion (at least in the very near future). We will see where it goes – perhaps we should put some more students on the topic…

Andreas Riener defends his PhD in Linz

After a stop-over in Stansted/Cambridge at the TEI conference I was today in Linz, Austria, as external for the PhD defense of Andreas Riener. He did his PhD with Alois Ferscha and worked on implicit interaction in the car. The set and size of experiments he did is impressive and he has two central results. (1) using tactile output in the car can really improve the car to driver communication and reduce reaction time. And (2) by sensing the force pattern a body creates on the seat driving relates activities can be detected and to some extend driver identification can be performed. For more details it makes sense to have a look into the thesis 😉 If you mail Andreas he will probably sent you the PDF…
One of the basic assumptions of the work was to use implicit interaction (on input and output) to lower the cognitive load while driving – which is defiantly a valid approach. Recently however we also discussed more the issues that arise when the cognitive load for drivers is to low (e.g. due to assistive systems in the car such as ACC and lane keeping assistance). There is an interesting phenomenon, the Yerkes-Dobson Law (see [1]), that provides the foundation for this. Basically as the car provides more sophisticated functionality and requires less attention of the user the risk increase as the basic activation of the driver is lower. Here I think looking into multimodality to activate the user more quickly in situations where the driver is required to take over responsibility could be interesting – perhaps we find a student interested in this topic.
[1] (there is a link to the 1908 publication by Yerkes, & Dodson)

Demo day at TEI in Cambridge

What is a simple and cheap way to get from Saarbrücken to Linz? It’s not really obvious, but going via Stansted/Cambridge makes sense – especially when there is the conference on Tangible and Embedded Interaction ( and Raynair offers 10€ flight (not sure about sustainability though). Sustainability, from a different perspective was also at the center of the Monday Keynote by Tom Igeo which I missed.

Nicolas and Sharam did a great job and the choice to do a full day of demos worked out great. The large set of interactive demos presented captures and communicates a lot of the spirit of the community. To get an overview of the demos one has to read through the proceedings (will post a link as soon as they are online in the ACM-DL) as there are too many to discuss them here.
Nevertheless here is my random pick:
One big topic is tangible interaction on surfaces. Several examples showed how interactive surfaces can be combined with physical artifacts to make interaction more graspable. Jan Borcher’s group showed a table with passive controls that are recognized when placed on the table and they provide tangible means for interaction (e.g. keyboard keys, knobs, etc.). An interesting effect is that the labeling of the controls can be done dynamically.
Microsoft research showed an impressive novel table top display that allows two images to be projected – on the interactive surface and one on the objects above [1]. It was presented at large year’s UIST but I have tried it out now for the first time – and it is a stunning effect. Have a look at the paper (and before you read the details make a guess how it is implemented – at the demo most people guessed wrong 😉
Embedding sensing into artifacts to create a digital representation has always been a topic in tangible – even back to the early work of Hiroshi Ishii on Triangles [2]. One interesting example in this year’s demo was a set of cardboard pieces that are held together by hinges. Each hinge is technically realized as a potentiometer and by measuring the potion the structure can be determined. It is really interesting to think this further.
Conferences like TEI let you inevitably think about the feasibility of programmable matter – and there is ongoing work in this in the robotics community. The idea is to create micro-robots that can create arbitrary shapes – for a starting point see the work at CMU on Claytronics.
[1] Izadi, S., Hodges, S., Taylor, S., Rosenfeld, D., Villar, N., Butler, A., and Westhues, J. 2008. Going beyond the display: a surface technology with an electronically switchable diffuser. In Proceedings of the 21st Annual ACM Symposium on User interface Software and Technology (Monterey, CA, USA, October 19 – 22, 2008). UIST ’08. ACM, New York, NY, 269-278. DOI=
[2] Gorbet, M. G., Orth, M., and Ishii, H. 1998. Triangles: tangible interface for manipulation and exploration of digital information topography. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Los Angeles, California, United States, April 18 – 23, 1998). C. Karat, A. Lund, J. Coutaz, and J. Karat, Eds. Conference on Human Factors in Computing Systems. ACM Press/Addison-Wesley Publishing Co., New York, NY, 49-56. DOI=

Voice interaction – Perhaps it works …

Today we visited Christian Müller at DFKI in Saarbrücken. He organized a workshop on Automotive User Interfaces at IUI last week. My talk was on new directions for user interfaces and in particular arguing for a broad view on multimodality. We showed some of our recent projects on car user interfaces. Dagmar gave a short overview of CARS our simulator for evaluating driving performance and driver distractions and we discussed options for potential extensions and shortcomings of the Lane Change Task.
Being a long time skeptic about voice interfaces I was surprise to see a convincing demo of a multimodal user interface combining voice and a tactile controller in the car. I think this could be really an interesting option for future interfaces. 
Classical voice-only interfaces usually lack basic properties of modern interactive systems, e.g. as stated in Shneiderman’s Golden Rules or in Norman’s action cycle. In particular the following points are most often not well realized in voice-only system:
  • State of the system is always visible
  • Interactions with the system provide immediate and appropriate feedback
  • Actions are easily reversible
  • Opportunities for interaction are always visible 
By combing a physical controller with voice and having at the same time the objects of interaction visible to the user (as part of the physical system that is controlled, e.g. window, seat) these problems are addressed in a very interesting way. I am looking forward to seeing more along these lines – perhaps we should also not longer ignore speech interaction in our projects 😉 

Towards interaction that is begreifbar

Since last year we have in Germany a working group on graspable/tangible interaction in mixed realities.
In German the key term we use is “begreifbar” or “begreifen” which has the meaning of acquire a deep understanding of something and the words basic meaning is to touch. Basically understand by touching – but in a more fundamental sense than grasping or getting grip. Hence the list of translations for “begreifen” given in the dictionary is quite long.
Perhaps we should push more for the word in the international community – Towards interaction that is begreifbar (English has too few foreign terms anyway 😉

This meeting was organized by Reto Wettach at Potsdam and the objective was to have two days to invent things together. The mix of people mainly included people from computer science and design. It is always amazing how many ideas come up if you put 25 people for a day in a room 🙂 We followed this week up on some of the ideas related to new means for communication – there are defiantly interesting student projects on this topic.

In the evening we had a half pecha-kucha (each person 10 slides of 20 seconds – in total 3:20, the original is 20 slides) It is a great way of getting quickly to know about work, research, ideas, and background of other people. It could be format we could use more in teaching a perhaps for ad-hoc sessions at a new conference we plan (e.g. … prepared my slides on the train in the morning – and it is more challenging that expected to get a set of meaningful pictures together for 10 slides.

Overall the workshop showed that there is a significant interest and expertise in Germany moving from software ergonomics to modern human computer interaction.
There is a new person on our team (starting next week) – perhaps you can spot him on the pics.
For a set of pictures see my photo gallery and the photos on flickr.

Mechanical Computing, Beauty of Calculating Machines

Instead of covering the history of calculating machines in the DSD lecture, we took the train and went to the Arithmeum in Bonn to the see the artefacts live and to play with some of them.
We started with early means for counting and record keeping. The tokens and early writings did not use numbers as abstract concepts, rather as representatives of concrete objects – this is very inspiring, especially from a tangible interaction point of view. The knots, as used in south America, show impressively how the tools for calculation have to fit the context people live in. Interestingly all these artefacts highliht how the ability to calculate and store information is related to the ability to do trade – quite a good motivation for the setup we have in Essen business studies and computer science within one faculty.
I was again impressed by the ingenuity by the early inventers of calculating tools and machines. There is an interesting separation between calculating tools and machines – the first ones require the user to take care of the carry and the second do it by themselves. We tried out replicas of Napier’s calculating tool and Schickard’s calculating machine.
The beauty and the mechanical precision required of those early machines is impressive. These prototypes (most of them took years and massive funds to be built complete) can teach us something for research today. These inventors had visions and the will to get it implemented, even without a clear application or business model in mind. They were excited by the creating of systems than can do things, machines could not do before. From the professions of the inventors (e.g. Philipp Matthäus Hahn was a clergyman)  it becomes apparent that at these times some considered religion and calculation as closely related – which to mondern understanding is very very alien.

Seeing the Hollerith machine that was used for the US census more than 100 years ago can teach you a lot about data processing. Punch cards, electrical reading and electrical counters (using mainly relays) were the basis for this technology. Looking at the labels on the counters showed that the US has a long tradition in collecting data that is after some time is not seen as political correct 😉
Having learned binary calculations during the DSD course it was nice to see a machine that did binary additions, using small steel balls and gravity. On each place (1,2,4,8, …) there is space for one ball. If a second one comes to this place one moves up to the next place (carry) and one is discarded. This is implemented with very simple mechanics and the working prototype (recently build) is based on designs of Schickard (but he never built – if I am correct).
Moving on with binary systems and finally to silicon, we got to see the Busicom 141 – a desk calculator that uses the Intel 4004. It is impressive to see that this is not even 40 years ago – starting with 2300 transistors and 180kHz. 
you can find the full set of photos at:

Biometrics will come, who will care about privacy

Arriving at the new Terminal 5 at Heathrow airport I saw some extra installations (and extra lines) for iris scan immigration. Arriving at 4 am in the morning they were closed and there were not queues – but I could see that it is very attractive at other times of day when queues are long. On the official website they claim that border control will be down to 20 seconds. There is a more detailed document on the schema – I saved the document to have it in 10 years when we will have a very different view on privacy.

Will we have face-2-face PC meetings in the future?

On Thursday morning I flew to Boston for the CHI 2009 PC meeting. The review and selection process was organized very professional and efficient. We discussed all papers in one and a half days – and I think an interesting program came out and I learned a lot about what values my colleagues see or see not in papers. On Friday afternoon I flew back to the UK for the Pervasive 2009 PC meeting in Cambridge (with the same crew on the plane).

Nevertheless the question remains how sustainable is it that 100 people fly to a face-2-face meeting. In what way could we do such a meeting remotely? Video conferencing still does not really work well for larger group discussions (just collecting experience here in Cambridge during the Pervasive PC meeting)… Can it be so difficult to make a reasonable video link between two meeting room? How could we recreate the social aspects (like a joined dinner or walking back through the city with Gregory) as well as side conversations in the meetings? We probably should try harder – It cannot be that difficult – there have been massive amounts of work in CSCW research? Perhaps we should try linking two rooms at different universities as a group project next term? 

Exporting your cars information to the mobile phone

In our user interface engineering class one of the tasks in the exercise is to create a concept design for providing information from the car on the mobile phone (e.g. millage, amount a fuel in the car, next service date, alram status, etc). The first part is to assess what information could be made accessible and what value it would create for the user. 
Today I came across a device (Tyredog TD-1000A) that is concerned with a one sub-part of this scenario: checking your pressure in the tires of the car. It is a simple sensor system, screwed on to each of the tires, connected to a wireless receiver. There is also a version that includes features for the car alarm (Tyredog TD-3000A).
Another group is looking yet again into the domain of  restaurant finders or more general night life. Apropos restaurant finders, Saturday night we got out of the subway onto union square and discussed where to go for dinner (an we probably looked disoriented). A local lady stoped and recommended the Union Square Café – and it was just great… sometimes just talking to someone in the street may provide you with an excellent alternative to technologies 😉 Perhaps the students find a solution that can reflect personal recommendations well…

Information vs. Mobility, Percom PC meeting in New York

The PC meeting for Percom 2009 took place at IBM in Hawthorne, NY. Percom had about 200 submissions and many good ones – so we could compile an exciting program across the whole field of pervasive computing and communication. As one of three program vice chairs I have looked in detail in about 1/3 of the submissions that were application related. It is interesting to observe that research as a whole in the field becomes more major and at the same time more incremental. 

To me this puts up the big question in which domains will the new big innovations happen, what is the next trend after we have pervasive computing? There are luckily plenty of options, but at the moment it seems that there develops an interesting relationship between information, communication, mobility and energy. It seems that we can compensate mobility by information and communication and similarly we can reduce energy required by information available. One example is: if I know where things are (=information) I can reduce the effort required to find them (=mobility). Is there more to it?
Each time in the US – even in New York were public transport works quite well – one is surprise how alien it appears to many that it could be an option to take public transport on a business trip (e.g. there are no first class coaches on regional trains). Flying from Düsseldorf into Newark it was convenient to take the train to Penn Station in NY City and then an express train to White Plaines. If we would not have gone for a walk in the city we probably would have been equally fast as by car. With the again low gas prices in the US (less than 2U$ per gallon, down from 4 just a few month ago) I would expect public transport and small cars will not gain too much popularity – before the next rise in gas prices.

PS: it is amazing how many possiblities there are to serve coffee (and this is probably not one of the most environment friendly)

My first hotel fire alarm, debugging smart environments

We arrived in the evening in Köln and went to our hotel and around 10:30 pm the fire alarm sounded (really loud – you want to leave) and a voice over the speaker system asked us to leave the hotel immediately. When we checked in an hour earlier we overheard that they called for the elevator repair man…

Better safe than sorry I packed up my laptop and rucksack and we went downstairs. At the reception they were pretty busy – but it seemed everyone clear that this a false alarm but it seemed they had no way of really understanding why the system behaved in this way [gap of evaluation 😉 teaching user interface engineering this term]. The error search reminded me on one error search strategy in C (if you do not have a debugger). Comment out part of the code (here: disable fire sensing for certain areas in the hotel) till you can tell which parts cases the error. If you have found this part and it is not essential just leave it as a comment (you can do the same with fire sensor – hope they did not…)

A fire alarm system has compared to smart environments we envision a very low complexity. I think providing appropriate means for debugging smart environments by end-users could be a topic worthwhile to look at.
PS: the elivator had the best display for showing the level you are in I have seen so far. From a UI perspective it is really a boring recreation of the non-digital version… 

Trip to Dublin, Aaron’s Display Project

Visiting Dublin is always a pleasure – even if the weather is rainy. Most of the day I was at Trinity College reading master theses (which is the second best part of being external examiner, best part is to have lunch at the 1592 😉
In the evening I met with Aaron Quigley and we talked about some ongoing display and advertsing projects in our groups. He told me about one of their recent workshop papers [1] on public displays where they investigated what people take in and what people remember of the content on displays in an academic environment. It is online available in the workshop proceedings of AIS08 [2]. I found it worthwhile to browse the whole workshop proceedings.
[1] Rashid U. and Quigley A., “Ambient Displays in Academic Settings: Avoiding their Underutilization”, Ambient Information Systems Workshop at UbiComp 2008, September 21, Seoul, South Korea (download [2], see page 26 ff)

Which way did you fly to Korea?

We got a new USB GPS tracker(from Mobile Action, GT100) and had to try it out on the trip to Korea. It worked very well compared to the other devices we had so far. It got the bus trip in Düsseldorf airport right and the entire flight from Amsterdam to Seoul. Tracking worked well in the taxi from the Airport to the hotel. While walking in downtown Seoul it still performed OK (given the urban canyons) with some outliers.

It did not get any signal while we were on the Fokker-50 from Düsseldorf to Amsterdam 🙁 I slept a few hours on the flight to Seoul but I think someone took a photo (probably of me) over Mongolia… If you wonder if it is allowed to used your GPS in the plane or not – it is – at least with KLM (according to a random website 🙂