Talk at the University of New Hampshire, Durham

Andrew Kun invited me to give a talk at the Univeristy of New Hampshire in Durham on my way back from CHI. The talk was on “Embedding Interaction – Human Computer Interaction in the Real World”. In the afternoon I got to see interesting projects in the automotive domain as well as an application on a multi-touch table. At CHI we ran a SIG on Automotive User Interfaces [1].

Seeing the implementation of Project54 live was very exciting. I came across the project first at Pervasive 2005 in Munich [2]. This project is an interesting example of how fast research can become deployed on a large scale.

Andrew chairs together with Susanne Boll the 2nd Int. Conf. on Automotive User Interfaces and Interactive Vehicular Applications – check out the call for papers on! (deadline 2nd of July 2010)

PS: if you ever stay in Durham – here is my favorite hotel: Three Chimneys Inn Durham.

[1] Schmidt, A., Dey, A. K., Kun, A. L., and Spiessl, W. 2010. Automotive user interfaces: human computer interaction in the car. In Proceedings of the 28th of the international Conference Extended Abstracts on Human Factors in Computing Systems (Atlanta, Georgia, USA, April 10 – 15, 2010). CHI EA ’10. ACM, New York, NY, 3177-3180. DOI=

[2] Laslo Turner and Andrew L. Kun, “Evaluating the Project54 speech user interface,” Third International Conference on Pervasive Computing (Adjunct Proceedings), Munich, Germany, May 8-13, 2005

Talk at demola, Finnish Ubicomp program

Jari Ikonen from the Finnish Ubicomp program contacted me last week – interestingly because I shared on this blog the information that I will be in Tampere – and it worked out that we met.

He showed and explained me the demola approach. I find this concept of teaching, training and innovation very exciting. In short demola offers space for students to work on challenging problems that are real and creates opportunities opportunities. Basically companies offer tasks/project to works on. Teams of students (potentially from different universities and fields) will work together to solve it as part of their studies – but the students also will own the IPR. I think that creates interesting teams in realistic settings and has probably a great potential for creating start-ups. Perhaps we should look at this model closer and see how we could create something similar…

As always when meeting interesting people time was too short… I gave an ad-hoc talk based on previous slides on “Mobile & Ubiquitous Computing and Beyond: Mobile Communication changed the world – what else do we need?” and we had a short but very interesting discussion.

Visit to TU Dortmund: Impressive Demos on Vision and Audio

After several tries we finally managed to travel to Dortmund (half an hour on the S-Train) to visit Gernot A. Fink‘s group at the Technical University Dortmund. Bastian Pfleging did with this group his master thesis before he joined us. The research focus of the group is on signal processing and computer vision. They also follow an experimental approach – building systems that work (which we saw in the demos). In their lab space they have setup a building (basically a house inside a house – impressive!).

I have learned about a new location technology based on passive infrared sensors. The idea is to pick heat emitted from people and combine the output from several sensors to localize the person. The technology is very simple, potentially cheap, and privacy preserving. Sometime back we thought of a project topic using thermal imaging (not really cheap or privacy preserving) for context-awarenes – but so far there was no student who wanted to do it. Perhaps we should try again to find a student.

The other demos were situated in a meeting room that is equipped with several cameras and microphones. It was interesting to see how robust several of the vision prototypes managed to track people in the room and to detect pointing actions. One basic mechanism the use to detect interesting regions in an image is saliency based on different features – and it works well.

The audio demo used two arrays of 8 microphones each; the arrays are nicely integrated in a ceiling panel. Using these signals they can calculate the energy that originates from a certain spatial region in the room. Looking at the complexity of the hardware and software for sound localization it appears not in the far future that this could become ubiquitous. We talked about the work James Scott did on sound localization (snipping on a light switch) – here is the reference [1].

The room is equipped with sensors, lights, switches and a UI panel that are linked over a commercial bus system (KNX). Sometime ago we had a bachelor project in Essen that looked at EnOcean (another home networking technology). We discussed how well these systems are positioned in comparison to web technologies.

I personally think medium term we will move – at least on a control and user interface level – to web protocols. The moment you use web protocols it is so much easier to create user interfaces (e.g. using a Web browser as frontend) and it is simple integrate with existing systems (e.g. facebook). It would be interesting to assess how easy it is to use RESTful services to replicate some of the features of home automation systems. Sounds like an interesting project topic. There is a workshop on the Web of Things at PerCom in Mannheim – I am curious what is coming up there.

[1] James Scott, Boris Dragovic: Audio Location: Accurate Low-Cost Location Sensing. Pervasive Computing: Third International Conference, PERVASIVE 2005, Munich, Germany, May 8-13, 2005. Springer LNCS 3468/2005. pp 1-18.

Visiting TU-Berlin and T-Labs

We have a number of student projects that look at novel applications and novel application platforms on mobile phones. As Michael Rohs from T-Labs is also teaching a course on mobile HCI we thought it would be a good opportunity to meet and discuss some application ideas.

I gave a talk in Michael’s lecture discussing the concept of user interfaces beyond the desktop, context as enabling technology, and future applications in mobile, wearable and ubiquitous computing. We had an interesting discussion – and in the end it always comes down to privacy and impact on society. I see this as a very positive development as it shows that the students are not just techies but that they see the bigger picture – and the impact (be it good or bad) they may have with their developments. I mentioned to books that are interesting to read: the transparent society [1] and total recall [2].

In the afternoon we discussed two specific projects. One was an application for informal social while watching TV (based on a set iconic communication elements) that can be used to generate meta data on the program shown. The other is a platform that allows web developers to create distributed mobile applications making use of all the sensors on mobile phones. It is essential a platform an API that provides access to all functions on the phones available in S60 phones over a RESTful API, e.g. you can use a HTTP call to make a photo on someone’s phone. We hope to release some of the software soon.

In the coffee area at T-labs was a printout with the 10+1 innovation principles – could not resist to take a photo 😉 Seems innovation is really trival – just follow the 11 rules and you are there 😉

[1] David Brin. The Transparent Society: Will Technology Force Us to Choose Between Privacy and Freedom. Basic Books. 1999. ISBN-13: 978-0738201443. Amazon-link. Webpage:

[2] Gordon Bell, Jim Gemmell. Total Recall: How the E-Memory Revolution Will Change Everything. Dutton Adult. 2009. ISBN-13: 978-0525951346. Amazon-link. Webpage:

Visit to NEC labs in Heidelberg

In the afternoon I gave a talk at NEC labs in Heidelberg on ubiquitous display networks. Over the last year we developed and number of ideas and prototypes of interactive public display systems. We run a lab class (Fallstudien) on pervasive computing technologies and advertising together with colleagues from marketing. In another class (Projektseminar) we investigated how to facilitate interaction between interactive surfaces (e.g. multi touch table) and mobile devices. One of the prototypes will be shown as poster at mobile HCI 2009 in Bonn. In some thesis projects we introduced the notion of mobile contextual displays and their potential applications in advertising, see [1] and [2].

Seeing the work at NEC and based on the discussion I really think there is a great of potential for ubiquitous display networks – at the same time there are many challenges – including privacy that allways ensures discussion 😉 It would be great to have another bachelor or master thesis to address some of them – perhaps jointly with people from NEC. To understand the information needs in a particular display environment (at the University of Duisburg-Essen) we currently run a survey to better understand requirements. If you read German you are welcome to participate in the survey.

Predicting the future usually features in my talks – and interestingly I go a recommendation from Miquel Martin for a book that takes its own angle on that: Predictably Irrational by Dan Ariely (the stack of book gets slowly to large – time for holidays).

[1] Florian Alt, Albrecht Schmidt, Christoph Evers: Mobile Contextual Displays. Pervasive Advertising Workshop @ Pervasive 2009. Nara, Japan 2009.

[2] Florian Alt, Christoph Evers, Albrecht Schmidt: Users’ View on Car Advertisements. In: Proceedings of the Seventh International Conference on Pervasive Computing, Pervasive’09. Springer Berlin / Heidelberg Nara, Japan 2009.

Visit at Microsoft in Redmond

AJ Brush and John Krumm organize for the people who are in Redmond for the Ubicomp PC meeting a visit to Microsoft. In the morning we got a tour at the home lab – Microsoft’s vision of future home environments – was quite interesting, but had to sign an NDA.
After lunch we went over to Microsoft Research (which is in a new building). We got to see some cool demos. Andy Wilson showed us some new stuff moving the SURFACE forward (physics rocks!). I learned more about depth sensing cameras and Andy showed a fun application [1] – there is video about it, too. Patrick Baudisch talked us through the ideas of LucidTouch [2] and more general about future interaction with small mobile devices. The idea of using the finger behind the screen and the means to increase the precision has many interesting aspects. I found the set of people that work at MSR as impressive as the demos – it seems to be a really exciting work environment.

The atrium of the new building is amazing for playing Frisbee and shoot rubber band missiles. And waiting for the pizza with those toys around proved yet again that researchers are often like kids 😉

[1] Wilson, A. Depth-Sensing Video Cameras for 3D Tangible Tabletop Interaction. Tabletop 2007: The 2nd IEEE International Workshop on Horizontal Interactive Human-Computer Systems, 2007.

[2] Wigdor, D., Forlines, C., Baudisch, P., Barnwell, J., Shen, C. LucidTouch: A See-Through Mobile Device. In Proceedings of UIST 2007, Newport, Rhode Island, October 7-10, 2007, pp. 269–278

Humangrid – are humans easier to program than systems?

In the afternoon I visited humangrid, a startup company in Dortmund. Their basic idea is to create a platform that offers opportunities for crowdsourcing – basically outsourcing small tasks that are easy to perform by humans to a large number of clickworkers. One example for such a scenario is tagging and classification of media. It is interesting that they aim to create a platform that offers real contracts and provides guaranties – which makes it in my eyes more ambitious than Amazon’s Mechanical Turk.

One interesting argument is that programming humans (as intelligent processors) to do a certain task that involves intelligence is easier and cheaper than creating software that does this completely automated. Obviously with software there is nearly zero-cost for performing the tasks – after the software is completed, however if the development costs are extremely high paying a small amount to the human processor for each task may still be cheaper. The idea is a bit like creating a prototype using wizard of oz – and not replacing the wizard in the final version.

In our discussion we developed some idea where pervasive computing and mobile technologies can link to the overall concept of the human grid and crowdsourcing creating opportunities for new services that are currently not possible. One of our students will start next month a master thesis on this idea – I am already curious if we get the idea working.

Visiting the inHaus in Duisburg

This morning we visited the inHaus innovation center in Duisburg (run by Fraunhofer, located on the University campus). The inHaus is a prototype of a smart environment and a pretty unique research, development and experimentation facility in Germany. We got a tour of the house and Torsten Stevens from Fraunhofer IMS showed us some current developments and several demos. Some of the demos reminded me of work we started in Lancaster, but never pushed forward beyond a research prototype, e.g. the load sensing experiments [1], [2].

The inHaus demonstrates impressively the technical feasibility of home automation and the potential of intelligent living spaces. However beyond that I strongly believe that intelligent environments have to move towards the user – embracing more the way people life their lives and providing support for user needs. Together with colleagues from Microsoft Research and Georgia Tech we organize the workshop Pervasive Computing at Home which is held as a part of Pervasive 2008 in Sydney that focuses on this topic.

Currently the market size for smart homes is still small. But looking at technological advances it is not hard to image that some technologies and services will soon move from “a luxury gadget” to “a common tool”. Perhaps wellness, ambient assistive living and home health care are initial areas. In this field we will jointly supervise a thesis project of one of our students over the next month.

Currently most products for smart homes are high quality, premium, high priced, and providing a long lifetime (typically 10 to 20 years). Looking what happened in other markets (e.g. navigation systems, now sold at 150€ retail prices including a GPS unit, maps, touch screen and video player) it seems to me there is definitely an interesting space for non-premium products in the domain of intelligent environments.

[1] Schmidt, A., Strohbach, M., Laerhoven, K. v., Friday, A., and Gellersen, H. 2002. Context Acquisition Based on Load Sensing. In Proceedings of the 4th international Conference on Ubiquitous Computing (Göteborg, Sweden, September 29 – October 01, 2002). G. Borriello and L. E. Holmquist, Eds. Lecture Notes In Computer Science, vol. 2498. Springer-Verlag, London, 333-350.

[2] Albrecht Schmidt, Martin Strohbach, Kristof Van Laerhoven, Hans-Werner Gellersen: Ubiquitous Interaction – Using Surfaces in Everyday Environments as Pointing Devices. User Interfaces for All 2002. Springer LNCS.

Visit to the Arithmeum in Bonn

For people who already arrive on Sunday, the day before the conference, we organised some museum visits: Arithmeum, Haus der Geschichte, Deutsches Museum, and Art Gallery. I only had time to see the Arithmeum ( which was pretty impressive. Hiroshi Ishii (the keynote speaker of the conference) and Brygg Ullmer (last years conference co-chair) joined us, too.

It was unexpected how close the displayed artefacts are to our current research on tangible interaction. We had a very good guided tour by Nina Mertens, who gave us an interesting overview from counting tokens to calculation machines. Some of the exhibit we could even try out our selves.

I found the aspect of aesthetics in some of the calculation aids and machines quite fascinating. Especially the fact that some were so precious that they were not really used for calculating but more for showing off is a concept that is amazing. Similarly interesting was one artefact that was mainly built as a proof that calculation can be automated.