Multi-touch displays seen as great opportunity

In our course “case studies in pervasive computing” we re-build systems that are described in the research literature and try to improve them or to apply them to new domains. This term the topic is on multi-touch displays. Starting out with Jeff Han’s paper “Low-cost multi-touch sensing through frustrated total internal reflection” we think about novel interaction methods for large screens. And after some initial problems (filter in the web cam we used) we have a first hardware prototype that shows the FTIR effect. The page of Thomas M. Brand is a good starting point if you too think of a DIY-project.

On Monday Giulio Jacucci from the Helsinki Institute for Information Technology told me that they set up a start-up that has an new interesting and different way of doing multi-touch displays – and they look into enormous sizes of displays, up to 16 meters long. Their web page is http://www.multitouch.fi/. Perhaps we should try to get some of there technology for next term – would save us some serious drilling, soldering and polishing…

Slowly settling in

Time is flying. We are nearly one month in Essen. Teaching started well and quite a number of students came to our course (see http://www.pervasive.wiwi.uni-due.de/en/teaching/). We have a reliable Internet connection and more furniture than we need 😉 Starting from scratch is a great experience – especially as everybody was very helpful.

Nevertheless it takes a lot of time and effort. I am extremly happy that I was able to start with a real team (Dagmar and Paul)!

Guest course at the University of Linz, MSc Pervasive Computing

I am teaching a guest course at the University of Linz in the Pervasive Computing master program. The topic is Unconventional User Interaction – User Interfaces in a Pervasive Computing World (http://www.ubicomp.net/uui). Today we started with an introduction to motivate how pervasive computing changes human computer interaction. I am already looking forward to the projects!

At dinner I learned why you can never have enough forks in a good restaurant. In case you loose your pen for the mobile phone a fork will do… The topic of the lecture is everywhere!

CardioViz Demo at Ubicomp 2007

Alireza Sahami presented our CardiViz project at the demo session at Ubicomp. We were very happy that the project that was the result of our IPEC course on developing mobile applications was accepted as a demo.

For more details see:
Alireza Sahami Shirazi, Diana Cheng, Oliver Kroell, Dagmar Kern, Albrecht Schmidt. CardioViz: Contextual Capture and Visualization for Long-term ECG Data. Adjunct Proceedings of Ubicomp 2007 (Demo).

Jonna Häkkilä, Anind Dey, Kari Hjelt, and I organized organized the Ubiwell workshop (Interaction with Ubiquitous Wellness and Healthcare Applications) at this years pervasive. Alireza presented another paper on heartbeat monitoring there:
Florian Alt, Alireza Sahami Shirazi, Albrecht Schmidt. Monitoring Heartbeat per Day to Motivate Increasing Physical Activity. UbiWell workshop@Ubicomp 2007.

Schools history, tangibles again

After my daughter started school on Saturday we visited a historic school on Sunday. Comparing teaching materials is interesting. Especially providing up to date information in geography must haven quite a costly task. Many expensive charts and maps that were printed on canvas are now freely available in digital form. It seems that instead of having a film project, a slight projector, an overhead projector and canvas displays a computer and projector with internet access will do. Similarly having stamps to reproduce maps seems like ancient history, even though it has been still in use 20 years ago.

However I wonder what we loose by make things digital and whether or not this matters. Having a database (a box with cardboard dividers and a lot of paper slips) or a typewriter (with types that are moved by pressing buttons) on your desktop gives you a very immediate impression how things work. It is remarkable to see that historically tangibility of teaching materials was very common.

I think in the digital we should make more effort to provide means that people can understand the mechanism behind the technology (basics of HCI – conceptual models :-). This is however extremely difficult for purely digital products. My generation seems very lucky to have been witnesses of this transformation for many products from the physical to the digital – providing a lot of insight to us.

Great tutorial on eXtreme Programming/Agile Methods

Today Karl-Heinz Sylla and Reinhard Budde (both senior researcher at Fraunhofer IAIS) gave for the summer research project a tutorial on agile methods for software engineering. The experience they have from large scale real world projects is impressive! We looked at different approaches to software construction and discussed the pros and cons. Short iterations, user stories, pair programming and test driven development seem to fit very well to our work approach and project goals. A good starting point for more on the topic in particular with a teaching perspective are the following 2 papers: LeJeune, N. F. 2006. Teaching software engineering practices with Extreme Programming. J. Comput. Small Coll. 21, 3 (Feb. 2006), 107-117 and Schneider, J. and Johnston, L. 2003. eXtreme Programming at universities: an educational perspective. In Proceedings of the 25th international Conference on Software Engineering (Portland, Oregon, May 03 – 10, 2003). International Conference on Software Engineering. IEEE Computer Society, Washington, DC, 594-599.

From a user interface engineering perspective is very positive that agile methods are good to integrate with user centred design – in my experience much better than traditional software construction processes. Especially the fact that XP (eXtreme Programming) is open to change in functionally throughout the process is a key.

In this summer research project one great challenge is that the students have to build up knowledge in various areas (e.g. search technologies, web technology, user interfaces) while creating high quality code. There is a very interesting paper that discusses software engineering issues in the context of web applications: Jazayeri, M. 2007. Some Trends in Web Application Development. In 2007 Future of Software Engineering (May 23 – 25, 2007). International Conference on Software Engineering. IEEE Computer Society, Washington, DC, 199-213.

Always when the discussion comes to programming languages a debate on strong typing starts. Especially in the web context this seems come up again and again…

the count down started – about 5 weeks to the prototype

Yesterday our summer project started at IAIS. The students are highly motivated and the combined skill set of the participants is impressive. We discussed a lot what we want to achieve over the next weeks.

Creating a new special purpose search service – basically from the rough idea to a working prototype – in 5 weeks seems a bit crazy but I am confident that we get there 😉 In certain areas we already have an idea how much pages we have to crawl and how much content we have to analyze.

It is interesting that it already now becomes apparent that user interface issues and system architecture decisions are closely linked. E.g. doing a meta search while the user is waiting requires some other content that we can present while the user is expecting the results.

Ubiquitous, Pervasive and Ambient Computing – Clarification of Terms

In the resent month the question about ubiquitous, pervasive, ambient computing came up several times. An email by Jos Van Esbroeck motivated me to write my view on it…

Clarifying the terms seems an ongoing process as various communities and individuals use each of those terms for new things they are doing.

For me the best way to discriminate the terms ubiquitous computing, pervasive computing, and ambient intelligence is by their origin, history and research communities.

The first term (ubiquitous computing, ubicomp) is linked to Mark Weiser and his vision of computing in the 21st century [1]. In the research community its is very much connected to ubiquitous and pervasive systems that have the user somewhere in the loop. The ubicomp conference [2] seems more focused on user experience than on pure technology.

Pervasive Computing was pushed in the mid 1990s, more by industry and in particular by IBM. Pervasive computing seems from its origin more focused on technologies and solutions than on a particular vision. The two major conferences related to this topic: pervasive [3] and percom [4] are more systems and network focused, however always keeping some attention to the user experience perspective. Here, in particular with percom, many in the research community have their origin in the networking and distributed systems world. To me pervasive computing seems more technical than ubiquitous computing and includes systems that do no have direct human users involved.

The term ambient intelligence was introduced by the European funding agencies in the Framework 5 vision. Around the same time as the Philips Home-lab that drives the term, too. Here, similar to ubicomp, the vision of a new quality of user experience is a driving factor. The research that falls under this label by now is broad and I think it is very similar to the research in ubiquious computing. There is also a European conference on ambient intelligence [5].

Many people that are involved in ubicomp/pervasive/percom are also active in one more traditional research community. In particular many people are additionally involved in user interface research (e.g. CHI-Community), mobile computing and mobile systems, networking and distributed systems.

A very early topic related to the whole field is context-awareness as introduced by Schilit [6] who was working with Weiser. In my PhD dissertation I have looked more into the relationship between ubicomp and context-awareness – it has the title Ubiquitous Computing – Computing in Context [7]

In parallel subtopic in the above field have emerged that look at specific aspects, e.g. internet of things [8] (not necessarily a human in the loop), wearable computing (computing in cloth), smart environments (computing in buildings and furniture), tangible and embedded interaction [9] (looking at the interaction side), smart objects, … and probably many more.

There is also an interesting trend that many of the topics, if they are a bit matured, move back into the traditional communities.

[1] Mark Weiser. The Computer for the Twenty-First Century. Scientific American 265, 3 (September 1991), 94-104
[2] http://www.ubicomp.org/
[3] http://pervasive2008.org/
[4] http://www.percom.org/
[5] http://www.ami-07.org/
[6] B. Schilit, N. Adams, and R. Want. (1994). “Context-aware computing applications“. IEEE Workshop on Mobile Computing Systems and Applications (WMCSA’94), Santa Cruz, CA, US: 89-101 .
[7] Albrecht Schmidt(2003). “Ubiquitous Computing – Computing in Context“. PhD dissertation, Lancaster Univeristy.
[8] http://
www.internetofthings-2008.org
[9] http://www.tei-conf.org/

Deadline for Summer@IAIS soon

Not much time left to apply for the student research project. From 20.8. to 30.9.2007 we plan to design and implement a new specific search engine. The program is open to all computer science and media informatics students, primarily in Germany. We assume it will be very competitive. For accepted students we provide a HIWI-job at Fraunhofer IAIS for the 5 weeks. It will be possible to get credits for the course (IPEC lab course at the University of Bonn).

For more information please see: www.iais.fraunhofer.de/summer2007.html

bi-t Student demo lab results at Fraunhofer IAIS

This morning we presented selected demos of the lab on location and context awareness to people at the Fraunhofer IAIS. Besides the fact that our main infrastructure component (the Ubisense indoor system) did not work the demos went well. It was very strange – the infrastructure worked for the last 6 weeks (including several reboots) and this morning after rebooting the server it did not find the sensors anymore for several hours.

The majority of demos were based on the second assignment which was to create a novel application that makes use of an indoor location system. The applications implemented by the students included a heat-map (showing where a room is mainly used), co-location depended displays (enabling minimal setup effort and admin effort), museum information system (time and location depend display of different levels of information), and a security system (allowing a functionality only inside a perimeter dynamically defined by tags). Overall it was very interesting what the students created in 4 weeks of hard work.

We also briefly showed the location post its which were based on GPS and were done for the first group assignment, the CardioViz prototype (from the lab in the winter term), and the Web annotation tool that is now nearly ready.

Even though there were some difficulties in running some of the demos I am still convinced in a research environment we need to show live demos and not just ppt-slide-ware 😉 We probably have to demo more to get more professional with non-working components.

More pictures are online at http://foto.ubisys.org/iais_presentation/

Interaction Ivrea

Pascal Bihler send me today a link to an interesting teaching/research project at Ivrea (http://courses.interaction-ivrea.it/strangely/). The assignment is to think how a common artefact can be enhanced with digital technology. The results of the exercise are quite impressive. It seems that Interaction Ivrea is now somehow integrated into the Domus Academy in Milan.

When I was at Ivrea in November 2003 for the Jamboree of the disappearing computer initiative it was amazing to see the level of creativity – from the Ivrea design institute as well as from the various European projects in the disappearing computer program.

There at Ivrea I also met a really amazing person who has greatly influenced interaction design in the last 20 year: Gillian Crampton Smith. She was the founding director of the institute in Ivrea and has moved on to Venice (IUAV).

Ubisense system at IAIS used in the lab

During our lab on context- and location-aware systems we have currently one task to create a novel application using the Ubisense indoor location system. We have set up a little server in C# that reads out the tag-ids and the current x,y,z coordinates and multi-casts it via UDP to several clients.

The students have to create an application that reads the tags and their locations from UDP and make use of it. One first interesting result was a heat-map, that would allow to map out the usage of the room. By the end of term we hope to have a nice set of applications.

For many students it was the first time at the Fraunhofer lab and hence they found interesting things to play with. E.g. Diana explored the PointScreen , a system developed in the division Virtual Environments at Fraunhofer IAIS. Further reading on the point screen:
Li et al. Gesture Frame – a Screen Navigation System for Interactive Multimedia and Strauss et al. : Information Jukebox – A semi-public device for presenting multimedia
information content
(2003).

Audio Tapes Soon be History, Printed Paper Next?

News papers in the UK have reported that sales for audio tapes at Currys were down from 83 million in 1989 to 0.1 million in 2006 and hence Currys is going to stop stocking them. We recently discussed how expensive talking toys (in particular dolls) were just twenty years ago. The were based on mini records or endless tapes. By now storage chips are so cheap that a good birthday card can sing you happy birthday.

How long will it take before electronic paper will replace printed paper in the large? At least it is close enough to seriously think about applications and business models that arise.

Lecture at CDTM in Munich

The CDTM is a joint elite study program from LMU Munich and TU Munich (http://www.cdtm.de/). It offers a set of complementary course for students from different backgrounds including math, business studies, engineering, computer science and media informatics. In their course on multimodal HCI I gave today an introductory lecture on the motivation for and the basics of user interface engineering.

In the seminar room was a real typewrite – in fact a travel type write. It was a real good prop to discuss micro- and macro-efficiency.

One question about the sustainability of a competitive advantage based on user interfaces made me really think. As the user interface is visible it is really hard to protect the competitive advantage and IPR are difficult on this topic. Desktop GUIs are a good example how quickly ideas propagate between competing systems. The only real option to maintain a competitive advantage is to continuously innovate – keeping the status quo means falling behind. The fact that one can not keep new user interface concepts secret (if one includes them in a product) is one of the most exciting aspects of user interfaces research – it is fast moving because everyone shows off their results!

User tests and final presentations of the lab course



This week the final presentations of the lab course were due. It was really interesting to see what motivated students can achieve in just 4 weeks! The projects explored the idea of contextual ECG and students implemented the data acquisition, transmission over the network, and visualisation. The user studies showed that there is quite some potential in the idea (even though there is also still some way to go before the system is perfect 😉 We plan to publish a paper on the results of the course.

Business in a Nutshell, Thoughts on Patents

Thomas Doppelberger and Andreas Aepfelbacher from Fraunhofer Venture Group (http://www.venturecommunity.de/) came to our lab course to teach us how to get from a technical idea to a successful business. They talked us through the essentials of a business plan, discussed common mistakes and gave us an insight how companies are evaluated and furthermore encouraged our “entrepreneurial thinking”.

After such a day and the interesting discussions with them and the students one wonders why we are not trying more often to look into the option of starting a company. Obviously not every start-up is the next youtube or second life – but without trying ideas out, there is not way in telling if the work or not.

Over lunch we talked very briefly about issues on patents and licensing. Even though the value of patents for companies is fairly comprehensible many questions remain open in publicly funded research institutions, – especially with regard to computer science. Thinking a little more about the figures they presented in the morning I wonder how the balance (over all publicly funded research in universities and non-profit institutions) between investments in securing rights (basically making patents and holding them) and income (e.g. licensing) is. Are there any figures available? Within the whole Fraunhofer Gesellschaft it is impressive to see the contribution of one development such as MP3 to such an equation.

Broadcasting your Heart Beat



This morning the first 3 students completed the exercise part of our lab class. For the first team the basics are done and we start with the exciting part 😉

We have different sensors that can be connected via Bluetooth to the phone (e.g. heart rate, pulse oximeter, GPS) and the task for the project is to invent a new application that makes use of sensors and creates a new user experience.

In the brainstorming session some were tearing out their hair – but it was rewarding. Some of the ideas that came out are really novel – and perhaps a bit to crazy to implement them. One example of such an idea is to create a web radio station that broadcasts the live heart beat of celebrities as an audio stream (not sure if this is the right way to fame). Other ideas centered on support for sport, exercise and mobile health.

Alexander De Luca (a former colleague from LMU Munich) was spending the last few days with us here. He helped greatly to write some code to get the Alivetec-ECG data parsed.

Lab started: Developing Interactive Mobile Applications

Our Lab course on “Developing Interactive Mobile Applications” started today with 11 students (the limit was 8 – but 3 more is OK as we got some additional phones from Nokia).

Shortly after lunch everyone had there first application written and deployed to the phone. In the afternoon we looked into sending an SMS from JAVA using the messaging API…. And this is fairly easy.

For more details on the course see: http://uie.bit.uni-bonn.de/developing.php

Seeing the possibilities of JAVA ME and how quick people get applications running made me wonder how long it takes till we have massive mal-ware, viruses and spam on the phone. Secutity on phones may be one of the upcomming challenges.

Educational Fair in St. Petersburg


Together with Alexandra Reitelmann I am in St. Petersburg (Russia) to advertise our study programs at the B-IT and University of Bonn (http://www.bit.uni-bonn.de/). In particular we talk to potential students about the master programs in Media Informatics and Life Science Informatics. The educational fair is held in the centre of St. Petersburg at the Russian Museum of Ethnography, which is a truly magnificent building. Overall I was surprised by the city, the buildings, the shops and the quality of the hotel.

The discussions with potential students and in some cases with their parents were interesting. It showed that there is a demand for high quality education and that Germany is an attractive place to study. Given the background of many people we talked to it could be interesting to look into the potential of a conversion master in media informatics, where people with different degrees (e.g. psychology, business studies, math, or engineering) can enter. In comparison to other countries education seems to be a bargain in Germany.