Linking the activities in the physical world to actions in the digital/virtual

Currently we have an assignment in our Pervasive Computing class that asks students to design and develop a system where actions are associated with artifacts. Technically students should develop a web based solution using RFID. Apropos RFID, … if you look for a good introduction on RFID read Roy Want’s IEEE Pervasive Magazin paper [1].

We use the hardware from (Ztamp:s and Mir:ror) as the focus is on the concept and application and not on the underlying technology. To ease development Florian and Ali have developed a little system that offers WebCallBacks (students can register a URL and that is called when a tag is read).

Linking by tagging of objects has been well explored, e.g. [2] and [3], and I think it is about time that this technologies will make an impact in the consumer market – the technology gets cheap enough now (and perhaps one of our students has a great idea).

Some years back (in the last millennium) a company tried to push linking of paper adverts and digital content with the CueCat ( – I was impressed and inspired at that time but in my view it had two major weaknesses: (1) technically too early and (2) encoding of serial numbers instead of URLs. The RadioShack catalog and the Wired Magazine that included codes showed the potential – but it was too cumbersome as it was restricted to the PC …

We did some work on the topic, too around that time – at RFID reader integrated in a glove – which resulted in a Poster at ISWC [4] and a patent [5].

[1] Want, R. 2006. An Introduction to RFID Technology. IEEE Pervasive Computing 5, 1 (Jan. 2006), 25. DOI=

[2] Harrison, B. L., Fishkin, K. P., Gujar, A., Portnov, D., and Want, R. 1999. Bridging physical and virtual worlds with tagged documents, objects and locations. In CHI ’99 Extended Abstracts on Human Factors in Computing Systems (Pittsburgh, Pennsylvania, May 15 – 20, 1999). CHI ’99. ACM, New York, NY, 29-30. DOI=

[3] Ljungstrand, P. and Holmquist, L. E. 1999. WebStickers: using physical objects as WWW bookmarks. In CHI ’99 Extended Abstracts on Human Factors in Computing Systems (Pittsburgh, Pennsylvania, May 15 – 20, 1999). CHI ’99. ACM, New York, NY, 332-333. DOI=

[4] Schmidt, A., Gellersen, H., and Merz, C. 2000. Enabling Implicit Human Computer Interaction: A Wearable RFID-Tag Reader. In Proceedings of the 4th IEEE international Symposium on Wearable Computers (October 18 – 21, 2000). ISWC. IEEE Computer Society, Washington, DC, 193. (Poster as large PNG)

[5] US Patent 6614351 – Computerized system for automatically monitoring processing of objects. September 2, 2003.

Some Interesting Papers and random Photos from Pervasive 2009

Pervasive 2009 had a really exciting program and provided a good overview of current research in pervasive and ubiquitous computing. Have a look at the proceedings of the pervasive 2009 conference. The Noh theater in Nara was a very special and enjoyable venue and it was organized perfectly – as one would expect when travelling to Japan.

The idea of having short and long papers together in the main track worked very well in my view. The number of demos and posters was much higher than in the years before – and that was great and very inspiring. Have a look at the photos for some of the posters and demos.
The program consisted of 20 full papers (18 pages) and 7 notes (8 pages) which were selected in a peer review process out of 147 submissions (113 full papers, 34 notes) which is a acceptance rate of 18%.

John Krumm presented his paper Realistic driving tips for location privacy – again having a good idea making the presentation interesting beyond its content (having review snippets in the footer of the slides – including a fake review). The paper explores the difficulties that arise when creating fake GPS tracks. He motivated that the probabilities need to be taken into account (e.g. you are usually on a road). I liked the approach and the paper is worthwhile to read. I think it could be interesting to compare the approach is not create the tracks but just share them between users (e.g. other people can use parts of my track as fake track and in return I get some tracks that I can use as fake tracks).

If you phone knows where you are you can use this information to control your heating system. This was the basic idea of the research presented by Stephen Intille. They explored using GPS location of the users to automate control of the heating / air condition control in a house. It seems there is quite some potential for saving energy with technology typically used in the US (one temperature control for the whole house). In Europe where heating systems typically offer finer control (e.g. room level) the potential is probably larger.

James Scott presented a paper that showed how you can use force gestures to interact with a device. In contrast to previous research (e.g. GUMMI) the approach works with a ridged device and could be used with current screen technologies.

What do you need to figure out who is holding and using the remote control? This question is addressed in the paper “Inferring Identity Using Accelerometers in Television Remote Controls” that was presented by Jeff Hightower. They looked at how well button press sequences and accelerometer data give you information about which person is using the device.

Geo-fencing: confining Wi-Fi Coverage to Physical Boundaries is an example of how to create technological solutions to fit a user’s conceptual model of the world. As people have experience with the physical world and they have mechanisms to negotiate and use space and hence linking technologies that have typically other characteristics (e.g. wireless radio coverage) to the known concept is really interesting.

Situvis, a tool for visualizing sensor data, was presented by Adrian Clear from Aaron’s group in Dublin. The software, papers and a video is available at: The basic idea is to have a parallel coordinate visualization of the different sensor information and to provide interaction mechanisms with the data.

Nathan Eagle presented the paper “Methodologies for continuous cellular tower data analysis”. He talked about the opportunities that arise when we have massive amounts of information from users – e.g. tracks from 200 million mobile phone user. It really is interesting that based on such methods we may get completely new insights into human behavior and social processes.

If you have seen a further interesting paper in the conference (and there are surely some) that I have missed feel free to give a link to them in the comments to this post.

CfP Workshop on Pervasive Advertising

We organize at this year’s Pervasive computing conference in Nara, Japan a workshop on Pervasive Advertising –
We expect that there is a lot of interesting research going on in the area and it is clearly a controversial topic. Being an optimist – I see the new options that arise. In particular a future with less annoying advertisements is one hope 🙂
But many people are focusing on the risks that arise – an interesting positing with some criticism of our workshop objective can be found at the near future laboratory  I do not share their views 🙂
To me the idea that if you do not research it, it does not happen seems not a very viable option. I still think with research we can shape the future!
I am already looking forward to the submission and to the workshop. You have a contribution? Deadline is Feb, 11 2008.

Information vs. Mobility, Percom PC meeting in New York

The PC meeting for Percom 2009 took place at IBM in Hawthorne, NY. Percom had about 200 submissions and many good ones – so we could compile an exciting program across the whole field of pervasive computing and communication. As one of three program vice chairs I have looked in detail in about 1/3 of the submissions that were application related. It is interesting to observe that research as a whole in the field becomes more major and at the same time more incremental. 

To me this puts up the big question in which domains will the new big innovations happen, what is the next trend after we have pervasive computing? There are luckily plenty of options, but at the moment it seems that there develops an interesting relationship between information, communication, mobility and energy. It seems that we can compensate mobility by information and communication and similarly we can reduce energy required by information available. One example is: if I know where things are (=information) I can reduce the effort required to find them (=mobility). Is there more to it?
Each time in the US – even in New York were public transport works quite well – one is surprise how alien it appears to many that it could be an option to take public transport on a business trip (e.g. there are no first class coaches on regional trains). Flying from Düsseldorf into Newark it was convenient to take the train to Penn Station in NY City and then an express train to White Plaines. If we would not have gone for a walk in the city we probably would have been equally fast as by car. With the again low gas prices in the US (less than 2U$ per gallon, down from 4 just a few month ago) I would expect public transport and small cars will not gain too much popularity – before the next rise in gas prices.

PS: it is amazing how many possiblities there are to serve coffee (and this is probably not one of the most environment friendly)

Visitors to our Lab

Christofer Lueg (he is professor at the School of Computing & Information Systems at the University of Tasmania) and Trevor Pering (he is a senior researcher at Intel Research in Seattle) visited our lab this week. The timing is not perfect but at I am not the only interesting person in the lab 😉

Together with Roy Want and others Trevor published some time ago an article in the IEEE Pervasive Magazine that is still worthwhile to read “Disappearning Hardware” [1]. It shows clearly the trend that in the near future it will be feasible to include processing and wireless communication into any manufactured product and outlines resulting challenges. One of those challenges which we look into in our lab is how to interact with such systems… Also in a 2002 paper Christopher raised some very fundamental questions how far we will get with intelligent devices [2].

[1] Want, R., Borriello, G., Pering, T., and Farkas, K. I. 2002. Disappearing Hardware. IEEE Pervasive Computing 1, 1 (Jan. 2002), 36-47. DOI=

[2] Lueg, C. 2002. On the Gap between Vision and Feasibility. In Proceedings of the First international Conference on Pervasive Computing (August 26 – 28, 2002). Lecture Notes In Computer Science, vol. 2414. Springer-Verlag, London, 45-57.

How to proof that Ubicomp solutions are valid?

Over the last years there have been many workshops and sessions in the ubicomp community that address the evaluation of systems. At Pervasive 2005 in Munich I co-organized a workshop on Application led research with George Coulouris and others. For me one of the central outcomes was that we – as ubicomp researchers – need to team up in evaluating our technologies and solutions with experts in the application domain and that we stay involved in this part of the research. Just handing it over for evaluation into the other domain will not bring us the insights we need to move the field forward. There is a workshop report which appeared in the IEEE Pervasive Magazine, that discusses the topic in more detail [1].

On Friday I met we a very interesting expert in the domain of gerontology. Elisabeth Steinhagen-Thiessen is chief consultant and director of the protestant geriatric centre of Berlin and professor of internal medicine/gerontology at the Charite in Berlin. We talked about opportunities for activity recognition in this domain and discussed potential set-ups for studies.

[1] Richard Sharp, Kasim Rehman. What Makes Good Application-led Research? IEEE Pervasive Computing Magazin. Volume 4, Number 3. July-September 2005.

Birthday candles going electronic

What is a birthday cake without a candle? Sometimes it is hard to find a candle but having a creative team there is always a solution – less than 3 minutes away 😉 As always with new technologies – after deployments ideas for Version 2 (which will include much more functionality) emerge… An there was another business idea – interactive wedding cakes – perhaps we explore this later this year 😉

Teaching in primary school, digital photography, civilization

I had a day off an was as “teaching assistant” on a school trip with the kids my wife is teaching. The trip went to a museum village (Wackershofen), which tries to preserve and communicate how people lived about 100 years ago.

On side observation was that in digital photography the limiting factor is now not anymore the memory space but the batteries in the camera. This has changed over the last 2 years – there children still selected which pictures they have to delete – now that is no issue anymore. This shows that some of the trends in pervasive computing (in this case unlimited memory) is already there…

In a project we converted manually flax into threads and theoretically into linen fabric. Some years ago I was involved in doing a similar project – with a focus on the multimedia docummentation – also with a primary school. We learned that it took a person one winter to make one piece of garment. Putting this into perspective we see an interesting trend of devaluation of physical object (cloth are one example, but applies also to high tech goods such as MP3 players) due to advances in engineering. This devaluation of physical goods led to a higher standard of living and consequently to a higher life expectancy. I wonder how further advances – especially in digital engineering will affect the quality of life…

Impressions from Pervasive 2008

Using electrodes to detect eye movement and to detect reading [1] – relates to Heiko’s work but uses different sensing techniques. If the system can really be implemented in goggles this would be a great technologies for eye gestures as suggested in [2].

Utilizing infrastructures that are in place for activity sensing – the example is a heating/air condition/ventilation system [3]. I wondered and put forward the question how well this would work in active mode – where you actively create an airflow (using the already installed system) to detect the state of an environment.

Further interesting ideas:

  • Communicate while you sleep? Air pillow communication… Vivien loves the idea [4].
  • A camera with additional sensors [5] – really interesting! We had in Munich a student project that looked at something similar [6]
  • A cool vision video of the future is SROOM – everything becomes a digital counterpart. Communicates the idea of ubicomp in a great and fun way [7] – not sure if the video is online – it is on the conference DVD.

[1] Robust Recognition of Reading Activity in Transit Using Wearable Electrooculography. Andreas Bulling, Jamie A. Ward, Hans-W. Gellersen and Gerhard Tröster. Proc. of the 6th International Conference on Pervasive Computing (Pervasive 2008), pp. 19-37, Sydney, Australia, May 2008.

[2] Heiko Drewes, Albrecht Schmidt. Interacting with the Computer using Gaze Gestures. Proceedings of INTERACT 2007.

[3] Shwetak N. Patel, Matthew S. Reynolds, Gregory D. Abowd: Detecting Human Movement by Differential Air Pressure Sensing in HVAC System Ductwork: An Exploration in Infrastructure Mediated Sensing. Proc. of the 6th International Conference on Pervasive Computing (Pervasive 2008), pp. 1-18, Sydney, Australia, May 2008.

[4] Satoshi Iwaki et al. Air-pillow telephone: A pillow-shaped haptic device using a pneumatic actuator (Poster). Advances in Pervasive Computing. Adjunct proceedings of the 6th International Conference on Pervasive Computing (Pervasive 2008).

[5] Katsuya Hashizume, Kazunori Takashio, Hideyuki Tokuda. exPhoto: a Novel Digital Photo Media for Conveying Experiences and Emotions. Advances in Pervasive Computing. Adjunct proceedings of the 6th International Conference on Pervasive Computing (Pervasive 2008).

[6] P. Holleis, M. Kranz, M. Gall, A. Schmidt. Adding Context Information to Digital Photos. IWSAWC 2005.

[7] S-ROOM: Real-time content creation about the physical world using sensor network. Takeshi Okadome, Yasue Kishino, Takuya Maekawa, Kouji Kamei, Yutaka Yanagisawa, and Yasushi Sakurai. Advances in Pervasive Computing. Adjunct proceedings of the 6th International Conference on Pervasive Computing (Pervasive 2008).

Paul presented our paper at Pervasive 2008

Paul presented after lunch our full paper on a development approach and environment for mobile applications that supports underlying user models [1]. In the paper he shows how you can create applications while programming by example where the development environment automatically adds a KLM model. In this way the developer becomes automatically aware of estimated usage times for the application. The paper is work that builds on our paper on KLM for physical mobile interaction which was presented last year at CHI [2]. The underlying technology is the embedded interaction toolkit [3] – have a look – perhaps it makes you applications easier, too.

[1] Paul Holleis, Albrecht Schmidt: MAKEIT: Integrate User Interaction Times in the Design Process of Mobile Applications. In: Proceedings of the Sixth International Conference on Pervasive Computing, Pervasive’08. Sydney, Australia 2008, S. 56-74.

[2] Holleis, P.; Otto, F.; Hußmann, H.; Schmidt, A.: Keystroke-Level Model for Advanced Mobile Phone Interaction. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (San Jose, California, USA, April 28 – May 03, 2007). CHI ’07. ACM Press, New York, NY, 1505-1514.. 2007.