Reading List: Developing Ubiquitous Computing Devices

 

Together with Thomas Kubitza I was teaching a class in the UBI summer school on Developing Ubiquitous Computing Devices. The summer school was held in Oulu and organized by Timo Ojala.

In total the summer school include the following 4 courses:

  • EXPERIENCE-DRIVEN DESIGN OF UBIQUITOUS INTERACTIONS IN URBAN SPACES Prof. Kaisa Väänänen-Vainio-Mattila, Tampere University of Technology, Finland & Dr. Jonna Häkkilä, University of Oulu, Finland
  • DESIGNING MOBILE AUGMENTED REALITY INTERFACES Prof. Mark Billinghurst, University of Canterbury, New Zealand
  • DEVELOPING UBIQUITOUS COMPUTING DEVICES Prof. Albrecht Schmidt, University of Stuttgart, Germany
  • URBAN RESOURCE NETWORKS Prof. Malcolm McCullough, University of Michigan, USA

There was more than work… if you are curious have a look at flickr for photos and more photos.

As some people asked for the reading list for our course on Developing Ubiquitous Computing Devices, I thought I post it here…. The reading list is also available as PDF for download.

The reading list comprises 4 areas that are relevant to our course. We expect that you have come across the original paper by Marc Weiser, introducing the concept of ubiquitous computing [1].

In the first part we have included papers that provide an overview of interaction concepts that are relevant in the context of ubiquitous computing. In particular this is tangible interaction [2a] [2b], reality based interaction [3], embedded interaction [4]. The concept of informative art [5] is introduced as well as the notion of persuasive technologies [16].This part is concluded with an overview of interaction with computers in the 21st century [6].

In the second part we have included a paper on how to create smart devices [7], which gives an overview of sensors that may be useful for creating novel and reactive devices. In [8] sensing is extended to context and context-awareness. In the third part we introduce the .NET Gadgeteer platform [9] and show some trends in the development of ubiquitous computing devices: how can we create new products once we can fabricate things [10] and enclosures [10b] and how ubicomp technologies enable new devices and devices concepts [11].

The final part provides some ideas for application scenarios that we plan to assess during the course. In [12] a concept of how to change a bed into a communication media is presented and in [13] a social alarm clock is presented. A recent study [14] shows the impact of technology on communication and in [15] an overview of novel alarm clocks and sleep monitoring devices is given.

References
[1] Weiser, M. (1991). The computer for the 21st century. Scientific american,265(3), 94-104. http://wiki.daimi.au.dk/pca/_files/weiser-orig.pdf
[2a] Ishii, H., & Ullmer, B. (1997, March). Tangible bits: towards seamless interfaces between people, bits and atoms. In Proceedings of the ACM SIGCHI Conference on Human factors in computing systems (pp. 234-241). ACM. http://doi.acm.org/10.1145/258549.258715 http://labs.rightnow.com/colloquium/papers/tangiblebits.pdf
[2b] Ishii, H. (2008, February). Tangible bits: beyond pixels. In Proceedings of the 2nd international conference on Tangible and embedded interaction (pp. xv-xxv). ACM. http://doi.acm.org/10.1145/1347390.1347392
[3] Jacob, R. J., Girouard, A., Hirshfield, L. M., Horn, M. S., Shaer, O., Solovey, E. T., & Zigelbaum, J. (2008, April). Reality-based interaction: a framework for post-WIMP interfaces. In Proceedings of the SIGCHI conference on Human factors in computing systems (pp. 201-210). ACM. http://doi.acm.org/10.1145/1357054.1357089 http://research.cs.queensu.ca/~audrey/papers/chi08.pdf
[4] Kranz, M., Holleis, P., & Schmidt, A. (2010). Embedded interaction: Interacting with the internet of things. Internet Computing, IEEE, 14(2), 46-53. http://dx.doi.org/10.1109/MIC.2009.141 http://pure.ltu.se/portal/files/39756776/FINAL_PRINT_w2iot_preprint.pdf
[5] Ferscha, A. (2007). Informative art display metaphors. In Universal Access in Human-Computer Interaction. Ambient Interaction (pp. 82-92). Springer Berlin Heidelberg. http://www.pervasive.jku.at/Research/Publications/_Documents/InformativeArtDisplayMetaphors-ferscha2007.pdf
[6] Schmidt, A., Pfleging, B., Alt, F., Sahami, A., & Fitzpatrick, G. (2012). Interacting with 21st-Century Computers. Pervasive Computing, IEEE, 11(1), 22-31. http://www.hcilab.org/wp-content/uploads/schmidt-ieeepc-21century.pdf http://dx.doi.org/10.1109/MPRV.2011.81
[7] Schmidt, A., & Van Laerhoven, K. (2001). How to build smart appliances?.Personal Communications, IEEE, 8(4), 66-71. http://www.comp.lancs.ac.uk/~albrecht/pubs/pdf/schmidt_ieee_pc_08-2001.pdf
[8] Schmidt, A. (2013). Context-Aware Computing: Context-Awareness, Context-Aware User Interfaces, and Implicit Interaction. The Encyclopedia of Human-Computer Interaction, 2nd Ed. http://www.interaction-design.org/encyclopedia/context-aware_computing.html
[9] Villar, N., Scott, J., Hodges, S., Hammil, K., & Miller, C. (2012). . NET gadgeteer: a platform for custom devices. In Pervasive Computing (pp. 216-233). Springer Berlin Heidelberg. http://research.microsoft.com/pubs/163162/Gadgeteer%20Pervasive%202012%20Proof.pdf
[10] Schmidt, A., Doring, T., & Sylvester, A. (2011). Changing How We Make and Deliver Smart Devices: When Can I Print Out My New Phone?. Pervasive Computing, IEEE, 10(4), 6-9. http://www.hcilab.org/wp-content/uploads/2012/04/schmidt2011changing.pdf http://dx.doi.org/10.1109/MPRV.2011.68
[10b] Weichel C., Lau M., Gellersen,H. (2013). Enclosed: A Component-Centric Interface for Designing Prototype Enclosures. Tangible, embedded, and embodied interaction conference (TEI 2013) http://doi.acm.org/10.1145/2460625.2460659 http://www.csweichel.de/papers/2013-enclosed.pdf
[11] Hodges, S., Villar, N., Scott, J., & Schmidt, A. (2012). A New Era for Ubicomp Development. Pervasive Computing, IEEE, 11(1), 5-9. http://dx.doi.org/10.1109/MPRV.2012.1 http://research.microsoft.com/pubs/163175/ANewEraForUbiCompDevelopment-IEEEPervasiveComputing.pdf
[12] Dodge, C. (1997, March). The bed: a medium for intimate communication. InCHI’97 extended abstracts on Human factors in computing systems: looking to the future (pp. 371-372). ACM. http://doi.acm.org/10.1145/1120212.1120439
[13] Schmidt, A., Shirazi, A. S., & van Laerhoven, K. (2012). Are You in Bed with Technology?. Pervasive Computing, IEEE, 11(4), 4-7. http://dx.doi.org/10.1109/MPRV.2012.63
[14] Schmidt, A. (2006). Network alarm clock (The 3AD International Design Competition). Personal and Ubiquitous Computing, 10(2-3), 191-192. http://dx.doi.org/10.1007/s00779-005-0022-y http://old.hcilab.org/documents/Schmidt_NetworkAlarmClock.pdf
[15] Shirazi, A. S., Clawson, J., Hassanpour, Y., Tourian, M. J., Schmidt, A., Chi, E. H., Borazio, M., & Van Laerhoven, K. (2013). Already Up? Using Mobile Phones to Track & Share Sleep Behavior. International Journal of Human-Computer Studies. http://www.sciencedirect.com/science/article/pii/S1071581913000244
[16] Fogg, B. J. (2009, April). A behavior model for persuasive design. In Proceedings of the 4th international conference on persuasive technology (p. 40). ACM. http://bjfogg.com/fbm_files/page4_1.pdf

Appendix: .NET Gadgeteer Links (optional)

Zeitgeist, GNOME Activity Journal etc. – Workshop at CHI

On Saturday there was a workshop on monitoring, logging and reflecting. Know Thyself: Monitoring and Reflecting on Facets of One’s Life. In the workshop we discussed technologies and concepts for monitoring and using personal information. I started out with asking the question who knows what about you? The list is quickly growing (e.g. telecom provider, travel agent, super market, bank, mail provider, facebook, etc.) and so is the set of information they know about you. And it becomes clear that these entities keep a better record about an individual that the individuals themselves. Hence our central suggestion is that the user who is the one who could have easy access to all this information should make more of it and benefit from this information, for more see the paper [1] and the slides from the talk.

Zeitgeist Magic from Seif Lotfy on Vimeo.

There is more information about the workshop and the topic in general:

My pick of the contributions is the Dunbar email mining system from Stanford.

PS: CHI is good for your health 🙂

[1] Thorsten Prante, Jens Sauer, Seif Lotfy, Albrecht Schmidt. Personal Experience Trace: Orienting Oneself in One’s Activities and Experiences. CHI 2010 workshop on Know Thyself: Monitoring and Reflecting on Facets of One’s Life.

NSF/EU workshop in Mannheim

Mohan Kumar and Marco Conti organized an EU/NSF workshop on Future Directions in Pervasive Computing and Social Networking for Emerging Applications. They managed to get together an interesting set of people and the discussion in the break out session were very enjoyable and I got a number of ideas what really are the challenges to come.

There are the position statements on the web page and at some point the identified grand challenges will be available.

PS: blackboards are still highly effective 😉

German Workshop on Tangible Interaction

Friday and Saturday we hosted the GI working groups on tangible interaction (GI Arbeitskreis Be-greifbare Interaction in gemischten Wirklichkeiten) for the second inventor’s workshop. We were positively surprised that more than 30 people took part. We started with a set of nice demos from our students and then Nicolas Villar presented an invited talk.

The concept of the workshop was to create ideas by looking more closely at different materials that may be used for interaction and by assessing novel interaction technologies. On Friday afternoon we had several tables with material (e.g. smart materials, conductive fabric, magnetic materials, etc.) and technologies (e.g. VoodooIO, EduWear kit, Physiological sensing based on the Nexus system). We bought a set of materials from MUTR Teaching Materials. In groups the participants create many design ideas by imagining a certain technology to be used in a specific context.

The social event was at the unperfekthaus.de in Essen – if you visit Essen this is a “must see”. The concept is unique!

On Saturday we started with an exercise to investigate the qualities of tangible UIs in comparison to traditional GUIs. For this we looked at different games (e.g. Space invaders, Super Mario, Sims, Halo) and though how we would make them more tangible and what pros and cons are there for the tangible and GUI version. In this context we also discussed the approach of exertion interfaces [1].

The rest of the morning was spent working on more concrete ideas for the concepts created on Friday afternoon – with remarkable results – see the Wiki for details.

[1] http://exertioninterfaces.com/

Creativity Workshop at NRC Tampere

Creativity is a key issue of creating novel applications and interaction methods and techniques. Over two days we ran a hands on prototyping workshop on physical user interaction. We setup teams of 3 people, each team including at least one person with design and one with programming skills. Within about 5 hours the teams had to create a multiplayer interactive game – using a mouse or several mice as basic sensors. We discussed how novelty and the learning curve of interaction technologies relate to physical interaction. The results of the workshop were most impressive… and I think some of them could be really pushed further.
In 2008 there was an interesting issue of Interactions Magazine on innovation. The cover article by Dubberly provides a good conceptual models of innovation [1]. In my lecture on HCI I tyically also introduce the TRIZ (“The theory of solving inventor’s problems”) methodology, introduced by the Soviet engineer and researcher Genrich Altshuller. TRIZ is interesting to me, as it is in contrast to many other creativity approaches,a systemtic (algorithmic) approach for generating innovative ideas and solutions for problem solving. However I am not sure how well it works in the real world…
During the workshop our approach was very practical. We did also a speed invention exercise – the task was to create 3 game concepts (related to physical penny games) within 10 minutes. The results were pretty impressive – perhaps someone has the time to implement them.

And being in Finland we got the real sauna experience – inside +79°C and then outside into the snow -23° – this is cool (in the very literal sense!).

[1] Dubberly, H. 2008. Toward a model of innovation. interactions 15, 1 (Jan. 2008), 28-36. DOI= http://doi.acm.org/10.1145/133

Workshop on Pervasive Advertising at Informatik 2009 in Lübeck

Following our first workshop on this topic in Nara during Pervasive 2009 earlier this year we had on Friday the 2nd Pervasive Advertising Workshop in Lübeck as part of the German computer science conference Informatik 2009.

The program was interesting and very diverse. Daniel Michelis discussed in his talk how we move from an attention economy towards an engagement economy. He argued that marketing has to move beyond the AIDA(S) model and to consider engagement as central issue. In this context he introduced the notion of Calm Advertising and interesting analogy to Calm Computing [1]. Peter van Waart talked about meaningful adverting and introduced the concept of meaningful experience. To stay with the economy term consider advertising in an experience economy. For more detail see the workshop webpage – proceedings will be soon online.

Jörg Müller talked about contextual advertising and he had a nice picture of the steaming manhole coffee ad – apparently from NY – but it is not clear if it is deployed.

If you are interested in getting sensor data on the web – and having them also geo-referenced – you should have a look at http://www.52north.org. This is an interesting open source software system that appears quite powerful.

Florian Alt presented our work interactive and context-aware advertising insight a taxi [2].

[1] Weiser, M., Brown, J.S.: The coming age of calm technology. (1996)

[2] Florian Alt, Alireza Sahami Shirazi, Max Pfeiffer, Paul Holleis, Albrecht Schmidt. TaxiMedia: An Interactive Context-Aware Entertainment and Advertising System (Workshop Paper). 2nd Pervasive Advertising Workshop @ Informatik 2009. Lübeck, Germany 2009.

Workshop at MobileHCI: Context-Aware Mobile Media and Mobile Social Networks

Together with colleagues from Nokia, VTT, and CMU we organized a workshop on Context-Aware Mobile Media and Mobile Social Networks at MobileHCI 2009.

The topic came up in discussions some time last year. It is very clear that social network have moved towards mobile scenarios and that utilizing context and contextual media adds a new dimension. The workshop program is very diverse and ranges studying usage practices to novel technological solutions for contextual media and application.

One topic that is interesting to further look at is to use (digital) social networks for health care. Taking an analogy in history it is evident that the direct social group you were in took were the set of people that helped you in case of illness or accident. Looking at conditions and illnesses that cause a loss of mobility or memory it could be interesting to find applications on top of digital social networks to provide help. Seems this could be a project topic.

In one discussion we explored what would happen if we would change our default communication behavior from closed/secret (e.g. Email and SMS) to public (e.g. bulletin boards). I took the example of organizing this workshop: our communication has been largely on email and has not been public. If it would had been open (e.g. public forum) we probably would have organized the workshop in the same way but at the same time provided an example how one can organize a workshop and by this perhaps provided useful information for future workshop chairs. In this case there are little privacy concerns but images all communication is public? We would learn a lot about how the world works…

About 10 years ago we published at paper there is more to context than location [1]. However, looking at our workshop it seems: location is still the dominant context people think of. Many of the presentations and discussions included the term context, but the examples focused on location. Perhaps we do need location only? Or perhaps we should look more closely to find the benefit of other contexts?

[1] A. Schmidt, M. Beigl, H.W. Gellersen (1999) There is more to context than location, Computers & Graphics, vol. 23, no. 6, pp. 893-901.

CfP: What can the Internet of Things do for the Citizen?

Together with Stephan Karpischek (ETH Zurich) and Florian Michahelles (ETH Zurich & Auto-ID labs) I organize a workshop at Pervasive 2010 in Helsinki looking for an end-user perspective on the Internet of Things.

The call is online at: http://www.autoidlabs.org/events/ciot2010 and the deadline is the 15th of January 2010.

The topics that we hope for include emerging applications, interaction paradigms, infrastructure, and social impact of new technologies and the implications for citizens.

For more information you can subscribe to the facebook group.

Workshop on Pervasive Computing in Advertising

We got a good set of submission for our workshop and had about 20 participants who joined us in Nara to discuss how pervasive computing will shape advertising in the future. The papers and a selection of talks is online on the workshop website: http://pervasiveadvertising.org

One question that was central to our discussion was: what is advertising and how is it different from information. It became quickly clear that there is a lot of information that has an influence on behavior and in particular shopping decisions and some of it is considered advertising but much is not. Hence it seems really interesting to imagine a world where advertising is replaced by information. One could image that replacing advertising by information (e.g. as it happens already in some domains such a hotel recommendations) would change the whole approach for creating product or providing services.

We have presented in the workshop our work on contextual mobile displays. The idea is that in the future we could have mobile displays (that replace current printed items, like bumper stickers, bags with printed logos, and t-shirts with prints) could become active and could act as contextual displays. Have a look at the paper for more details [1].

[1] Florian Alt, Albrecht Schmidt, Christoph Evers. Mobile Contextual display system. Pervasive Advertising Workshop at Pervasive 2009. (contact Florian Alt for a copy of the paper)

Towards interaction that is begreifbar

Since last year we have in Germany a working group on graspable/tangible interaction in mixed realities.
In German the key term we use is “begreifbar” or “begreifen” which has the meaning of acquire a deep understanding of something and the words basic meaning is to touch. Basically understand by touching – but in a more fundamental sense than grasping or getting grip. Hence the list of translations for “begreifen” given in the dictionary is quite long.
Perhaps we should push more for the word in the international community – Towards interaction that is begreifbar (English has too few foreign terms anyway 😉

This meeting was organized by Reto Wettach at Potsdam and the objective was to have two days to invent things together. The mix of people mainly included people from computer science and design. It is always amazing how many ideas come up if you put 25 people for a day in a room 🙂 We followed this week up on some of the ideas related to new means for communication – there are defiantly interesting student projects on this topic.

In the evening we had a half pecha-kucha (each person 10 slides of 20 seconds – in total 3:20, the original is 20 slides) http://www.pecha-kucha.org/. It is a great way of getting quickly to know about work, research, ideas, and background of other people. It could be format we could use more in teaching a perhaps for ad-hoc sessions at a new conference we plan (e.g. http://auto-ui.org) … prepared my slides on the train in the morning – and it is more challenging that expected to get a set of meaningful pictures together for 10 slides.

Overall the workshop showed that there is a significant interest and expertise in Germany moving from software ergonomics to modern human computer interaction.
There is a new person on our team (starting next week) – perhaps you can spot him on the pics.
For a set of pictures see my photo gallery and the photos on flickr.