Category Archives: mobile

Karin Bee has defended her dissertation.

Posted on by
Reply
Karin Bee (nee Leichtenstern) has defended her dissertation at the University of Augsburg. In her dissertation she worked on methods and tools to support a user centered design process for mobile applications that use a variety of modalities. There are some papers that describe her work, e.g. [1] and [2]. To me it was particularly interesting that she revisited the experiment done in her master thesis in a smart home in Essex [3] and reproduced some of it in her hybrid evaluation environment.


It is great to see that now most of our students (HiWis and project students) who worked with us in Munich on the Embedded Interaction Project have finished their PhD (there are some who still need to hand in – Florian? Raphael?, Gregor? You have enough papers – finish it ;-)

In the afternoon I got to see some demos. Elisabeth André has a great team of students. They work on various topics in human computer interaction, including public display interaction, physiological sensing and emotion detection, and gesture interaction. I am looking forward to a joined workshop of both groups. Elisabeth has an impressive set of publications which is always a good starting point for affective user interface technologies.

[1] Karin Leichtenstern, Elisabeth André,and Matthias Rehm. Tool-Supported User-Centred Prototyping of Mobile Applications. IJHCR. 2011, 1-21.

[2] Karin Leichtenstern and Elisabeth André. 2010. MoPeDT: features and evaluation of a user-centred prototyping tool. In Proceedings of the 2nd ACM SIGCHI symposium on Engineering interactive computing systems (EICS ’10). ACM, New York, NY, USA, 93-102. DOI=10.1145/1822018.1822033 http://doi.acm.org/10.1145/1822018.1822033

[3] Enrico Rukzio, Karin Leichtenstern, Vic Callaghan, Paul Holleis, Albrecht Schmidt, and Jeannette Chin. 2006. An experimental comparison of physical mobile interaction techniques: touching, pointing and scanning. In Proceedings of the 8th international conference on Ubiquitous Computing (UbiComp’06), Paul Dourish and Adrian Friday (Eds.). Springer-Verlag, Berlin, Heidelberg, 87-104. DOI=10.1007/11853565_6 http://dx.doi.org/10.1007/11853565_6

>Opening Keynote at AMI 2011: Margaret Morris

Posted on by
1
>Margaret (Margi) Morris presented the opening keynote at the 2011 conference on ambient intelligence in Amsterdam (AMI2011) with the title “Left to our own devices”.

Margaret brought up an interesting point on motivation: Showing people what they lose is a stronger motivator than the prospective of gain. She made the point in order to implement this the depicted loss has to be very specific. She showed a facebook application “With a little help from my friends”, where this basic concept is applied.   I had recently seen a bill board adverting campaign for safe driving on motorways in Germany using this approach (basically showing the risk of loss of family).

In the talk several examples of devices and applications were presented. To learn more about her work I recommend the following two papers: at tool to improve emotional self-awareness [1] and an investing in social networks and their utility to promote health [2].

Another point that made me think was the question of how we design interventions. One conceptual example was about an obesity campaign. The official UK campaign starts out with the statement that obesity is a problem for 9 million kids. Her alternative is to provide instead of the information a specific hint about an opportunity for action for an individual (e.g. telling the kid when it leaves school in the afternoon: now is probably a good time to play soccer with your friends, as 16 of them like to play soccer). An open research question that relates to this seems to me to investigate the impact of information about the norm, e.g. how will it affect my behavior if I know that 70% of my friends think driving too fast is OK vs. if I know that only 20% find it acceptable. I think this could be further explored in the context of social networks to create interesting persuasive technologies.


There has been an interesting discussion after the talk. Norbert Streitz questioned if it is a good idea to ask people to engage more with digital devices (e.g. self monitoring one’s mood). The question is hinting that the engagement with the digital device keeps us from interaction in the “real” world. I think this separation is disappearing fast – making a phone call, listening to MP3, chatting with friends on facebook is for many of us real, we live in a world that is augmented by technology and the boundaries are bluring…

[1] Morris ME, Kathawala Q, Leen TK, Gorenstein EE, Guilak F, Labhard M, Deleeuw W. Mobile Therapy: Case Study Evaluations of a Cell Phone Application for Emotional Self-Awareness. Journal of Medical Internet Research 2010;12(2):e10. URL: http://www.jmir.org/2010/2/e10/

[2] Margaret E. Morris. 2005. Social Networks as Health Feedback Displays. IEEE Internet Computing 9, 5 (September 2005), 29-37. DOI=10.1109/MIC.2005.109 http://dx.doi.org/10.1109/MIC.2005.109

Opening Keynote at AMI 2011: Margaret Morris

Posted on by
1
Margaret (Margi) Morris presented the opening keynote at the 2011 conference on ambient intelligence in Amsterdam (AMI2011) with the title “Left to our own devices”.

Margaret brought up an interesting point on motivation: Showing people what they lose is a stronger motivator than the prospective of gain. She made the point in order to implement this the depicted loss has to be very specific. She showed a facebook application “With a little help from my friends”, where this basic concept is applied.   I had recently seen a bill board adverting campaign for safe driving on motorways in Germany using this approach (basically showing the risk of loss of family).

In the talk several examples of devices and applications were presented. To learn more about her work I recommend the following two papers: at tool to improve emotional self-awareness [1] and an investing in social networks and their utility to promote health [2].

Another point that made me think was the question of how we design interventions. One conceptual example was about an obesity campaign. The official UK campaign starts out with the statement that obesity is a problem for 9 million kids. Her alternative is to provide instead of the information a specific hint about an opportunity for action for an individual (e.g. telling the kid when it leaves school in the afternoon: now is probably a good time to play soccer with your friends, as 16 of them like to play soccer). An open research question that relates to this seems to me to investigate the impact of information about the norm, e.g. how will it affect my behavior if I know that 70% of my friends think driving too fast is OK vs. if I know that only 20% find it acceptable. I think this could be further explored in the context of social networks to create interesting persuasive technologies.


There has been an interesting discussion after the talk. Norbert Streitz questioned if it is a good idea to ask people to engage more with digital devices (e.g. self monitoring one’s mood). The question is hinting that the engagement with the digital device keeps us from interaction in the “real” world. I think this separation is disappearing fast – making a phone call, listening to MP3, chatting with friends on facebook is for many of us real, we live in a world that is augmented by technology and the boundaries are bluring…

[1] Morris ME, Kathawala Q, Leen TK, Gorenstein EE, Guilak F, Labhard M, Deleeuw W. Mobile Therapy: Case Study Evaluations of a Cell Phone Application for Emotional Self-Awareness. Journal of Medical Internet Research 2010;12(2):e10. URL: http://www.jmir.org/2010/2/e10/

[2] Margaret E. Morris. 2005. Social Networks as Health Feedback Displays. IEEE Internet Computing 9, 5 (September 2005), 29-37. DOI=10.1109/MIC.2005.109 http://dx.doi.org/10.1109/MIC.2005.109

>Lukowicz Consumer Confidence Index

Posted on by
Reply
>Paul Lukowicz taked in the symposium on Interaction with Smart Artifacts in Tokyo about “Large scale, context aware socio-technical systems”. As a number of other talks in our symposium he also pointed out that value of the massive amounts of data that become available by mobile sensing and crowd interaction.

Paul gave one example how such sensing data could be used. I think this is really interesting that is why I share here the Paul “Lukowicz Consumer Confidence Index” (CCIL) which is based on detecting the whereabouts of people. It assumes that we can estimate the number of people shopping at various places based on the use of their mobile devices.

Lukowicz Consumer Confidence Index

Posted on by
1
Paul Lukowicz taked in the symposium on Interaction with Smart Artifacts in Tokyo about “Large scale, context aware socio-technical systems”. As a number of other talks in our symposium he also pointed out that value of the massive amounts of data that become available by mobile sensing and crowd interaction.

Paul gave one example how such sensing data could be used. I think this is really interesting that is why I share here the Paul “Lukowicz Consumer Confidence Index” (CCIL) which is based on detecting the whereabouts of people. It assumes that we can estimate the number of people shopping at various places based on the use of their mobile devices.

Keynote by Josh Ulm at Mobile HCI 2010

Posted on by
Reply
Josh Ulm discussed how branding (and marketing in more general) has evolved and how this is now central to user experience and user interface design. He started out with showing how Nike changed marketing with the “Just Do It” campaign. He suggested that this was a transition from a product focus to a personal usage focus asking “who you become if you use the product“.

Moving towards more resent trends he argued that the iPod made the interaction with the product the essential part of the branding. With examples such as eBay and Google he showed that interaction in combination with information presentation becomes that discriminating factor and the way these brands define themselves. Overall this suggests that the user interaction and user experience is the central part for making a brand to stand out.

Standing out is not sufficient however. The experience has to be ownable. Using zappos as an example he showed how such an experience needs to be consistent across all touch points with the user – especially if you are defining your brand by experience – in short a brand has to have a unique user experience that is associated with the brand only. To achieve this there are three ingredients:
  • Values – “values have to extent into every single touchpoint of the experience”
  • Differentiation “you need to stand out in the market place” – many companies do not innovate enough, you have to take risks to stand out because you have to stand out a lot to be different – most companies do only innovate a little
  • Integrity – internal consistency – the brand is about what customers really touch – what reaches the customers – can a customer recognize that this is your experience, detail matter
To be successful an experience needs to be ownable and good. You need both – one is not enough. As a concluding example he showed Vodafone 360 – (which very few people in the audience knew). My short assessment is that the brand failed on “Differentiation” – to me (and obviously I would expect Josh Ulm is not agreeing) it is too similar to other things in this space…

As a further example of a strong Differentiation he showed Jeff Fong’s Metro UI for the new Windows mobile phone platform. If you have not seen the design check this out. It is very different from current Android and iPhone UIs. Everyone is really curious how it will do in the market…

>Keynote by Josh Ulm at Mobile HCI 2010

Posted on by
Reply
>Josh Ulm discussed how branding (and marketing in more general) has evolved and how this is now central to user experience and user interface design. He started out with showing how Nike changed marketing with the “Just Do It” campaign. He suggested that this was a transition from a product focus to a personal usage focus asking “who you become if you use the product“.

Moving towards more resent trends he argued that the iPod made the interaction with the product the essential part of the branding. With examples such as eBay and Google he showed that interaction in combination with information presentation becomes that discriminating factor and the way these brands define themselves. Overall this suggests that the user interaction and user experience is the central part for making a brand to stand out.

Standing out is not sufficient however. The experience has to be ownable. Using zappos as an example he showed how such an experience needs to be consistent across all touch points with the user – especially if you are defining your brand by experience – in short a brand has to have a unique user experience that is associated with the brand only. To achieve this there are three ingredients:
  • Values – “values have to extent into every single touchpoint of the experience”
  • Differentiation “you need to stand out in the market place” – many companies do not innovate enough, you have to take risks to stand out because you have to stand out a lot to be different – most companies do only innovate a little
  • Integrity – internal consistency – the brand is about what customers really touch – what reaches the customers – can a customer recognize that this is your experience, detail matter
To be successful an experience needs to be ownable and good. You need both – one is not enough. As a concluding example he showed Vodafone 360 – (which very few people in the audience knew). My short assessment is that the brand failed on “Differentiation” – to me (and obviously I would expect Josh Ulm is not agreeing) it is too similar to other things in this space…

As a further example of a strong Differentiation he showed Jeff Fong’s Metro UI for the new Windows mobile phone platform. If you have not seen the design check this out. It is very different from current Android and iPhone UIs. Everyone is really curious how it will do in the market…

Opening Keynote of Mobile HCI by Patrick Baudisch

Posted on by
Reply
Patrick Baudisch presented the opening Keynote. He gave an interesting overview of work on interaction with small screen devices. It is very apparent that currently the form factor of devices is nearly exclusively determined by the I/O components. Throughout the examples it became very explicit, that the usefulness of a device for certain tasks it determined by the options for interaction. E.g. by increasing the input precision or by reducing the fat finger problem new tasks (and eventually new applications) will become possible on mobile devices. Patrick showed an example where he asks his students, to design a UI for a device as small as a one Euro coin (e.g. digital jewelry) – I think I borrow this idea for our next course :-)

How showed many examples of his work – too many to write it in a blog entry – but well documented on his web page. It is a page worthwhile to check regularly as his group is outstandingly creative and productive.

Patrick made an insightful analogy between small screens and theater stage, arguing that the both have a similar problem: the lack space to show everything you would like to show due to a constraint space. The solutions in the theater can give a good inspiration for designing mobile UIs. He mentioned three typical solutions used in the theater:
  • Partially out of the frame – you show something only to a small part and the user’s imagination will fill in for the rest. In the theater an example is a ocean liner where you only see a very small part but people imagine the rest. The halo visualization technique [1] is one example for this approach on a mobile device.
  • In and out points – people enter and leave the stage at given points providing an illusion of a comprehensible world around the stage (and in the UI case around the screen)
  • Direction of interaction to invisible targets – by talking/pointing/looking into a certain direction actors can create an impression of interacting with someone at a certain location – who is not really there and who’s position is off the stage.
Technically I found the video he showed on using a depth cam extremely exciting. He speculated that this may solve many problems we have currently with using computer vision in difficult light conditions. In his example he showed the image recorded by a depth cam (based on time of flight) “filming” into direct sunlight – and as one would expect the sun does not matter. Looking forward to seeing some more results on this…

Looking at the current re-appearance of watch based computers and phones the idea of back side interaction becomes more and more interesting. In one design he showed how one can interact with a watch size device by interacting on buckle of the watch band. For more on back of device interaction see [2].

[1] Baudisch, P. and Rosenholtz, R. 2003. Halo: a technique for visualizing off-screen objects. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Ft. Lauderdale, Florida, USA, April 05 – 10, 2003). CHI ’03. ACM, New York, NY, 481-488. DOI= http://doi.acm.org/10.1145/642611.642695

[2] Baudisch, P. and Chu, G. 2009. Back-of-device interaction allows creating very small touch devices. In Proceedings of the 27th international Conference on Human Factors in Computing Systems (Boston, MA, USA, April 04 – 09, 2009). CHI ’09. ACM, New York, NY, 1923-1932. DOI= http://doi.acm.org/10.1145/1518701.1518995

>Opening Keynote of Mobile HCI by Patrick Baudisch

Posted on by
Reply
>Patrick Baudisch presented the opening Keynote. He gave an interesting overview of work on interaction with small screen devices. It is very apparent that currently the form factor of devices is nearly exclusively determined by the I/O components. Throughout the examples it became very explicit, that the usefulness of a device for certain tasks it determined by the options for interaction. E.g. by increasing the input precision or by reducing the fat finger problem new tasks (and eventually new applications) will become possible on mobile devices. Patrick showed an example where he asks his students, to design a UI for a device as small as a one Euro coin (e.g. digital jewelry) – I think I borrow this idea for our next course :-)

How showed many examples of his work – too many to write it in a blog entry – but well documented on his web page. It is a page worthwhile to check regularly as his group is outstandingly creative and productive.

Patrick made an insightful analogy between small screens and theater stage, arguing that the both have a similar problem: the lack space to show everything you would like to show due to a constraint space. The solutions in the theater can give a good inspiration for designing mobile UIs. He mentioned three typical solutions used in the theater:
  • Partially out of the frame – you show something only to a small part and the user’s imagination will fill in for the rest. In the theater an example is a ocean liner where you only see a very small part but people imagine the rest. The halo visualization technique [1] is one example for this approach on a mobile device.
  • In and out points – people enter and leave the stage at given points providing an illusion of a comprehensible world around the stage (and in the UI case around the screen)
  • Direction of interaction to invisible targets – by talking/pointing/looking into a certain direction actors can create an impression of interacting with someone at a certain location – who is not really there and who’s position is off the stage.
Technically I found the video he showed on using a depth cam extremely exciting. He speculated that this may solve many problems we have currently with using computer vision in difficult light conditions. In his example he showed the image recorded by a depth cam (based on time of flight) “filming” into direct sunlight – and as one would expect the sun does not matter. Looking forward to seeing some more results on this…

Looking at the current re-appearance of watch based computers and phones the idea of back side interaction becomes more and more interesting. In one design he showed how one can interact with a watch size device by interacting on buckle of the watch band. For more on back of device interaction see [2].

[1] Baudisch, P. and Rosenholtz, R. 2003. Halo: a technique for visualizing off-screen objects. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Ft. Lauderdale, Florida, USA, April 05 – 10, 2003). CHI ’03. ACM, New York, NY, 481-488. DOI= http://doi.acm.org/10.1145/642611.642695

[2] Baudisch, P. and Chu, G. 2009. Back-of-device interaction allows creating very small touch devices. In Proceedings of the 27th international Conference on Human Factors in Computing Systems (Boston, MA, USA, April 04 – 09, 2009). CHI ’09. ACM, New York, NY, 1923-1932. DOI= http://doi.acm.org/10.1145/1518701.1518995

SiMPE 2010, Keynote: Trends and Challenges in Mobile Interaction

Posted on by
Reply
I was invited to give a keynote talk at the 5th Workshop on Speech in Mobile and Pervasive Environments that was held as a part of ACM MobileHCI 2010 in Lisbon, Portugal. Over the last years we looked at speech as an additional modality in the automotive user interface domain; besides this my experience with speech interfaces is limited.

My talk, with the title “Trends and Challenges in Mobile Interaction” looked at different issues in mobile interaction. In some parts I reflected on modalities and opportunities for speech interaction.

When characterizing mobile interaction I pointed out the following points:
  • Interacting while on the move
  • Interaction is one of the user’s tasks (besides others – e.g. walking, standing in a crowd)
  • Environment in which the interaction takes place changes (e.g. on the train with varying light and noise conditions)
  • Interruptions happen frequently (e.g. boarding the bus, crossing the road)
  • Application usage is short (typically seconds to minutes)
My small set of key issues and recommendations for mobile UIs is:
  • Simple and Understandable
  • Perceptive and Context-Aware
  • Unobtrusive, Embedded and Integrated
  • Low Cognitive Load and Peripheral Usage
  • Users want to be in Control (especially on the move)
The presentations and discussion at the workshop were very interesting and I got a number of ideas for multimodal user interfaces – including speech.

One issue that made me think more was the question about natural language speech vs. specific speech commands. A colleague pointed me to Speech Graffiti [1] / Universal Speech Interface at CMU. I wonder if it would make sense to invent a Human Computer Interaction language (with a simple grammar and a vocabulary) that we could teach in a course over several weeks (e.g. similar effort than touch typing on a QUERTY keyboard) or as a foreign language at school to have a new effective means for interaction. Could this make us more effictive in interacting with information? Or should we try harder to get natural languge interaction working? Looking at the way (experienced) people use Google we can see that people adapt very successfully – probably faster than systems improve…

From some of the talks it seems that “Push to talk” seems to be a real issue for users and a reason for many user related errors in speech systems. Users do not push at the appropriate time, especially when there are other tasks to do, and hence utterances are cut off at the start and end. I would guess continuous recording of the speech and using the “push to talk” only as an indicator where to search in the audio stream may be a solution.

[1] Tomko, S. and Rosenfeld, R. 2004. Speech graffiti vs. natural language: assessing the user experience. In Proceedings of HLT-NAACL 2004: Short Papers (Boston, Massachusetts, May 02 – 07, 2004). Human Language Technology Conference. Association for Computational Linguistics, Morristown, NJ, 73-76. http://www.cs.cmu.edu/~usi/papers/HLT04.pdf