Perci (PERvasive ServiCe Interaction)

The Perci (PERvasive ServiCe Interaction) project is funded by NTT DoCoMo Euro-Labs . The goal of the project is to investigate and develop new methods for mobile interactions with the Internet of Things.

We are currently looking for students who are interested in doing their project or diploma thesis within the Perci Project. For more details on open theses topics, please contact Gregor Broll.

Motivation

Within this project we develop methods and tools to ease the process of creating real world user interfaces for Semantic Web Services. We focus on implicit and explicit interaction and consider the personal mobile device as the central means for user-service-interaction. The overall vision is that Semantic Web Services can be connected to or implemented in augmented physical objects and that real world user interfaces can be derived (semi-)automatically from the service specification. These paradigms and development concepts to derived ubiquitous interfaces could facilitate user interaction with the service, speed up development and ultimately shorten the time to market for pervasive services. Novel applications could become immediately available to users with minimal need of expensive and time consuming UI programming. More technically, we construct such end-user services out of a set of Semantic Web Services. The definition of an intermediate abstract user interface layer that is derived (semi-)automatically from the service description. Subsequently, abstract UIs are used as the basis for the concrete user interfaces creation.

Interaction with real world objects

Physical objects in the real world or locations may be natural, enhanced with minimal technology, or may contain a computer. A possible scenario is this: a person is pointing with mobile device with an embedded camera at a marker on an advertisement in a fashion journal. This interaction allows connecting to the web service that is related to ordering these articles. Another scenario could be a Bluetooth-based interaction between a mobile device and a public display using the personal display as a controller.

Such forms of interaction are feasible with current technology and many of them are implemented in research or products. However, today creating such service-interactions is done manually. The developer specifies and programs the interaction. This process is time consuming and does not scale to a vision where services are tailored to individuals, as it is not economic to manually create a fitting interfaces and interaction behavior for each instance.

One important research issue is how services that are directly or indirectly related to real world objects could be described and linked. This relationship is general a many to many relation, where several augmented real world objects can be related to a number of services. The services related to or provided by different real world objects are end-user services that may be a collection or combination of services available. Furthermore there are diverse users with different devices that want to interact with the physical world.

To describe and specify the connection between end-user-services and user interface components it appears feasible to use techniques developed in the area of semantic web technologies. Typical interactive functions that should be supported are:

advertising and discovering services,
access to a detailed description of a service's operation
use of combined services

Interactions with the real world and with objects in the physical environment provide opportunities for new applications and services. It is important to understand what novel forms of interaction, what metaphors, and user interface primitives are available to create a platform to support these applications and services. In particular the we address the following issues within the project :

What interaction relationships are common between users, locations, and objects in the real world (e.g. user sees an object, user approaches an object, user handles an object)?
How are objects in the real world linked to information in the virtual domain and what type of relationships are these (e.g. information related to an object, information related to a class of objects, information linked to a region)?
What are basic mechanisms for real world interaction (e.g. getting identity of an object, determine type/class of an object, control based on an object, controlling additional devices)?
What technologies can be used to perform real world interaction with mobile phones (e.g. visual markers, bar code, RFID-tags, image recognition, location/position, movement)?
What is a basic set of operations and widgets for real world interaction that application programmers could rely on independent of the technologies used?
What is an abstract representation for user interfaces capabilities with regard to real world interaction with mobile devices.

Research goals

The Perci project investigates the potential of real world interaction with mobile devices in the context of Semantic Web Services. It will provide an understanding of how generic interaction can be abstracted, modeled, and implemented. An important aspect herby is the relationship between the offered service described by Semantic Web Technologies and the concrete interaction performed by the user.

People

The following persons of the embedded interaction group are directly involved into the project:

Albrecht Schmidt : Head of the group (albrecht @ hcilab . org)
Enrico Rukzio : Research Assistant (enrico @ hcilab . org)
Gregor Broll: Research Assistant (gregor.broll @ ifi.lmu.de)
Thorsten Buering: Postdoc (thorsten.buering @ ifi.lmu.de)
Sven Siorpaes: Student
Diploma thesis: A Physical Mobile Interactions Framework based on Semantic Descriptions
Supervisors: M. Paolucci, J. Hamard (NTT DoCoMo), E. Rukzio, A. Schmidt (ei)
Gregor Broll: Student
Diploma thesis: Mobile Interaction with Real World Services
Supervisors: M. Paolucci, J. Hamard (NTT DoCoMo), E. Rukzio, A. Schmidt (ei)
Jessica Aust: Student
Project thesis: Security and Usability Aspects of NFC based Mobile Interactions
Supervisors: M. Paolucci, J. Hamard (NTT DoCoMo), E. Rukzio, A. Schmidt (ei)
Johannes Vetter: Student
Project thesis: Near Field Communication based Mobile Interaction Techniques
Supervisors: M. Paolucci, J. Hamard (NTT DoCoMo), E. Rukzio, A. Schmidt (ei)
Christian Braun : Student
Project thesis: Authoring Tool for Mobile Interactions with the Internet of Things
Supervisors: M. Paolucci, J. Hamard (NTT DoCoMo), E. Rukzio, G. Broll, A. Schmidt (ei)
Andreas Obergfell : Student
Project thesis: Real World Interfaces for Mobile Service Interaction
Supervisors: G. Broll, M. Paolucci, J. Hamard (NTT DoCoMo)
Markus Haarlaender : Student
Diploma thesis: Advanced Mobile Interaction with the Real World
Supervisors: G. Broll, M. Paolucci, J. Hamard (NTT DoCoMo)
Rene Klueber : Student
Project thesis: Towards Design and Application of Multi-Tag Interaction with Information from the Real World
Supervisors: G. Broll, M. Paolucci, J. Hamard (NTT DoCoMo)

Open Project and Diploma Thesis

We are currently looking for students who are interested in doing their project or diploma thesis within the Perci Project. For more details on open theses topics, please contact Gregor Broll.

Publications and Demos

Massimo Paolucci, Gregor Broll, John Hamard, Enrico Rukzio, Matthias Wagner, Albrecht Schmidt
Bringing Semantic Services to Real World Objects
Accepted for the International Journal on Semantic Web and Information Systems 2008.

Johannes Vetter, John Hamard, Massimo Paolucci, Enrico Rukzio, Albrecht Schmidt
Physical Mobile Interaction with Dynamic Physical Objects
Demo at the 9th International Conference on Human Computer Interaction with Mobile Devices and Services (MobileHCI) 2007, Singapore, September 9, 2007.

Enrico Rukzio, Gregor Broll, Karin Leichtenstern, Albrecht Schmidt
Mobile Interaction with the Real World: An Evaluation and Comparison of Physical Mobile Interaction Techniques (PDF)
AmI-07: European Conference on Ambient Intelligence, Darmstadt, Germany, 7-10 November 2007.

Gregor Broll, John Hamard, Massimo Paolucci, Markus Haarlaender, Matthias Wagner, Sven Siorpaes, Enrico Rukzio, Albrecht Schmidt, Kevin Wiesner
Mobile Interaction with Web Services through Associated Real World Objects (PDF)
Demo at the 9th International Conference on Human Computer Interaction with Mobile Devices and Services (MobileHCI) 2007, Singapore, September 9, 2007.

Gregor Broll, Alexander De Luca, Enrico Rukzio, Chie Noda, Paul Wisner
Mobile Interaction with the Real World (MIRW) 2007
Workshop in conjunction with the 9th International Conference on Human Computer Interaction with Mobile Devices and Services (MobileHCI) 2007, Singapore, September 9, 2007.

Gregor Broll, Sven Siorpaes, Enrico Rukzio, Massimo Paolucci, John Hamard, Matthias Wagner, Albrecht Schmidt
Comparing Techniques for Mobile Interaction with Objects from the Real World (PDF)
Workshop Permid 2007 in conjunction with Pervasive 2007, Toronto, Ontario, Canada, May 13 2007.

Massimo Paolucci, Gregor Broll, John Hamard, Enrico Rukzio, Albrecht Schmidt, Matthias Wagner.
PERCI: Delivering services trough real world objects
In Proceedings of the 1st European NFC Competition "Touching the Future" in conj. with WIMA 2007, Monaco, April 2007.

Gregor Broll, Sven Siorpaes, Enrico Rukzio, Massimo Paolucci, John Hamard, Matthias Wagner, Albrecht Schmidt
Supporting Mobile Service Usage through Physical Mobile Interaction (PDF)
5th Annual IEEE International Conference on Pervasive Computing and Communications, White Plains, NY, USA, March 19 - 23, 2007.

Gregor Broll, Sven Siorpaes, Enrico Rukzio, Massimo Paolucci, John Hamard, Matthias Wagner, Albrecht Schmidt
Supporting Mobile Service Usage through Physical Mobile Interaction (PDF)
Demo at the 5th Annual IEEE International Conference on Pervasive Computing and Communications, White Plains, NY, USA, March 19 - 23, 2007. WINNER OF 2007 IEEE PERCOM BEST DEMO AWARD

Gregor Broll, Sven Siorpaes, Enrico Rukzio, Massimo Paolucci, John Hamard, Matthias Wagner, Albrecht Schmidt
Supporting Mobile Service Interaction through Semantic Service Description Annotation and Automatic Interface Generation (PDF)
Semantic Desktop and Social Semantic Collaboration Workshop at the International Semantic Web Conference, 6 November 2006, Athens, GA, USA

Enrico Rukzio, Massimo Paolucci, Tim Finin, Paul Wisner, Terry Payne.
Mobile Interaction with the Real World. (PDF)
Workshop @ Mobile HCI 2006. In Proceedings of 8th International Conference on Human Computer Interaction with Mobile Devices and Services, Espoo, Finland, 12 to 15 September 2006.

Enrico Rukzio, Massimo Paolucci, Matthias Wagner, Hendrik H. Berndt, John Hamard, Albrecht Schmidt
Mobile Service Interaction with the Web of Things. ( PDF )
13th International Conference on Telecommunications (ICT 2006), Funchal, Madeira island, Portugal, May 9-12 2006.

Sven Siorpaes, Gregor Broll, Massimo Paolucci, Enrico Rukzio, John Hamard, Matthias Wagner, Albrecht Schmidt
Mobile Interaction with the Internet of Things (PDF)
Late Breaking Results & Posters in conjunction with Pervasive 2006, Dublin, Ireland, May 8 2006.

Gregor Broll, Sven Siorpaes, Enrico Rukzio, Massimo Paolucci, John Hamard, Matthias Wagner, Albrecht Schmidt
Supporting Service Interaction in the Real World (PDF)
Workshop Permid 2006 in conjunction with Pervasive 2006, Dublin, Ireland, May 7 2006.

Talks

John Hamard (NTT DoCoMo): PERvasive serviCe Interaction. October 14 and 15, Workshop Near field interactions @ Nordichi 2006.

Massimo Paolucci (NTT DoCoMo): Bringing Web services to Mobile Environments. Invited talk at the 4th International Workshop on AI for Service Composition @ ECAI 2006. August 28 2006.

Enrico Rukzio (ei): Perci (PERvasive ServiCe Interaction). Invited talk at the Second International Workshop on Semantic Web Technology for Ubiquitous and Mobile Applications (SWUMA'06) @ ECAI 2006. August 28 2006.

Matthias Wagner (NTT DoCoMo): PERCI - Pervasive Service Interaction in the Web of Things. June 6 2006, Workshop UC and RFID today – Breakthrough or still on hold? @ ETRICS 2006.

Workshops

The Perci - project was involved in the organization of the Mobile HCI 2006 workshop Mobile Interaction with the Real World.