Keyboard I/O hack

During my research visit at Microsoft Research in Cambridge with Steve Hodges group I spend a lot of time building things and programming (Windows Phone 7, C#, microcontroller…) – which I really enjoyed!

In this post I want to share a mini project, I did together with Nic Villar between two coffees … it reminded me somehow of the Friday afternoon projects (=2h projects you do because they are fun and sometimes you learn something) while I did my PhD.

Perhaps some context first: working on Microsoft .NET Gadegteer we discussed many ideas of how to ease code/system creation for physical and embedded computing. Looking into it I came across a number of interesting approaches for software development that runs in the browser. Examples include: http://ideone.com/ (a online compiler for a large number of languages), http://wonderfl.net/ is a online development environment for Flash that allows to build on other people’s code by forking their project, and http://mbed.org/ is a online tool for rapid prototyping of a microcontroller system.

Now to the mini project: a keyboard hack that supports input as well as output. You find many example of keyboard hacks on the web – basically you take out the PCB of the keyboard an replace the keys with something more interesting… here is ours that supports input (which is common) as well as output – from the computer (or web browser) to the environment. Using AQW210EH allows more flexibility in what you drive and what you create the inputs with. The attractive thing in using a keyboard is that it will work without a driver – you just plug it in and it works.

To control it I wrote a java applet and some JavaScript code. Basically the java-applet is controlling the Scroll-Lock LED and from the website this is called via JavaScript (try here if it works on your computer – if it works your CAPS, NUM, and SCROLL LED will blink once the web page is open). I tried it on some Windows machines and it worked well – the java applet seems not to work on a Mac (so you probably have to find another way to set you Scroll Lock LED programmatically). You can extend the output to CAPSLOCK and NUMLOCK – but these are sometimes used – in contrast to the SCROLL LOCK. Input is simple – by connecting one of the rows to one of the colloms you generate a letter comes in as if you would type it. The nice thing with USB is that you can have multiple keyboards connected at the same time (however the LEDs are synchronized between them).

In C# you can use the SendKeys.Send(“{SCROLLLOCK}”) command (for details see: http://msdn.microsoft.com/en-us/library/system.windows.forms.sendkeys.send.aspx,
http://support.microsoft.com/Default.aspx?id=177674) The SendKeys.Send may only give an pulse to the LED and hence you may need to call a system function.

Using a JavaApplet it looks like this.

import java.applet.*;
import java.awt.*;
import java.awt.Toolkit;
import java.awt.event.KeyEvent;
public class keyLed extends Applet {
Font f = new Font("TimesRoman",Font.BOLD,20);
String strMessage;
Toolkit tk = Toolkit.getDefaultToolkit();
public void init() {
setBackground(Color.white);
setStrMessage("Applet loaded.");
}
public void paint(Graphics g) {
g.setFont(f);
g.drawString(this.strMessage, 100 , 25);
}
public void ScrollLedOn() {
tk.setLockingKeyState(KeyEvent.VK_SCROLL_LOCK,true);
setStrMessage("Scroll on");
}
public void ScrollLedOff() {
tk.setLockingKeyState(KeyEvent.VK_SCROLL_LOCK,false);
setStrMessage("Scroll off");
}
}

And in HTML/JavaScript you call it:

"." CODE="keyLed.class" WIDTH=300 HEIGHT=50 NAME="keyLedApplet">

>Keyboard I/O hack

>During my research visit at Microsoft Research in Cambridge with Steve Hodges group I spend a lot of time building things and programming (Windows Phone 7, C#, microcontroller…) – which I really enjoyed!

In this post I want to share a mini project, I did together with Nic Villar between two coffees … it reminded me somehow of the Friday afternoon projects (=2h projects you do because they are fun and sometimes you learn something) while I did my PhD.

Perhaps some context first: working on Microsoft .NET Gadegteer we discussed many ideas of how to ease code/system creation for physical and embedded computing. Looking into it I came across a number of interesting approaches for software development that runs in the browser. Examples include: http://ideone.com/ (a online compiler for a large number of languages), http://wonderfl.net/ is a online development environment for Flash that allows to build on other people’s code by forking their project, and http://mbed.org/ is a online tool for rapid prototyping of a microcontroller system.

Now to the mini project: a keyboard hack that supports input as well as output. You find many example of keyboard hacks on the web – basically you take out the PCB of the keyboard an replace the keys with something more interesting… here is ours that supports input (which is common) as well as output – from the computer (or web browser) to the environment. Using AQW210EH allows more flexibility in what you drive and what you create the inputs with. The attractive thing in using a keyboard is that it will work without a driver – you just plug it in and it works.

To control it I wrote a java applet and some JavaScript code. Basically the java-applet is controlling the Scroll-Lock LED and from the website this is called via JavaScript (try here if it works on your computer – if it works your CAPS, NUM, and SCROLL LED will blink once the web page is open). I tried it on some Windows machines and it worked well – the java applet seems not to work on a Mac (so you probably have to find another way to set you Scroll Lock LED programmatically). You can extend the output to CAPSLOCK and NUMLOCK – but these are sometimes used – in contrast to the SCROLL LOCK. Input is simple – by connecting one of the rows to one of the colloms you generate a letter comes in as if you would type it. The nice thing with USB is that you can have multiple keyboards connected at the same time (however the LEDs are synchronized between them).

In C# you can use the SendKeys.Send(“{SCROLLLOCK}”) command (for details see: http://msdn.microsoft.com/en-us/library/system.windows.forms.sendkeys.send.aspx,
http://support.microsoft.com/Default.aspx?id=177674) The SendKeys.Send may only give an pulse to the LED and hence you may need to call a system function.

Using a JavaApplet it looks like this.

import java.applet.*;
import java.awt.*;
import java.awt.Toolkit;
import java.awt.event.KeyEvent;
public class keyLed extends Applet {
Font f = new Font("TimesRoman",Font.BOLD,20);
String strMessage;
Toolkit tk = Toolkit.getDefaultToolkit();
public void init() {
setBackground(Color.white);
setStrMessage("Applet loaded.");
}
public void paint(Graphics g) {
g.setFont(f);
g.drawString(this.strMessage, 100 , 25);
}
public void ScrollLedOn() {
tk.setLockingKeyState(KeyEvent.VK_SCROLL_LOCK,true);
setStrMessage("Scroll on");
}
public void ScrollLedOff() {
tk.setLockingKeyState(KeyEvent.VK_SCROLL_LOCK,false);
setStrMessage("Scroll off");
}
}

And in HTML/JavaScript you call it:

<APPLET CODEBASE="." CODE="keyLed.class" WIDTH=300 HEIGHT=50 NAME="keyLedApplet">
</APPLET>
<script language="JavaScript" type="text/javascript"> 
function LEDOn()
{
document.keyLedApplet.ScrollLedOn();
}
 
function LEDOff()
{
document.keyLedApplet.ScrollLedOff();
}
 </script>

Keyboard I/O hack

During my research visit at Microsoft Research in Cambridge with Steve Hodges group I spend a lot of time building things and programming (Windows Phone 7, C#, microcontroller…) – which I really enjoyed!

In this post I want to share a mini project, I did together with Nic Villar between two coffees … it reminded me somehow of the Friday afternoon projects (=2h projects you do because they are fun and sometimes you learn something) while I did my PhD.

Perhaps some context first: working on Microsoft .NET Gadegteer we discussed many ideas of how to ease code/system creation for physical and embedded computing. Looking into it I came across a number of interesting approaches for software development that runs in the browser. Examples include: http://ideone.com/ (a online compiler for a large number of languages), http://wonderfl.net/ is a online development environment for Flash that allows to build on other people’s code by forking their project, and http://mbed.org/ is a online tool for rapid prototyping of a microcontroller system.

Now to the mini project: a keyboard hack that supports input as well as output. You find many example of keyboard hacks on the web – basically you take out the PCB of the keyboard an replace the keys with something more interesting… here is ours that supports input (which is common) as well as output – from the computer (or web browser) to the environment. Using AQW210EH allows more flexibility in what you drive and what you create the inputs with. The attractive thing in using a keyboard is that it will work without a driver – you just plug it in and it works.

To control it I wrote a java applet and some JavaScript code. Basically the java-applet is controlling the Scroll-Lock LED and from the website this is called via JavaScript (try here if it works on your computer – if it works your CAPS, NUM, and SCROLL LED will blink once the web page is open). I tried it on some Windows machines and it worked well – the java applet seems not to work on a Mac (so you probably have to find another way to set you Scroll Lock LED programmatically). You can extend the output to CAPSLOCK and NUMLOCK – but these are sometimes used – in contrast to the SCROLL LOCK. Input is simple – by connecting one of the rows to one of the colloms you generate a letter comes in as if you would type it. The nice thing with USB is that you can have multiple keyboards connected at the same time (however the LEDs are synchronized between them).

In C# you can use the SendKeys.Send(“{SCROLLLOCK}”) command (for details see: http://msdn.microsoft.com/en-us/library/system.windows.forms.sendkeys.send.aspx,
http://support.microsoft.com/Default.aspx?id=177674) The SendKeys.Send may only give an pulse to the LED and hence you may need to call a system function.

Using a JavaApplet it looks like this.

import java.applet.*;
import java.awt.*;
import java.awt.Toolkit;
import java.awt.event.KeyEvent;
public class keyLed extends Applet {
Font f = new Font("TimesRoman",Font.BOLD,20);
String strMessage;
Toolkit tk = Toolkit.getDefaultToolkit();
public void init() {
setBackground(Color.white);
setStrMessage("Applet loaded.");
}
public void paint(Graphics g) {
g.setFont(f);
g.drawString(this.strMessage, 100 , 25);
}
public void ScrollLedOn() {
tk.setLockingKeyState(KeyEvent.VK_SCROLL_LOCK,true);
setStrMessage("Scroll on");
}
public void ScrollLedOff() {
tk.setLockingKeyState(KeyEvent.VK_SCROLL_LOCK,false);
setStrMessage("Scroll off");
}
}

And in HTML/JavaScript you call it:

"." CODE="keyLed.class" WIDTH=300 HEIGHT=50 NAME="keyLedApplet">

Tangible User Interface – Understandable, Ergonomic, Aesthetic

As I am always on the lookout for user interfaces it was delightful to see the bridge and the engine room of the Queen Mary with truly tangible man-machine-interfaces. Large wheels and all sorts of knobs, cranks, and levers are there to operate the many functions. At a first glance they look nice, intuitive, and easy to use – but are they? Looking closer it becomes clear that many user interface elements were built primarily to enable mechanical control and not because of ergonomic considerations. The size of physical controls (e.g. the length of a lever) is often due to the fact that certain forces must be created to control the machine. The size and visibility of the mechanical properties are helpful for the user to understand the operation and to some extent foresee the impact of the operation. At the same time many of these user interface elements make the operation very visible to co-workers – you do not have to tell that you change the speed as everyone around you can clearly see it from your actions. It seems that ergonomic properties are of lesser importance in many of these controls, e.g. some controls require strong forces or large physical movements. All in all I would conclude that these tangible controls (that are designed due to physical constraints) are helping with the understandability of a user interface but are not necessarily a good model for creating ergonomic controls.


In some recent examples of tangible user interfaces I feel that people took the worst from both worlds – they argue for the physical controls – and all they get are less ergonomic UIs (e.g. needs more forces, movements, etc.) which are less understandable as the physical constraints do not map to the constraints of the digital system… Hence I think it is important to keep this in mind: the physicality of the controls should be used to make interaction understandable and the design should not compromise ergonomics (as we do not live in the mechanic era anymore).

One further thing that can be learned for these mechanical interfaces is the beauty of the design and implementation. Some are in shinny and polished metal, others are nicely decorated, and on others it is just pleasant to touch the wood. Beauty and esthetical qualities play a major role – and we know this for screen based UIs from Noam Tractinsky’s work [1,2] as well as from the market success of devices like the iPhone.

[1] Noam Tractinsky. 1997. Aesthetics and apparent usability: empirically assessing cultural and methodological issues. In Proceedings of the SIGCHI conference on Human factors in computing systems (CHI ’97), Steven Pemberton (Ed.). ACM, New York, NY, USA, 115-122. DOI=10.1145/258549.258626 http://doi.acm.org/10.1145/258549.258626

[2] Tractinsky, N., Shoval-Katz A. and Ikar, D. (2000) What is Beautiful is Usable. Interacting with Computers, 13(2): 127-145.

>Tangible User Interface – Understandable, Ergonomic, Aesthetic

>As I am always on the lookout for user interfaces it was delightful to see the bridge and the engine room of the Queen Mary with truly tangible man-machine-interfaces. Large wheels and all sorts of knobs, cranks, and levers are there to operate the many functions. At a first glance they look nice, intuitive, and easy to use – but are they? Looking closer it becomes clear that many user interface elements were built primarily to enable mechanical control and not because of ergonomic considerations. The size of physical controls (e.g. the length of a lever) is often due to the fact that certain forces must be created to control the machine. The size and visibility of the mechanical properties are helpful for the user to understand the operation and to some extent foresee the impact of the operation. At the same time many of these user interface elements make the operation very visible to co-workers – you do not have to tell that you change the speed as everyone around you can clearly see it from your actions. It seems that ergonomic properties are of lesser importance in many of these controls, e.g. some controls require strong forces or large physical movements. All in all I would conclude that these tangible controls (that are designed due to physical constraints) are helping with the understandability of a user interface but are not necessarily a good model for creating ergonomic controls.


In some recent examples of tangible user interfaces I feel that people took the worst from both worlds – they argue for the physical controls – and all they get are less ergonomic UIs (e.g. needs more forces, movements, etc.) which are less understandable as the physical constraints do not map to the constraints of the digital system… Hence I think it is important to keep this in mind: the physicality of the controls should be used to make interaction understandable and the design should not compromise ergonomics (as we do not live in the mechanic era anymore).

One further thing that can be learned for these mechanical interfaces is the beauty of the design and implementation. Some are in shinny and polished metal, others are nicely decorated, and on others it is just pleasant to touch the wood. Beauty and esthetical qualities play a major role – and we know this for screen based UIs from Noam Tractinsky’s work [1,2] as well as from the market success of devices like the iPhone.

[1] Noam Tractinsky. 1997. Aesthetics and apparent usability: empirically assessing cultural and methodological issues. In Proceedings of the SIGCHI conference on Human factors in computing systems (CHI ’97), Steven Pemberton (Ed.). ACM, New York, NY, USA, 115-122. DOI=10.1145/258549.258626 http://doi.acm.org/10.1145/258549.258626

[2] Tractinsky, N., Shoval-Katz A. and Ikar, D. (2000) What is Beautiful is Usable. Interacting with Computers, 13(2): 127-145.

Tangible User Interface – Understandable, Ergonomic, Aesthetic

As I am always on the lookout for user interfaces it was delightful to see the bridge and the engine room of the Queen Mary with truly tangible man-machine-interfaces. Large wheels and all sorts of knobs, cranks, and levers are there to operate the many functions. At a first glance they look nice, intuitive, and easy to use – but are they? Looking closer it becomes clear that many user interface elements were built primarily to enable mechanical control and not because of ergonomic considerations. The size of physical controls (e.g. the length of a lever) is often due to the fact that certain forces must be created to control the machine. The size and visibility of the mechanical properties are helpful for the user to understand the operation and to some extent foresee the impact of the operation. At the same time many of these user interface elements make the operation very visible to co-workers – you do not have to tell that you change the speed as everyone around you can clearly see it from your actions. It seems that ergonomic properties are of lesser importance in many of these controls, e.g. some controls require strong forces or large physical movements. All in all I would conclude that these tangible controls (that are designed due to physical constraints) are helping with the understandability of a user interface but are not necessarily a good model for creating ergonomic controls.


In some recent examples of tangible user interfaces I feel that people took the worst from both worlds – they argue for the physical controls – and all they get are less ergonomic UIs (e.g. needs more forces, movements, etc.) which are less understandable as the physical constraints do not map to the constraints of the digital system… Hence I think it is important to keep this in mind: the physicality of the controls should be used to make interaction understandable and the design should not compromise ergonomics (as we do not live in the mechanic era anymore).

One further thing that can be learned for these mechanical interfaces is the beauty of the design and implementation. Some are in shinny and polished metal, others are nicely decorated, and on others it is just pleasant to touch the wood. Beauty and esthetical qualities play a major role – and we know this for screen based UIs from Noam Tractinsky’s work [1,2] as well as from the market success of devices like the iPhone.

[1] Noam Tractinsky. 1997. Aesthetics and apparent usability: empirically assessing cultural and methodological issues. In Proceedings of the SIGCHI conference on Human factors in computing systems (CHI ’97), Steven Pemberton (Ed.). ACM, New York, NY, USA, 115-122. DOI=10.1145/258549.258626 http://doi.acm.org/10.1145/258549.258626

[2] Tractinsky, N., Shoval-Katz A. and Ikar, D. (2000) What is Beautiful is Usable. Interacting with Computers, 13(2): 127-145.

Random Links for Scientific Search in CS

Scientific search sides:
http://scholar.google.de/
http://www.confsearch.org
http://academic.research.microsoft.com/
http://arnetminer.org/

Digital libraries:
http://acm.org/dl
http://ieeexplore.ieee.org
http://www.springerlink.com/

Listings of publications, co-authors, and relationships:
http://www.informatik.uni-trier.de/~ley/db/index.html
http://dblp.mpi-inf.mpg.de/dblp/index.php

Random Links for Scientific Search in CS

Scientific search sides:
http://scholar.google.de/
http://www.confsearch.org
http://academic.research.microsoft.com/
http://arnetminer.org/

Digital libraries:
http://acm.org/dl
http://ieeexplore.ieee.org
http://www.springerlink.com/

Listings of publications, co-authors, and relationships:
http://www.informatik.uni-trier.de/~ley/db/index.html
http://dblp.mpi-inf.mpg.de/dblp/index.php

>Random Links for Scientific Search in CS

>Scientific search sides:
http://scholar.google.de/
http://www.confsearch.org
http://academic.research.microsoft.com/
http://arnetminer.org/

Digital libraries:
http://acm.org/dl
http://ieeexplore.ieee.org
http://www.springerlink.com/

Listings of publications, co-authors, and relationships:
http://www.informatik.uni-trier.de/~ley/db/index.html
http://dblp.mpi-inf.mpg.de/dblp/index.php

Interviewing with Rikke Friis Dam and Mads Søgaard

Rikke Friis Dam and Mads Søgaard are currently working on a re-launch of the website interaction-design.org. The side has over the last years involved in a useful resource for researchers and practitioners in human computer interaction and interaction design. There is a very comprehensive calendar that includes most relevant events in HCI.


With their current work Rikke and Mads pursue a mission to create a new and free resource for teaching and learning interaction design and HCI. In a first step they work with researchers (like myself) around the world that are experts on a certain topic (in my case context-awareness and implicit interaction) to create new teaching materials. This includes a chapter (about 3000 words) that has tutorial character and interviews in which specific topics are discussed in more details.

It was great fun to work with them and I look forward to seeing the new material online.