Gesture UI/Thracker - a simple hand tracking device

Project description

Thracker is a 2-dimensional (eventually 3-dimensional) USB input device for personal computers. The mouse cursor follows the user's hand movements in front of the screen. Thracker uses some kind of electrical field sensing utilizing four "antenna" plates positioned around the screen. It is not required to wear any kind of transmitter/tracking point. Thracker is a student project ("Projektarbeit") at the LFE Media Informatics of the Ludwig-Maximilians-University Munich.

Status and Ongoing Work

Currently the hardware part is (nearly) finished. I'm working on Java classes and a demo application.

Uses

No usability tests have been conducted so far. Also recognition accuracy is not determined yet. However some possible uses might be:

Thracker is ideal for contact less operation of a computer like e.g. in a OP.
Thracker could be used for gesture recognition.
Three-dimensional tracking might be possible. This could provide an easy and intuitive way for manipulating 3D objects.

Problems

Resolution is not adequate for fine manipulation. The theoretical resolution of Thracker is 512x512. The practically reachable resolution is about 150x150. This resolution however should be sufficient for gesture recognition and simple sorting tasks.
Like the human hand the metal housing of TFT and CRT displays works also like a second capacitor plate. This attenuates the resolution enormously when the antennas are close to the screen. A solution is to use thin metal plates as antennas which are installed orthogonally to the display. This reduces the effect.
The antenna plates "sense" in two directions - towards the screen and away from it. Moving capacitive objects like humans passing by can distort the measurements.
Body movements like leaning forward can influence the measurements.

Physical Principles (Theremin)

Thracker is based on a principle employed by the Theremin, a music instrument named after its inventor Leon Theremin. A Theremin has two orthogonally oriented antennas. By moving his hand closer to an antenna or farther away, the musician can change the pitch of the generated tone (sound sample). The musicians hand acts as a counterpart to the antenna. Both form a capacitor. Moving the hand closer to the antenna increases the capacitors capacity. This in turn decreases the frequency of a resonant circuit.

Technical Implementation

The Thracker board converts the frequency of four such resonant circuits (placed on the four sides of the screen) in discrete values. Those values are then read out via USB. The resonant circuit uses a NAND-Gate of the 4093 IC. The frequency of the resonant circuit is ca. 100 kHz, decreasing when the hand approaches the antenna. This signal triggers a 4020 binary counter. Eight (higher) output bits are copied into a latch (IC 573). The copy is triggered by a 555 timer IC which produces a 50 Hz signal. The USB interface chip (IO-Warrior 40) reads the values from the latch. Software polls the IO-Warrior >50 times per second. As the antenna resolution is quite non-linear, the input device has to be calibrated. This creates value-to-distance lookup tables. With the distance data for each antenna, the position of the hand can be calculated.

Hardware Assembly

Thracker Eagle Files: thracker.brd thracker.sch (in Eagle format) and as PDF (slightly outdated, use only as reference for assembly): thracker.brd.pdf thracker.sch.pdf.

Parts list

1x Thracker board
1x IO-Warrior 40, DIL
1x 40 pin chip socket for IO-Warrior
1x 6 MHz ceramic resonator
1x USB B socket
1x three pin header
4x 1-pin header
4x 330k resistor
2x 100k resistor
1x 7k5 resistor
1x 5k5 resistor
1x 12R resistor
10nF capacitor
100nF capacitor
1µF capacitor
4.7µF capacitor
4x IC 4093, SOIC
4x IC 4020, SOIC
4x IC 573, SOIC
1x IC 555, SOIC

Assembly Instructions

Start by soldering in the 13 SOIC chips.
CAUTION: The carvings of the 4020 ICs face in the right direction. All other ICs face in the left direction.

Then solder in the pin headers, the chip socket and the USB socket.

Solder in all 9 resistors.

Finally solder in the 4 capacitors. Ignore the four capacitors with value "unused".

Put a jumper on pins 1-2 of the 3-pin-header. This selects low power mode for the USB device.

Finished board (earlier revision)

Software

A Java package org.hcilab.thracker is available soon.
~~A mouse driver for X11 (and possibly Windows) is planned.~~

Contact

Contact me at raphael@hcilab.org

Raphael Wimmer