Session 1: Augmenting Finger and Forearm (5/6)

Whole Hand Modeling using 8 Wearable Sensors: Biomechanics for Hand Pose Prediction full paper

Christopher-Eyk Hrabia, Katrin Wolf and Mathias Wilhelm

Although Data Gloves allow for the modeling of the human hand, they can lead to a reduction in usability as they cover the entire hand and limit the sense of touch as well as reducing hand feasibility. As modeling the whole hand has many advantages (e.g. for complex gesture detection) we aim for modeling the whole hand while at the same time keeping the hand’s natural degrees of freedom (DOF) and the tactile sensibility as high as possible while allowing for manual tasks like grasping tools and devices. Therefore, we attach motion sensor boards (accelerometer, magnetometer and gyroscope) to the human hand. We conducted a user study and found the biomechanical dependence of the joint angles between the fingertip close joint (DIP) and the palm close joint (PIP) in a relation of DIP = 0.88 PIP for all four fingers (SD=0.10, R²=0.77). This allows the data glove to be reduced by 8 sensors boards, one per finger, three for the thumb, and one on the back of the hand for modeling the whole hand through. Even though we found a joint flexing relationship also for the thumb, we decided to retain 3 sensor units here, as the relationship varied more (R²=0.59). One board is attached at the back of the hand as an orientation baseline. Our hand model could potentially serve for rich hand-model-based gestural interaction as it covers all 26 DOF in the human hand. The figure shows our initial whole hand model based on 16 motion sensor boards mounted on the green and red marked positions. Applying bio-mechanics allows the hand to be modeled using just the 8 green marked sensor positions shown.