× Projects Active and Future Projects Resume Useful Tools

Roll And Pinch Keyboard

A full keyboard in VR, designed for VR.

The roll at pinch keyboard is based on the rapMenu introduced at IEEE 3DUI in 2008 by Ni et. al, which explored the combition of wrist rotation and touching various fingers to the thumb to create a menu system for VR. The Roll and Pinch Keyboard expands this idea to explore using the Roll and Pinch Menu for typing, by using both hands and using each menu option as a key.

Pinching each of the four non-thumb fingers to the thumb provides 8 "keys" which provide passive haptic feedback (you can feel when you touch your own thumb) and can be found without visual representation using proprioceptive senses (you know where your own fingers are, even when you aren't looking at them).

If both hands are involved, we have 5 positions (touching one of the four non-thumb fingers, or touching none of them) per hand. This gives us 25 (maybe check this math) possible combinations of these 5 positions. This is enough to type all letters of the alphabet, with one left over as a caps lock equivalent, making it possible to type all letters of the English language, but not punctuation, numbers, special characters, or control characters.

By recognizing each of the 5 possible hand positions at a certain number of different wrist rotation positions, we can greatly expand the number of possible "keys". More wrist rotation positions of course are harder for a user to consistently repeat, and harder to recognize accurately. However, with just 2 wrist positions recognized per hand, we expand to (check math here too) 100 possible "keys", and with 3 we expand to (very check this math) 225, which seems like enough.

Given that typing speed is easy to measure, and the methods for evaluating the usability of keyboards are well-established, the possible layout variations are a rich area for potential HCI research.

As hand-tracking technology improves, it would be fascinating to attempt to remove the rolling aspect of the Roll and Pinch Keyboard, and instead increase the number of potential "keys" by recognizing touching combinations of fingers, such as both the index and middle fingers, to the thumb. This could create a typing method for virtual reality which is significantly more comfortable, rapid, and accurate than ray-casting systems which are currently commonplace.

By including "index and middle", "middle and ring", and "ring and pinky" (uncomfortable, but Q needs to go somewhere) to our set of 5 base hand positions expands to 8 per hand, or (seriously, check this math) 64 posssible keys without rotation.

Including both these multi-finger positions and rotation opens up the possibility for a truly impractical number of "keys". With 2 rotational positions and multi-finger keys, (CHECK THE MATH ON THIS) 256 keys are possible, and with 3 wrist rotation positions we can recognize 576 unique "keys" without a single physical button. While many of these "keys" are unlikely to be consistently recognized by current hand tracking technology, presumably the subset of 576 keys which can be recognized by current handtracking technology is enough for a passable keyboard.