× Projects Active and Future Projects

An Integrated System for Immersive Interaction

A grammar for how we engage with virtual reality.

This is work which I intend to focus on for my masters thesis, which means that it sould be completed in the spring of 2021.

I'm interested in creating a grammar for interacting with virtual reality in an accessible, thought-through manner which works with the three-dimensionality of the medium, rather than trying to manage 3d spaces with fundamentally 2d tools.

I find that many of our present interaction techniques for virtual reality tend to port methods from 2d technologies, and do not make the best possible use of VR as a medium. For example, a common menu interface in VR applications at the moment involves pointing a raycast at a 2d menu-- a technique which more or less works exactly like pointing a remote at a television.

I find that many of our present interaction techniques for virtual reality are insufficiently abstract for other techniques to be built on top of them. For example, a common design for buttonless mid-air interaction involves gestures. Gestures are not modular. With a purely gesture-based system, for each new possible unit of information the user may want to communicate to the system, they need to learn and remember another new gesture. Thus, once you have a gesture for opening a menu, a gesture for selecting, a gesture for rotating something, a gesture for resizing something, and maybe one to delete, you've started to run out of possible hand shapes. If a future developer wants to do something which you haven't anticipated, like allow a user to measure the distance between two points, it may be difficult for them to engage with your existing gesture system.

There are four basic things which I would like to be able to do in virtual reality. They are:

  1. Select things and move them around
  2. Carry objects and use abstract tools (menus)
  3. Move between spaces (for example, where you are and the beach)
  4. Move within a given space (for example, between where you are and the other side of the room)

Bird is my proposed solution to the first of these four problems.