Interaction via Google Cardboard

While devices like the Oculus Rift and HTC Vive generate a lot of the headlines in the world of VR, for most users, lower-cost and lower-fidelity devices like the Google Cardboard and Samsung GearVR are the more typical form of interaction.

The advanced headsets have many modes of interaction; at a minimum, they support gaming controllers or hand controllers, and many of them accurately track movement within a room via infrared technology.  When users are viewing content via Google Cardboard, the options for interaction are much more limited.

Cardboard provides a few methods for getting input from your user, depending on how creative you’re willing to be.  First off, all Cardboard units have a single button.  This button translates as a “tap” on the screen.  You can’t use it to track a specific touch location on the screen, but combined with gaze detection (figuring out what the user is looking at) you can build simple interactivity.  The “gaze – tap” interaction holds some nice potential as the basic “bread and butter” interaction for Google Cardboard.

We’ve been exploring this as a way to do simple interactive walkthroughs. In addition to the obvious “pick the direction you want to go” options, we believe additional controls can often be “hidden” at the top and bottom of the view sphere.  For example, your users may be able to look down to trigger a menu, or look up to go to a map.

Google Cardboard also gives you access to the various sensors available within a smartphone, like the accelerometer and gyroscopes.  These are useful, first and foremost, for tracking the movement of the headset itself, but these could also potentially be used for gesture control.  For example, you could watch for sudden impacts – such as tapping the side of the cardboard, or having the user jump up and down – to trigger certain interactions.

If you’re building a local iOS or Android app (as opposed to a web application) you’ll also have access to the device’s microphone.  If that’s the case, speech recognition could also provide a lot of interesting flexibility.  For example, even basic detection of loud noises can allow for start/stop controls.

While the Cardboard technology is limiting in many ways, the limitations can actually be exciting, because they spur you to think of new ways to leverage the technology.  We’re excited about what’s possible, and we’re excited to hear from others!

Leave a Reply