VR audio laser
Most virtual reality apps rely on visual cues, which make them inaccessible to people with visual impairments. To bring these incredible experiences to visually impaired people, Xavi Benavides Palos and I explored ways to use spatial audio cues for navigating and interacting with virtual environments.
The gear being used is the HTC Vive for room-scale VR along with headphones for spatial audio. The HMD is rendering pure black to simulate being blind.
The goal of the experience we created is for the user to find the toy gun within the virtual room, find open the window and shoot a duck flying outside. The virtual room has a door, window, floor, walls, ceiling, and toy gun on the floor. Every virtual surface and object has an ID with an associated recording of it’s name (eg: “door knob”, “floor”, etc).
To enable navigation in the pitch-black room, the user has a laser extending from the main Vive controller. When the user user clicks the controller's touchpad, it plays a short impulse response tone at location of the controller. Then the tone is played a few more times as it quickly progresses to the location of the virtual surface or object. Because all audio is processed using the Google VR Spatial Audio plugin, each tone provides location information to help gauge the distance and relative location of the object in the virtual space.
To test our prototype, we ran six non-visually-impaired users through the prototype, and all of them were able to complete the challenge successfully. After the task was completed, four of them went through the experience again, this time able to see the room without vision impairment. Because they had navigated the room by sound, we found that they were already familiar with their surroundings.
It’s a small step, but this experiment demonstrated that it’s possible to navigate and interact with a room in VR using only auditory cues. We hope others continue to explore ways to make VR accessible for everyone. There’s much more to do in this area!
Find more details in the technical disclosure.
Daydream Labs blog post