Clubhouse Adds Spatial Audio for Immersive Experience

Clubhouse Enhances Audio Experience with Spatial Audio Technology

Clubhouse, the popular social audio application, has been actively developing its platform this summer. Following the introduction of new messaging capabilities and an Android version, the company is now focused on refining its fundamental audio functionality.

The platform announced on Sunday the integration of spatial audio into its rooms. This enhancement aims to provide listeners with a more immersive and realistic experience, simulating the feeling of being present with a group in a live setting.

Insights from Justin Uberti

Justin Uberti, Clubhouse’s head of streaming technology, discussed the rationale behind incorporating spatial audio. He explained that the technology creates the perception of speakers originating from distinct physical locations, rather than a single point.

Uberti joined Clubhouse in May, bringing with him over a decade of experience from Google. During his time at Google, he was instrumental in the creation of Google Duo, led the Hangouts team, and contributed to the development of the Stadia cloud gaming platform.

Notably, Uberti also developed the WebRTC standard, which serves as the foundational technology for Clubhouse.

The Challenge of Virtual Audio

“In group audio environments, it’s often difficult to replicate the natural experience of being in a physical space,” Uberti stated.

Traditional voice chat applications, including Clubhouse, typically deliver audio that sounds relatively flat, as if emanating from a central source. However, in real-world gatherings, sound originates from various points throughout the room, including different positions on a stage and from audience members.

Implementation Details

To achieve this spatial audio effect, Clubhouse is utilizing licensed code from High Fidelity, a spatial audio company founded by Philip Rosedale, the creator of Second Life.

This code is being combined with the company’s own proprietary audio processing algorithms, specifically tuned for the Clubhouse application.

HRTF Technology Explained

High Fidelity’s HRTF (Head Related Transfer Function) technology maps speech to different virtual locations. It accomplishes this by introducing subtle time delays between stereo channels and replicating how frequencies are perceived as they enter the ear, depending on the sound’s origin.

This technology, commonly used in social VR, creates a sense of physical presence. It’s akin to listening to a stereo recording, such as Pink Floyd’s “Dark Side of the Moon,” with high-quality headphones, but instead of instruments, you perceive the voices of people positioned around you in a virtual space.

Subtle Yet Noticeable Impact

According to Uberti, the implementation of spatial audio on Clubhouse will be subtle, yet perceptible. The audio processing will “gently steer conversation,” positioning most speakers in front of the listener.

Users should experience a heightened sense that individuals are speaking from different locations within the virtual room.

Rollout and Accessibility

The new audio features began rolling out to the majority of iOS users on Sunday. The update will reach the remaining iOS and Android users over the coming weeks.

While the experience will eventually be available to all users, individuals will also have the option to disable spatial audio.

Adapting to Room Size

Clubhouse will employ the same virtual soundstage techniques to create the impression of larger rooms for sizable gatherings, while simultaneously making smaller rooms feel more intimate and contained.

The majority of Clubhouse users utilize headphones, allowing them to benefit from the effects achievable through two-channel stereo sound.

“We aim to mimic the feeling of being in a space, in a room, with people standing around talking,” Uberti explained.

Potential Benefits Beyond Immersion

Uberti also suggested that spatial audio could offer a less obvious advantage to Clubhouse users. He posited that conventional, non-spatialized audio in social applications may contribute to the phenomenon of Zoom fatigue.

The human brain processes virtual audio differently than it does in natural, in-person interactions.

“The mind must actively identify who is speaking. Without spatial cues, it relies on timbre, which requires more cognitive effort,” Uberti said. “This could potentially lead to a more enjoyable experience, in addition to increased immersion.”

Enhancing Various Experiences

It remains to be seen how Clubhouse’s diverse communities will respond to the spatial audio effects. However, it has the potential to significantly enhance experiences such as comedy performances, musical events, and even ASMR content on the platform.

“A joke often feels flat,” Uberti noted. “But on Clubhouse, when you feel the laughter coming from all around you, it more closely resembles the experience of being at a comedy club.”