VR 360 sound

Hi,

do you know how this "headphone-sensitive" sound of a 360 VR-Video is recorded and post-produced to work?

Thanks

No clue what you're referring to but I often feel I am in a technology backwater. Or I try to ignore stuff I don't need like 8k 360 drones.

I'm going to need more information or a sample or a link to something.

Playback of interactive VR audio requires some interesting programming that pushes four or more tracks to the sound delivery system in response to the position of the headset. this requires the sound to have been recorded in the first place.

Facebook has some software to help with spatial sound: facebook360.fb.com/spatial-workstation/

Also check out this page support.google.com/youtube/answer/6395969?hl=en

360 video is a technology/terminology minefield right now, so don't expect a simple solution. :silly:

My experience with several VR 360 setups (Ricoh & GoPro) is that the in-camera recorded audio is nearly worthless. It's mainly there for syncing purposes to be replaced by separately recorded audio. I know this doesn't help if you are already stuck with that as your only audio source but maybe you're testing for future shoots and could incorporate better quality audio recording for those.

---> Jarrod

MarkNelsonMovies wrote: Also check out this page support.google.com/youtube/answer/6395969?hl=en

360 video is a technology/terminology minefield right now, so don't expect a simple solution. :silly:

Wow, that's some nasty stuff! I herewith quote form the goggles:

Note: Don't use the YouTube video editor since it's not yet supported on 360-degree and VR videos.
take your spherical (360° and virtual reality) videos to the next level so that viewers can immerse themselves in your content. Only one audio track is supported. Multiple audio tracks, such as tracks with spatial and stereo/mono in the same file, are not supported.
You can only use spatial audio for 360-degree and virtual reality (VR) videos. learn all supported layouts and orderings,First Order Ambisonic (FOA) audio channel layouts ump supports MOV files with first-order ambisonics
ACN channel ordering SN3D normalization FOA components W, Y, Z, X should have channel map indexes 0, 1, 2, 3, respectively. The channel_map sequence should be specified as 0, 1, 2, 3.

bobmax48 wrote: Facebook has some software to help with spatial sound: facebook360.fb.com/spatial-workstation/

What exactly does that do? As far as I can tell, it's just a static audio track and you can spin the video. Because when I spin the video embedded on that page and look around, the panning does not change one bit, as it most definitely should. My answer is to just mix a normal stereo audio track.

EDIT: I found some 360 Youtube videos with spatial audio. Neat, but with only two speakers (your headphones) it doesn't seem worth it. The effect is nowhere near as immersive as, say, spinning your character around in a PlayStation 3 or 4 game with 7.1 surround sound.

The idea with the spatial audio stuff is that the sound is, as an editor, you position the sounds around the viewer. A bit like you do with surround sound at the moment where you choose what sound goes to which to speaker, except instead of choosing from say 6 possible positions in the case of 5.1, you can choose from any position. As the listener rotates the video (or the head) an algorithm in the player, calculates how to mix the sound to each air so it sounds like the audio is coming from a specific direction. If you've ever heard the binaural recordings (which you can find on youtube) where the audio appears to come from all around you through headphones, spatial audio is an attempt to achieve the same effect but interactively so the viewer can freely move their head.

In theory spatial audio through headphones should be superior to traditional surround sound because you have complete control over what the listener hears in each ear enabling you to exactly position the audio, rather than with surround where can just estimate a direction. As Redifier points out though, surround is a much more mature technology and probably practically sounds better.

To record for spatial audio ideally you would want to record mono sources in as much isolation an possible so the spatial algorithm will be as realistic as possible.

I don't know of they best workflow of easily achieving spatial audio at the moment. If I was tasked to do a video with spatial audio at the moment my first thought would probably be to cut the 360 video in fcp and then set up the spatial audio in a game engine like Unity. Game engines have been doing spatial audio for a long time and so the workflow fairly well sorted....how you would export that to youtube... don't know.

5.1 surround gaming headsets have been less than successful. The perception of 5.1 surround sound relies on the timing differences between speakers, which requires careful positioning of the speakers and the listener, and not entirely upon the position of the panning controls when mixing.
I don't know anything about this 360 stuff (another technology, like 8k, I personally do not need), but when surround first came out (back them it was called quad), we recorded 4 mono tracks with highly directional mics facing out every 90 degrees, on the compass points, from above the camera. When the camera panned, the four mics panned too. We shot ocnstruction sites and tunes and put the camera on a dolly and went trucking through crowds at the state fair. The playback effect was startlingly realistic and immersive. But insanely complex to mix and post using ye olde Betacam. I was glad to retire that experiment and sell the quad amplifiers.

Yeah, those surround gaming headphones are bizarre. They seem to be the worst of both worlds - the whole point of surround is to simulate directional sound from an environment, but if you have isolated inputs into each ear as you do with stereo headphones.

Betacam eh? Thank god those days are over.

Seamusvfx wrote: Betacam eh? Thank god those days are over.

Still got a deck! Hundreds of legacy reels in the corporate archives.